lexer

package
v0.3.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 4, 2023 License: MIT Imports: 7 Imported by: 2

Documentation

Overview

Package lexer provides basic helpers to implement parsers

Index

Constants

View Source 
const (
	// ReadBufferSize indicates the initial buffer size
	ReadBufferSize = 1 << 7 // 128B

	// DoublingBufferSizeLimit indicates when we stop doubling
	// and just add instead
	DoublingBufferSizeLimit = 1 << 17 // 128KiB
)

Variables

View Source 
var (
	// ErrUnacceptableRune indicates the read rune isn't acceptable in the context
	ErrUnacceptableRune = errors.New("rune not acceptable in context")

	// ErrNotImplemented indicates something hasn't been implemented yet
	ErrNotImplemented = errors.New("not implemented")
)
View Source 
var (
	// ErrInvalidUnreadRune indicates UnreadRune() was calls after an
	// action other than a successful ReadRune()
	ErrInvalidUnreadRune = errors.New("invalid UnreadRune() call")
)

Functions

func IsSpace added in v0.3.6 

func IsSpace(r rune) bool

IsSpace reports whether the rune is a space character as defined by Unicode's White Space property

func NewIsIn added in v0.3.6 

func NewIsIn(s string) func(rune) bool

NewIsIn generates a rune condition checker that accepts runes contained on the provided string

func NewIsInRunes added in v0.3.7 

func NewIsInRunes(s ...rune) func(rune) bool

NewIsInRunes generates a rune condition checker that accepts the runes specified

func NewIsNot added in v0.3.6 

func NewIsNot(cond func(rune) bool) func(rune) bool

NewIsNot generates a rune condition checker that reverses the decision of the given checker.

func NewIsOneOf added in v0.3.6 

func NewIsOneOf(s ...func(rune) bool) func(rune) bool

NewIsOneOf generates a run condition checker that accepts runes accepted by any of the given checkers

func Run added in v0.3.5 

func Run(fn StateFn) error

Run runs a state machine until the state function either returns nil or an error

Types

type Error added in v0.3.3 

type Error struct {
	Filename string
	Line     int
	Column   int

	Content string
	Hint    string
	Err     error
}

Error represents a generic parsing error

func (Error) Error added in v0.3.3 

func (err Error) Error() string

func (Error) Unwrap added in v0.3.3 

func (err Error) Unwrap() error

type Position added in v0.3.4 

type Position struct {
	Line   int
	Column int
}

Position indicates a line and column pair on a file. Counting starts at 1.

func (*Position) Add added in v0.3.9 

func (p *Position) Add(rel Position)

Add adds a relative position considering potential new lines

func (Position) GoString added in v0.3.4 

func (p Position) GoString() string

GoString generates a string representation of the Position for %#v usage

func (*Position) Reset added in v0.3.4 

func (p *Position) Reset()

Reset places a position at (1,1)

func (*Position) Step added in v0.3.4 

func (p *Position) Step()

Step moves the column one place

func (*Position) StepLine added in v0.3.4 

func (p *Position) StepLine()

StepLine moves position to the start of the next line

func (*Position) StepN added in v0.3.4 

func (p *Position) StepN(n int)

StepN moves the column N places forward

func (Position) String added in v0.3.4 

func (p Position) String() string

String generates a pretty "(Line, Column)"" representation of the Position

type Reader added in v0.3.1 

type Reader struct {
	// contains filtered or unexported fields
}

Reader is a RuneReader aimed at implementing text parsers

func NewReader added in v0.3.1 

func NewReader(r io.Reader) *Reader

NewReader creates a new runes Reader using the given io.Reader

func NewReaderBytes added in v0.3.1 

func NewReaderBytes(b []byte) *Reader

NewReaderBytes creates a new runes Reader using the given bytes

func NewReaderString added in v0.3.1 

func NewReaderString(s string) *Reader

NewReaderString creates a new runes Reader using the given string

func (*Reader) Accept added in v0.3.5 

func (b *Reader) Accept(cond func(r rune) bool) bool

Accept consumes a rune from the source if it meets the condition. it returns true if the condition was met and false if it wasn't.

func (*Reader) AcceptAll added in v0.3.5 

func (b *Reader) AcceptAll(cond func(r rune) bool) bool

AcceptAll consumes runes from the source as long as they meet the condition. it returns true if the condition was met for at least one rune, and false if it wasn't.

func (*Reader) Discard added in v0.3.1 

func (b *Reader) Discard()

Discard removes from the buffer everything that has been Read

func (*Reader) Emit added in v0.3.1 

func (b *Reader) Emit() string

Emit returns what's already being Read and discards it afterwards

func (*Reader) PeekRune added in v0.3.1 

func (b *Reader) PeekRune() (rune, int, error)

PeekRune returns information about the next rune without moving the cursor

func (*Reader) ReadRune added in v0.3.1 

func (b *Reader) ReadRune() (rune, int, error)

ReadRune reads the next rune

func (*Reader) String added in v0.3.1 

func (b *Reader) String() string

String returns what's already Read but not yet emitted or discarded

func (*Reader) UnreadRune added in v0.3.1 

func (b *Reader) UnreadRune() error

UnreadRune moves the cursor where it was before the last call to ReadRune

type StateFn 

type StateFn func() (StateFn, error)

StateFn is a State Function of the parser

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.