PEGO Cookbook
Short recipes for concrete problems. Each one is a complete solution: the problem in a sentence or two, the grammar and the Go program, the output they print, and a few lines on how it works and what to change. They differ from the other documents in how they are organized:
- The tutorial teaches the language from the first rule on.
- The guides explain one feature at a time, in depth.
- The cookbook starts from a task (read a configuration file, report three errors at once, reparse on every keystroke) and combines the features that the task needs.
Running a Recipe
Every program is a main package that you can run as it is. Put the files in a directory (a block whose first line is
// name.go or // name.pego is a file of that name, without that line), make it a module that uses PEGO, and run it:
mkdir recipe && cd recipe
go mod init example.com/recipe
go get github.com/ornew/pego
go run .
The recipes need Go 1.27 or later and nothing else, except where a recipe says otherwise (the ones that generate code
install the pego tool with go get -tool, and the one about ready-made parsers adds a module). The grammars are
embedded in the programs with go:embed, so the .pego files are real files that pego fmt, pego parse and the
editor support work with. Every output was produced by running the code shown, with Go 1.27.1 on an Apple M3 Max;
times differ from machine to machine.
Read Data
| Recipe | What it shows |
|---|---|
| Read a configuration file into Go structs | A grammar for key = value lines with typed values, and Go code that fills a struct and reports the line of a wrong value |
| Parse log lines fast | A generated parser with typed values (ParseAST) and Recognize; what each costs on 100,000 lines |
| Process a file that does not fit in memory | #stream and ParseStream: records one at a time, in 17 MiB for 5 million lines |
| Sum a column of a big CSV file | The CSV grammar of parsers/csv streamed record by record: a total in about 13 MB, and the offset of the first bad record |
| Read a header and then records | A stream whose first element is a header: telling the kinds of element apart in the callback |
| Stop a stream when the caller gives up | WithContext plus a reader that fails from outside, for a producer that goes quiet |
| Parse indentation-based input | Nested entries as nested Go maps, with a variable for the indentation and an error for inconsistent indentation |
| Parse an expression embedded in text | ParsePrefix: a {{ expression }} template renderer that parses the fragment where it starts and learns where it ends |
Evaluate and Query
| Recipe | What it shows |
|---|---|
| Evaluate arithmetic with variables and functions | An Operators Rule with precedence, associativity and calls, and an evaluator that runs the same tree for many inputs |
| Compile a filter language into a Go predicate | age > 30 and name ~ "a*" as closures, with word operators that do not collide with field names |
| Find nodes in a tree and at a position | Walk for every call with its span and depth, and a Cursor for the innermost node at an offset |
Report Problems
| Recipe | What it shows |
|---|---|
| Report an error with its source line and a caret | @error messages and the position of a *pego.SyntaxError, printed like a compiler |
| Report several errors in one run | #recover: a tree of what could be read, the errors, and what to watch for in the skip |
| Keep going after a bad statement in nested blocks | Recovery at the statement rule shared by every block: errors from all levels and the statements that were fine |
| Find out why a grammar is slow | pego profile on a 20,000-line input: which rule wastes input, the change that fixes it, and the profile that proves it |
Build Tools
| Recipe | What it shows |
|---|---|
| Tokenize source for syntax highlighting | A tokenizer that cannot fail, and HTML spans from the flat tree |
| Reparse a document on every keystroke | Document, Edit and Stats; a grammar shaped so that edits reuse most of the last parse |
| Keep the last good tree while the user types | An Editor around a Document that shows the last tree that parsed while a line is half typed, and what each keystroke costs |
Ship, Reuse and Test
| Recipe | What it shows |
|---|---|
| Embed a grammar in a program and share the parser | go:embed, one compiled parser used from many goroutines with WithStart, and a -race test |
| Ship a parser and keep it up to date | pego gen with go generate, a test that fails when the generated file is stale, and .pegoc for grammars that stay data |
| Validate a YAML file and point at the problem | A ready-made parser of parsers/: syntax errors with positions, and your own checks with the positions of the typed tree |
| Test hand-written code against a grammar | Inputs and near misses generated from the grammar, a differential test and a fuzz test |
| Fuzz the edits of a grammar in your tests | editfuzz in go test: random edits, undos and redos checked against fresh parses, saved failures as regression tests, and a case written by hand |
| Check a file in a TypeScript tool with the same grammar | pego gen -lang ts -types: a dependency-free module, typed values, recovered syntax errors and a check of the tool's own |
| Check grammars in continuous integration | One script for pego fmt -check, pego lint -strict and pego test, and a workflow that runs it |
Where to Go Next
| To learn about | Read |
|---|---|
| One feature in depth | Guides |
| The exact rules of a construct | Language specification |
| Complete grammars with tests | examples/ and parsers/ |