Lesson 8 of 10

Shipping a Parser

A grammar becomes useful when other code can call it. There are three ways to ship one: compile the grammar in your program at start-up, which is what the previous lessons did; generate Go source, so that the program needs neither the library nor the grammar; or save a compiled grammar and load it. This lesson shows the last two and the choice of a backend.

#What You Will Build

A config Go package generated from the configuration grammar, and a compiled grammar file that the command line accepts in place of the source.

#Before You Start

Complete Building Typed Trees: you need config.pego and a Go module (go mod init example.com/hello).

#Generating a Standalone Go Parser

pego gen turns a grammar into Go source, so your program does not need the PEGO library or the grammar at run time. The generated package (parser.go, api.go and a copy of the runtime in internal/runtime) uses only the standard library. -module is the import path the package will have, so it starts with your module path. Generate it into a config package:

pego gen -g config.pego -module example.com/hello/config -o config

and call it:

package main

import (
	"fmt"
	"log"

	"example.com/hello/config" // your module path
)

func main() {
	node, err := config.Parse("port = 8080\n")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(node)
}

Config{Entries: [Entry{Key: Key("port"), Value: Number("8080")}]}

The generated Node type mirrors pego.Node, and the generated parser gives the same trees and errors as the library. config.ParseRule(name, input) starts at another rule. The generated entry points accept functional options: config.Parse(input, config.WithUnit(config.Bytes), config.WithMaxDepth(600_000)). pego gen -max-depth 600000 sets the generated default; omitted or zero keeps 100,000.

#Backends and Compiled Grammars

A grammar can run on several backends with identical results: the default closure-compiled engine, a bytecode VM, and a bytecode VM that uses an explicit stack instead of the Go stack for deeply nested input. Choose one on the command line or in Go:

pego parse -g config.pego -f sexpr -backend bytecode < example.conf
pego parse -g config.pego -check -backend bytecode-iterative < example.conf   # validate only, prints ok

n, err := p.Parse("abc=12", pego.WithBackend(pego.Bytecode)) // also pego.WithUnit(pego.Bytes), pego.RecognizeOnly()

pego compile -g config.pego -o config.pegoc saves a compiled grammar that pego parse accepts in place of the .pego file (pego parse -g config.pegoc ...), and pego.LoadParser loads it without recompiling. The file serves the parses of one mode: -mode parse (the default) for parsing, -mode recognize for -check, and -mode document for documents and traces. pego convert -to json config.pego converts a grammar to JSON, which is handy for tools that generate grammars. The Running Parsers guide describes the backends and options in detail, and the Compiled Grammars guide the .pegoc format.

#Next Steps

Handling Big and Live Input covers input that does not fit in memory and text that is edited while parsed. The guides Generating Parsers, Generated TypeScript Parsers, Compiled Grammars and Using the Ready-Made Parsers go further, and the cookbook recipe Ship a Parser and Keep It Up to Date shows how to keep generated code in step with the grammar.