Check Grammars in Continuous Integration

Problem. A grammar is code that other code depends on, and its mistakes show up late as a wrong tree for some input. You want every change to be checked for formatting, lint findings and the fixtures, and the build to fail when one of them breaks.

Run three commands, each of which exits with a nonzero status when it finds something: pego fmt -check, pego lint -strict and pego test. Put them in one script that CI and developers both run.

Take a repository with grammars in grammars/ and one fixture suite per grammar next to it:

example.com/project
├── go.mod
└── grammars/
    ├── config.pego
    ├── config.suite.json     the fixtures of config.pego
    └── check.sh

Record pego as a tool of the module, so that the version is pinned and nothing has to be installed:

go get -tool github.com/ornew/pego/cmd/pego

The script runs the checks on every grammar and reports all failures, not only the first:

#!/bin/sh
# grammars/check.sh
status=0
for g in grammars/*.pego; do
    go tool pego fmt -check "$g" || status=1
    go tool pego lint -strict -g "$g" || status=1
done
for s in grammars/*.suite.json; do
    go tool pego test -f "$s" || status=1
done
exit $status

Run it from the root of the repository, in CI as one step:

# .github/workflows/grammars.yml
name: grammars
on: [push, pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: "1.27.1"
      - run: sh grammars/check.sh

#Expected Output

For a clean grammar the script prints only what pego test prints, and the exit status is 0. pego test prints one PASS line for each case on each backend, one for each backend equivalence check, and a summary. For a suite of one case on the closure backend:

PASS empty file [closure]
all fixtures passed

Lint with a mistake in the grammar (an alternative that cannot match, and a rule that nothing calls) fails the build, and the findings say where and how to fix them:

grammars/config.pego:2:17: error: alternative 2 (`"<="`) can never match: alternative 1 (`"<"` at 2:11) matches first wherever it could [shadowed-alternative]
	fix: move it before alternative 1
grammars/config.pego:4:1: warning: rule unused is never used: main does not call it, directly or indirectly [unreachable-rule]
	fix: remove the rule, or call it from a rule that main reaches
pego: lint: 1 error, 1 warning, 0 hints

A file that is not formatted is listed, and the command tells how to fix it:

grammars/config.pego
pego: fmt: 1 file not formatted; run pego fmt -w

#How It Works

• pego fmt -check lists the files whose formatting differs from the canonical layout and exits with status 1. It never changes a file, so it is safe in CI; developers run pego fmt -w to fix.
• pego lint exits with status 1 for an error. -strict also fails for a warning, which keeps the grammar free of unused rules. Hints never fail.
• pego test runs every case of the suite on the backends it names and exits with status 1 if one fails. With "equivalence": true it also fails when two backends disagree.
• The loops do not stop at the first failure, so one run shows everything that needs fixing.

#Variations and Limits

• To read the findings in another tool, use pego lint -f json: the findings are one JSON document on standard output (the summary line goes to standard error).
• To keep a deliberate exception, suppress the finding with a comment in the grammar rather than turning the check off for the whole grammar; see Linting Grammars.
• When generated parsers are committed, add a step that regenerates them and fails on a difference; see Ship a Parser and Keep It Up to Date.
• The commands are described in Testing Grammars, Linting Grammars and The pego Command.