Skip to content

Cross-grammar sync tests for the refs grammars#9

Merged
gandarfh merged 3 commits into
mainfrom
feat/refs-cross-grammar-sync
Jun 9, 2026
Merged

Cross-grammar sync tests for the refs grammars#9
gandarfh merged 3 commits into
mainfrom
feat/refs-cross-grammar-sync

Conversation

@gandarfh

@gandarfh gandarfh commented Jun 9, 2026

Copy link
Copy Markdown
Member

What

Shared-corpus sync tests asserting the Lezer and tree-sitter refs grammars agree on the same inputs, plus a CI job that runs them on every PR.

Design

  • 11 corpus cases covering aliases, dot paths, array indexing, env vars, surrounding text, and two error-recovery cases marked .diverges. where engines may legitimately differ.
  • Both engines are normalized to one canonical node vocabulary (normalize.ts); the tree-sitter side shells out to the CLI so no native bindings or WASM build are needed.
  • The assertion encodes the sync contract: the Lezer stream must be a coarsening of the tree-sitter stream (every Lezer node exists there with the same span and kind — Lezer may emit less structure, never different structure), and ref anchors must match exactly in both directions so a degenerate empty Lezer parse cannot pass.
  • Contiguous text nodes are coalesced before comparison: the engines chunk plain-text runs differently and the segmentation carries no rendering meaning.

Fixes found while wiring it up

  • tree-sitter parse --quiet suppresses the tree output on CLI 0.26, so the tree-sitter side returned nothing; the flag is gone.
  • The CLI exits non-zero when the tree contains ERROR/MISSING nodes (exactly the error-recovery corpus); the runner now keeps the tree printed to stdout instead of discarding it.

Validation

11/11 locally against tree-sitter CLI 0.26.8; the new cross-grammar CI job builds the Lezer package and runs the suite on ubuntu.

@gandarfh
gandarfh merged commit 4836dd9 into main Jun 9, 2026
7 checks passed
@gandarfh
gandarfh deleted the feat/refs-cross-grammar-sync branch June 9, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant