feat(ext): add tree-sitter grammar and refresh TextMate grammar 🌳 - #165
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a47a9d2968
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Named op-assign like `x multiply= 10` is valid Andy (the lexer glues an identifier to a single `=` not followed by `=`), but the grammar only accepted symbolic compound operators, so it silently mis-parsed the construct as two statements. Add an external scanner that recognises the glued identifier-equals token with the one char of lookahead the lexer uses, so `a==b` equality is unaffected. Reported by Codex on #165. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d726d59d50
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The locals query only treated a direct `identifier` name/pattern as a definition, so identifiers inside `let [a, b] = …`, `let (a, b) = …`, `let a, b = …` and nested for-patterns like `for (x, y), [a, b, c] in …` fell through to `@local.reference`, losing definition highlighting and go-to-definition. Descend through the list/tuple/pattern_sequence containers (up to two levels) for let, for, and destructured parameters. Reported by Codex on #165. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e0425ff94
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A list comprehension may yield a bare comma tuple, e.g. `[x, y for x in 1..10, y in 1..10]`. The grammar only accepted a single expression before the comprehension clauses, so the comma forced the list-literal path and the following `for` became an ERROR. Accept `_expression_or_sequence` as the body, mirroring how the parser parses a tuple before checking for `for`. Reported by Codex on #165. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc27fc676e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex flagged a possible issue where the external scanner advancing past `and`/`or`/`in` before returning false could drop the keyword. In practice tree-sitter resets the lexer to the pre-scan position on a false return, so `a and b not in c` parses correctly. Add a corpus test that locks this in. No scanner change needed. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7a123ed6f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Raw strings use a `#` fence specifically to embed quotes, e.g. `r#"with "" quotes"#` and `r###"... r#"x"# ..."###` from the manual. The regex token stopped at the first inner `"`, breaking the rest of the literal. Recognise raw strings in the external scanner, which counts the opening `#` run and scans until a `"` followed by exactly that many `#`. The leading `r` is disambiguated from identifiers (and named op-assign) by the character after it. Reported by Codex on #165. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de583e9c02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
## Context Follow-up to #165. Setting up the grammar in Helix surfaced two problems (Neovim was unaffected): 1. **No highlighting in Helix.** Helix compiles `highlights` + `injections` + `locals` into one tree-sitter query, so the Neovim-only `#lua-match?` predicate in `injections.scm` failed the *entire* highlight compile (`unknown predicate #lua-match?`). The LSP still worked, which is why it looked like only highlighting was broken. 2. **Broken Helix setup docs.** The README's `[[grammar]]` git source omitted `rev`, producing `data did not match any variant of untagged enum GrammarSource`. ## Changes - **`injections.scm`**: `#lua-match?` → `#match?`. Both Neovim and Helix support `#match?`, and `^#!` matches identically under each editor's regex. Verified the combined query now compiles via the `tree-sitter highlight` CLI (same `tree-sitter-highlight` crate Helix uses), and that Neovim still loads the query. - **README Helix section**: git sources now include `rev` (with a local-path alternative), and the build step uses `tree-sitter build -o …/grammars/andy-cpp.so` instead of `hx --grammar build` (which rebuilds every grammar and needs the output dir to pre-exist). Added an `hx --health` check and an `hx`-vs-`helix` binary note. Capture ordering was checked and left as-is: Helix's own bundled queries place the catch-all `(identifier) @variable` before the specific `@function`/`@variable.parameter` captures, confirming Helix resolves overlaps last-match-wins like Neovim, so the existing ordering is correct for both. 🤖 Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 participant
Context
Editors like Neovim, Helix, and Zed highlight via tree-sitter, not TextMate — the existing VS Code extension only ships a TextMate grammar, so those editors had no support. While adding tree-sitter, the TextMate grammar also turned out to predate the static-type work and had a couple of regex bugs.
Changes
New
ext/tree-sitter-andy-cpp/packagegrammar.js) with a precedence ladder mirroringndc_lexer/ndc_parser, plus committed generatedsrc/so consumers build without the CLI.test/corpus.ndc lspwiring..ndcprogram parses; only the deliberate// expect-error:cases fail.TextMate grammar refresh (
ext/andy-cpp)->return arrow, and:separators (the whole typecheck feature was previously invisible).fn …block had no inner patterns).continueandNaN.^=was mis-detected (\|\^) and\=was missing.[Unreleased].Notes for reviewers
src/parser.c; regenerate withtree-sitter generateafter editinggrammar.js.List<List<Int>>, named augmented assignmentacc max= x). None occur in the current corpus.🤖