Skip to content

Add dodder.nvim: tree-sitter grammars and neovim plugin#364

Closed
friedenberg wants to merge 6 commits into
masterfrom
claude/neovim-treesitter-dodder-tixugd
Closed

Add dodder.nvim: tree-sitter grammars and neovim plugin#364
friedenberg wants to merge 6 commits into
masterfrom
claude/neovim-treesitter-dodder-tixugd

Conversation

@friedenberg

Copy link
Copy Markdown
Collaborator

Introduces zz-nvim, a complete Neovim plugin replacing zz-vim's regex-based syntax highlighting with tree-sitter grammars and adding per-object-type body-language injection.

Summary

This adds three new tree-sitter grammars for dodder's domain-specific languages:

  • hyphence: metadata block format for dodder objects (zettel, tag, type, etc.)
  • doddish: dodder's query language
  • dodder_organize: organize-text format for bulk object operations

The plugin provides:

  • Real tree-sitter parsing and semantic highlighting (replacing zz-vim's regex approach)
  • Nested language injection: object bodies are highlighted in their declared language (markdown, toml, etc.)
  • Lua ftplugin porting behavioral features from zz-vim (gf navigation, object menus, organize folding)
  • Health checks and async body-language resolution via the dodder binary

Key Changes

Grammar definitions (zz-nvim/grammars/*/grammar.js):

  • Hyphence: metadata block with description, tags, type, blob references, and comments
  • Doddish: query language with terms, groups, negation, predicates (exact, type filter, dependent tags), sigils, and genres
  • Organize: headings, object lines with box format (compact one-line representation), and descriptions

Generated parser code (zz-nvim/grammars/*/src/parser.c, node-types.json, grammar.json):

  • Complete tree-sitter parser implementations for all three grammars
  • Symbol tables, metadata, and parse state machines
  • Node type definitions for query and highlighting integration

Lua plugin (zz-nvim/lua/dodder/):

  • init.lua: main entry point with setup/teardown
  • ftplugin.lua: buffer-local options, gf navigation, object/organize menus
  • injection.lua: async body-language resolution from object type metadata
  • filetype.lua: filetype detection for dodder objects
  • health.lua: :checkhealth dodder verification

Highlighting and injection queries (zz-nvim/queries/*/):

  • Semantic highlighting preserving zz-vim's color contracts
  • Language injection directives for nested body parsing

Nix integration (go/default.nix):

  • dodder-nvim package building the three grammars via tree-sitter.buildGrammar
  • Compiled .so parsers included in the plugin distribution

Test corpus (zz-nvim/grammars/*/test/corpus/):

  • Comprehensive test cases for each grammar's syntax

The plugin is designed to coexist with zz-vim; users choose one per their editor preference.

https://claude.ai/code/session_01KUwx95KwKfaw5HdjkWsCgR

claude and others added 5 commits July 13, 2026 00:31
Scaffold the zz-nvim tree-sitter plugin with the first two grammars that
replace zz-vim's regex syntax highlighting:

- hyphence: the object edit format (--- fenced metadata + injectable body).
  Shared rule modules (common/markl.js, common/metadata.js) parse markl-ids
  and metadata lines; the (body) node is left opaque for per-type language
  injection. highlights.scm preserves zz-vim's semantic contract; injections.scm
  delegates body-language resolution to a custom lua directive.
- doddish: the query language (predicate[sigil][genre], groups, negation).

Both grammars have committed generated parsers and passing tree-sitter corpus
tests (7 hyphence, 11 doddish).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KUwx95KwKfaw5HdjkWsCgR
- Add the dodder_organize grammar (hyphence header + # headings + -/% box
  object lines), reusing common/metadata.js and new common/box.js. Passing
  corpus tests and a highlights.scm covering box interiors and headings.
- Refactor metadata so it ends exactly at the closing "---": the fence
  newline, blank line, and body belong to the consumer. Fixes a newline
  shift/reduce conflict in organize and keeps hyphence body injection clean
  (body now requires \n\n so a metadata-only file yields no body node).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KUwx95KwKfaw5HdjkWsCgR
- filetype.lua: vim.filetype.add for object/workspace filetypes (replaces ftdetect)
- injection.lua: custom #dodder-injection-language! directive + async resolver
  (dodder show -format type.vim-syntax-type) with a static fallback table;
  degrades to markdown when dodder is off PATH; nvim<0.10 jobstart fallback.
- init.lua: setup() wiring language.register + FileType/BufWritePost autocmds.
- ftplugin.lua: ports equalprg, gf zettel navigation, action/copy/preview menus
  (vim.ui.select), organize # folding, and commentstrings from zz-vim.
- health.lua: :checkhealth dodder. plugin/dodder.lua: auto-setup.

Verified end-to-end in neovim: metadata highlighting, per-type body injection
(nested grammar), doddish/organize highlighting, ftplugin options, and health.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KUwx95KwKfaw5HdjkWsCgR
Add the dodder-nvim flake package parallel to dodder-vim: compile each grammar
with pkgs.tree-sitter.buildGrammar from its committed parser.c (generate =
false), then stage the plugin via buildVimPlugin with the parsers copied into
$out/parser/<lang>.so so neovim's built-in vim.treesitter auto-loads them off
the runtimepath (no nvim-treesitter dependency). Surfaced through the flake's
forwarded packages set and the non-flake import path. dodder-vim is retained
for legacy Vim users.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KUwx95KwKfaw5HdjkWsCgR
Simplify pass:
- Dedup the bin() resolution logic (was reimplemented separately in
  ftplugin.lua, health.lua, injection.lua) into a single dodder.bin()
  exported from init.lua.
- Extract the sepBy1 grammar helper (duplicated verbatim in doddish and
  organize grammar.js) into grammars/common/util.js.
- Drop injection.lua's type_to_lang table -- every entry mapped to
  'markdown', which is already the function's default fallback.
- Dedup the qlmanage/tacky "requires external tool" guard in
  ftplugin.lua into a single require_executable() helper.
- Derive go/default.nix's dodder-nvim postInstall cp lines from the
  dodder-nvim-grammars attrset instead of hardcoding three copies of
  the same three names.
- Add nodejs + tree-sitter to the devShell so `tree-sitter generate`/
  `test` actually work for the zz-nvim grammars (was silently broken:
  tree-sitter-cli shells out to node to evaluate grammar.js).

Review pass (found via live testing, not just static reading):
- health.lua called `dodder -version` (a flag) instead of the real
  `version` subcommand, dumping a full usage/stack-trace instead of a
  version string.
- injection.lua's async resolve callback checked only
  nvim_buf_is_valid(bufnr), not that the buffer still names the same
  file -- a slow `dodder show` completing after :bwipeout + buffer
  number reuse could apply a stale language resolution to an unrelated
  buffer. Now also compares the captured file name.
- ftplugin.lua's setup_organize applied window-local fold options to
  vim.api.nvim_get_current_win(), which isn't necessarily the window
  showing the organize buffer (FileType events carry no window). Now
  applies to every window actually displaying the buffer via
  win_findbuf, falling back to current only if none do yet.
- ftplugin.lua's gf_zettel discarded the checkout command's exit
  status, silently opening an empty buffer on a failed checkout instead
  of surfacing the error.
- README.md's grammar table used GFM pipe-table syntax, which this
  repo's pandoc-on-save formatting doesn't preserve (CLAUDE.md
  Formatting section) -- converted to a plain list.

New: six `explore-nvim-*` justfile recipes (root justfile, matching the
existing explore-haustoria-*/explore-live convention) that spin up a
throwaway dodder repo with real example zettels and walk through every
grammar/feature in Neovim: hyphence highlighting + live body injection,
forced-toml injection on .dodder-workspace, organize-text, doddish
query highlighting, and :checkhealth. Backed by a new
zz-nvim/examples/example.doddish fixture (verified parse-clean against
the real doddish grammar) and a README pointer. All six were exercised
end-to-end against the real nix-built dodder-nvim package and dodder
CLI, not just parsed.

:clown: Generated with [Clown](https://github.com/amarbel-llc/clown) (build 0.3.22+fa0406f, amarbel-llc/clown@fa0406f)
Co-Authored-By: Clown <[email protected]>
@friedenberg
friedenberg force-pushed the claude/neovim-treesitter-dodder-tixugd branch from 45aed79 to 5fe0ac2 Compare July 13, 2026 11:37
@friedenberg

Copy link
Copy Markdown
Collaborator Author

Took ownership of this PR: rebased onto master (4 commits, clean rebase, no conflicts), then ran it through simplify + review, plus added a guided local-testing workflow.

Simplify — deduped the bin() resolution logic (was reimplemented separately in ftplugin.lua, health.lua, injection.lua) into one dodder.bin() export; extracted the sepBy1 grammar helper (duplicated verbatim across doddish/organize grammar.js) into grammars/common/util.js; dropped injection.lua's type_to_lang table (every entry mapped to the already-default 'markdown'); deduped the qlmanage/tacky tool-guard in ftplugin.lua; derived go/default.nix's dodder-nvim postInstall copy lines from the dodder-nvim-grammars attrset instead of hardcoding three names twice; added nodejs/tree-sitter to the devShell so tree-sitter generate/test actually work for these grammars (previously silently broken — the CLI shells out to node).

Review — an 8-angle pass (3 correctness + reuse/simplification/efficiency/altitude/CLAUDE.md-conventions) surfaced 4 real bugs, all found and fixed via live testing (headless nvim + a real throwaway dodder repo), not just static reading:

  • health.lua called dodder -version (a flag) instead of the real version subcommand — was dumping a full usage/stack-trace instead of a version string.
  • injection.lua's async resolve callback only checked nvim_buf_is_valid(bufnr), not that the buffer still names the same file — a slow dodder show completing after :bwipeout + buffer-number reuse could apply a stale language resolution to an unrelated buffer. Now also compares the captured file name.
  • ftplugin.lua's setup_organize applied window-local fold options to nvim_get_current_win(), which isn't necessarily the window showing the organize buffer (FileType carries no window). Now applies to every window actually displaying the buffer via win_findbuf.
  • ftplugin.lua's gf_zettel discarded the checkout command's exit status, silently opening an empty buffer on a failed checkout.
  • README.md's grammar table used GFM pipe-table syntax, which this repo's pandoc-on-save formatting doesn't preserve — converted to a plain list.

New: six explore-nvim-* recipes in the root justfile (matching the existing explore-haustoria-*/explore-live convention) that spin up a throwaway dodder repo with real example zettels and walk through every grammar/feature in Neovim — hyphence highlighting + live body injection, forced-toml injection on .dodder-workspace, organize-text, doddish query highlighting, and :checkhealth. Backed by a new zz-nvim/examples/example.doddish fixture and a README pointer. All six were exercised end-to-end against the real nix-built dodder-nvim package and dodder CLI.

Tests: just test green (unit + full bats suite + fixture-generator smoke).

Filed a followup for wiring dodder-nvim into the default Neovim setup: amarbel-llc/eng#244.

:clown: Reviewed and cleaned up by Clown (build 0.3.22+fa0406f, amarbel-llc/clown@fa0406f)

@friedenberg
friedenberg deleted the claude/neovim-treesitter-dodder-tixugd branch July 13, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants