chore(index): name the FTS snippet column + record the declined #193 result - #212
Conversation
FTS5's snippet() takes a column NUMBER, not a name, so inserting any column before `text` silently re-points every excerpt at the wrong column instead of erroring. Not hypothetical: the #193 contextual-prefix experiment added a column ahead of it and excerpts started returning the note's title line ("Prefs — accessibility. Section: Details.") in place of the matched text. Three existing tests caught it. The experiment itself is reverted — measured on the live vault it moved recall@1/recall@5/MRR not at all for +31% index size, and #193 is closed not-planned with the data. This guardrail is worth keeping regardless. Also records the declined result in BACKLOG.md, including why the premise did not hold here: omind has always embedded title+heading+tags with every chunk and given BM25 its own title/heading/tags columns, so it was never the bare-chunk baseline the upstream 35-49% figure is measured against. Co-Authored-By: Claude Opus 5 <[email protected]>
|
Warning Review limit reached
Next review available in: 35 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Part of the v7.0.0 batch. Closes out #193 as not planned — the feature is not in this PR, only its fallout.
What happened
Built the #193 synthetic contextual prefix behind
OMI_CONTEXTUAL_CHUNKS, evaluated it both ways on the live 784-note vault with the semantic leg on:Three of five labelled cases were already rank 1 both ways; the two misses moved 7→8 and 13→11. Neither crossed k=5.
The premise didn't hold for omind. The issue assumed a mid-note chunk "competes on its own words alone" — true of claude-obsidian, not of us.
_ingesthas always embeddedtitle + heading + tags + chunk.text, andchunks_ftshas always given BM25 separate title/heading/tags columns. omind has been doing contextual retrieval without the name; the prefix's only real addition is theSummary, which on descriptive titles restates an already-indexed signal.Full data, including the n=5 caveat, is in the issue and in
BACKLOG.mdunder Not planned.What this PR actually contains
The experiment exposed a genuine trap. FTS5's
snippet()takes a column number:Adding
contextahead oftextsilently re-pointed every excerpt at the wrong column — no error, just excerpts reading"Prefs — accessibility. Section: Details."instead of the matched text. Three existing tests caught it, which is the system working.The literal is now
_FTS_TEXT_COLUMNwith a comment recording why it must move in step with the schema. Small, but it converts a silent failure into an obvious one.Gates
ruff check .·mypy src(strict) ·pytest(869 passed) ·pip-audit— green locally. No behavior change.🤖 Generated with Claude Code