fix(index): drop and recreate on a schema bump instead of deleting rows

[CryptoJones]

Closes #210. Part of the v7.0.0 batch.

This is already shipped

Anyone upgrading an existing install to v6.6.0 has a dead search index until they run omind reindex --rebuild by hand. Found while setting up the #193 evaluation gate: omind bench --quality against the live vault returned recall@1 = 0%.

Mechanism

def _wipe(self, db):
    for table in (...):
        db.execute(f"DELETE FROM {table}")   # rows only

_SCHEMA is entirely CREATE TABLE IF NOT EXISTS, and _connect runs it before _wipe. So on an existing file the tables are never recreated and the old column shape survives the wipe.

v6.6.0 (#195) added confidence + conflicts_with and bumped SCHEMA_VERSION 4 → 5. Every ingest INSERT then fails with no such column: confidence — a sqlite3.Error, which refresh() catches and converts to None. Permanently:

refresh() -> None
search()  -> None
doctor    -> 12.9 MiB, 0 notes, 0 vectors; 784 stale note(s)

Reproduced on the live 784-note vault and in a fixture.

Why nobody would notice

Invariant 2 did exactly what it promises — retrieval fell back to the pre-index substring scan. So there is no error, no crash, no traceback. Literal-substring queries keep returning something. The failure is silently much worse recall, which is the hardest kind to spot.

Measured on the live vault, wedged vs. rebuilt:

	wedged	rebuilt
recall@1	0%	60%
recall@5	0%	60%
MRR	0.00	0.64

omind doctor was the one honest signal (784 stale note(s); run 'omind reindex --rebuild') — worth remembering that it was right and nothing else was.

Fix

_wipe drops the tables and re-runs _SCHEMA. A wipe already means every row is disposable, so there is nothing to preserve by being gentle.

The general lesson

This is latent since SCHEMA_VERSION was introduced — I didn't add it, #195 was just the first bump to change a table's shape rather than only its contents. SCHEMA_VERSION promised a migration it never actually performed, and every existing test built a fresh index.

test_a_schema_bump_that_adds_a_column_rebuilds_the_index walks the real upgrade path: build at the current schema, strip the two columns back to the previous shape, set the old version, reopen. That test would have caught this, and will catch the next one.

Gates

ruff check . · mypy src (strict) · pytest (869 passed) · pip-audit — green locally.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

@CryptoJones, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 43 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0d78b317-da04-45a9-8bb6-ea1599a4c75c

📥 Commits

Reviewing files that changed from the base of the PR and between 2338ef5 and 3f52d45.

📒 Files selected for processing (4)

• BACKLOG.md
• CHANGELOG.md
• src/omind/searchindex.py
• tests/test_searchindex.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}