Reconcile cache with filesystem on get-projects so sessions stop going missing#60
Open
ymajoros wants to merge 1 commit into
Open
Reconcile cache with filesystem on get-projects so sessions stop going missing#60ymajoros wants to merge 1 commit into
ymajoros wants to merge 1 commit into
Conversation
…g missing The session cache is only rebuilt from the filesystem on a cold start (populateCacheViaWorker runs only when the cache is completely empty). Once the cache has any rows, a project folder that changed while the app was closed — or that was created before the build which first indexed it — is never re-scanned, so its sessions (and whole worktrees) silently disappear from the sidebar. The transcripts are intact on disk; only the cache is stale. Make get-projects reconcile first: re-index folders that are new or whose newest .jsonl is newer than what we last indexed. The check is a cheap, stat-only gate (getFolderIndexMtimeMs vs cache_meta.indexMtimeMs), so it's effectively free when nothing changed and only does real work for folders that actually need it. The previously dead populateCacheFromFilesystem() (defined and exported but never called) is repurposed into this gated reconcileCacheFromFilesystem().
274eb72 to
edf06c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #59.
Problem
Sessions and whole worktrees silently vanish from the sidebar while their transcripts are still on disk, and reopening/restarting doesn't bring them back.
get-projectsonly scans the filesystem on a cold start:Once the cache is non-empty it's never reconciled again, so folders that changed while the app was closed — or that were never indexed by the current build (
cache_meta.indexMtimeMs === 0) — never reappear. The livefs.watchonly covers changes while the app is running.Fix
Reconcile on
get-projectsbefore building from cache. The deadpopulateCacheFromFilesystem()(defined + exported, never called) is repurposed intoreconcileCacheFromFilesystem(), now stat-gated: it re-indexes only folders that are new or whose newest.jsonlis newer than what was last indexed.When nothing changed this is just a
readdir+statper folder — no transcript parsing, no DB writes — so the steady-state cost onget-projectsis negligible; real work happens only for folders that actually need it.refreshFolderalready does its own per-file mtime skip, so this composes cleanly.Why here (vs. only at startup)
Reconciling on
get-projectsis self-healing for both the "changed while closed" and "old build never indexed it" cases, and naturally re-runs if folders drift later — without a separate watcher or timer. The mtime gate keeps it cheap. Happy to move it to a one-time startup pass instead if you'd prefer.Testing
node --testpasses, including a newtest/reconcile-cache.test.jsthat asserts the gate: a never-indexed folder and a stale folder (indexMtimeMsolder than disk) get indexed, while an up-to-date folder is skipped.indexMtimeMs = 0) re-indexed and reappeared in the sidebar.Notes
Pure cache/indexing change — no schema change, no change to
session_meta(names/stars/archive) or settings.