Skip to content

Reconcile cache with filesystem on get-projects so sessions stop going missing#60

Open
ymajoros wants to merge 1 commit into
doctly:mainfrom
ymajoros:fix/reconcile-cache-on-get-projects
Open

Reconcile cache with filesystem on get-projects so sessions stop going missing#60
ymajoros wants to merge 1 commit into
doctly:mainfrom
ymajoros:fix/reconcile-cache-on-get-projects

Conversation

@ymajoros

@ymajoros ymajoros commented Jun 1, 2026

Copy link
Copy Markdown

Fixes #59.

Problem

Sessions and whole worktrees silently vanish from the sidebar while their transcripts are still on disk, and reopening/restarting doesn't bring them back.

get-projects only scans the filesystem on a cold start:

const needsPopulate = !isCachePopulated() || !isSearchIndexPopulated();
if (needsPopulate) { populateCacheViaWorker(); return []; }
return buildProjectsFromCache(showArchived);   // cache only, never reconciled

Once the cache is non-empty it's never reconciled again, so folders that changed while the app was closed — or that were never indexed by the current build (cache_meta.indexMtimeMs === 0) — never reappear. The live fs.watch only covers changes while the app is running.

Fix

Reconcile on get-projects before building from cache. The dead populateCacheFromFilesystem() (defined + exported, never called) is repurposed into reconcileCacheFromFilesystem(), now stat-gated: it re-indexes only folders that are new or whose newest .jsonl is newer than what was last indexed.

for (const folder of folders) {
  const meta = metaMap.get(folder);
  const folderPath = path.join(PROJECTS_DIR, folder);
  if (!meta || getFolderIndexMtimeMs(folderPath) > (meta.indexMtimeMs || 0)) {
    refreshFolder(folder);
  }
}

When nothing changed this is just a readdir + stat per folder — no transcript parsing, no DB writes — so the steady-state cost on get-projects is negligible; real work happens only for folders that actually need it. refreshFolder already does its own per-file mtime skip, so this composes cleanly.

Why here (vs. only at startup)

Reconciling on get-projects is self-healing for both the "changed while closed" and "old build never indexed it" cases, and naturally re-runs if folders drift later — without a separate watcher or timer. The mtime gate keeps it cheap. Happy to move it to a one-time startup pass instead if you'd prefer.

Testing

  • node --test passes, including a new test/reconcile-cache.test.js that asserts the gate: a never-indexed folder and a stale folder (indexMtimeMs older than disk) get indexed, while an up-to-date folder is skipped.
  • Verified live on Linux / 0.0.30: 27 worktree folders that were stuck empty (indexMtimeMs = 0) re-indexed and reappeared in the sidebar.

Notes

Pure cache/indexing change — no schema change, no change to session_meta (names/stars/archive) or settings.

…g missing

The session cache is only rebuilt from the filesystem on a cold start
(populateCacheViaWorker runs only when the cache is completely empty).
Once the cache has any rows, a project folder that changed while the app
was closed — or that was created before the build which first indexed it —
is never re-scanned, so its sessions (and whole worktrees) silently
disappear from the sidebar. The transcripts are intact on disk; only the
cache is stale.

Make get-projects reconcile first: re-index folders that are new or whose
newest .jsonl is newer than what we last indexed. The check is a cheap,
stat-only gate (getFolderIndexMtimeMs vs cache_meta.indexMtimeMs), so it's
effectively free when nothing changed and only does real work for folders
that actually need it.

The previously dead populateCacheFromFilesystem() (defined and exported but
never called) is repurposed into this gated reconcileCacheFromFilesystem().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sessions/worktrees silently disappear from sidebar; cache never reconciled after cold start

1 participant