Skip to content

perf(scheduler): resolve projectPath from cache_meta instead of re-reading JSONLs#65

Open
JeanBaptisteRenard wants to merge 1 commit into
doctly:mainfrom
JeanBaptisteRenard:contrib/scheduler-skip-jsonl-head-reads
Open

perf(scheduler): resolve projectPath from cache_meta instead of re-reading JSONLs#65
JeanBaptisteRenard wants to merge 1 commit into
doctly:mainfrom
JeanBaptisteRenard:contrib/scheduler-skip-jsonl-head-reads

Conversation

@JeanBaptisteRenard

Copy link
Copy Markdown

Problem

On every 60s scheduler tick, scanSchedules() re-reads the first 4KB of every project's JSONL files (fs.readFileSync(jsonl, 'utf8').slice(0, 4000)) for every project folder, solely to extract each project's cwd (its projectPath). With many projects this is repeated disk I/O and JSON parsing every minute, even though the same mapping is already cached.

Fix

db.js already maintains a cache_meta table and exports getAllFolderMeta() returning a Map<folder, row> where each row carries projectPath. The scheduler now:

  • Builds a folder → projectPath map from cache_meta once per scan (loadFolderMetaMap()).
  • Resolves projectPath from that cache first.
  • Falls back to the existing JSONL head-read (readProjectPathFromJsonl()) only when a folder is genuinely uncached.

db.js is lazy-required inside the function so that requiring schedule-runner.js never forces the native better-sqlite3 binding to load, and the lookup is wrapped in try/catch returning an empty map if the cache is unavailable — falling back to the original behavior.

Notes

Behavior-preserving: identical projectPath resolution (same fallback, same if (!projectPath) continue, same schedules/triggers). No db.js changes were needed — the existing getAllFolderMeta() export is reused as-is. Syntax-checked (node --check); the test suite passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant