Skip to content

feat(search): add since/before date window to mempalace_search and CLI search (#463)#2000

Open
mvalentsev wants to merge 1 commit into
MemPalace:developfrom
mvalentsev:feat/search-date-filter-463
Open

feat(search): add since/before date window to mempalace_search and CLI search (#463)#2000
mvalentsev wants to merge 1 commit into
MemPalace:developfrom
mvalentsev:feat/search-date-filter-463

Conversation

@mvalentsev

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #463

Adds an optional [since, before) date window to both search surfaces, filtered on each drawer's filed_at (the created_at shown in results):

  • MCP: mempalace_search accepts since and before (ISO date or datetime; since inclusive, before exclusive).
  • CLI: mempalace search gains --since / --before.

Semantics are shared with the list_drawers filter from #1128 through a new side-effect-free mempalace/date_window.py module (the #1128 helpers moved there; mcp_server keeps them under the old private names, so the existing list_drawers behavior and tests are untouched). Comparison is wall-clock and timezone-naive, Z/offset inputs are normalized, and drawers with a missing or unparseable filed_at are excluded while a bound is active, so a date-filtered result never silently includes rows of unknown age.

Design notes:

  • ChromaDB rejects string operands for $gte/$lt (verified on the locked 1.5.7), so the window cannot be a server-side where clause. It is applied as a post-filter on every candidate path: the vector scored loop, candidate_strategy="union" lexical merge, and the BM25-only sqlite path used when vector search is disabled.
  • A post-filter starves a fixed pool, so under an active window the vector candidate pool widens from n_results * 3 to max(min(n_results * 15, 500), n_results), the same ceiling the degraded-filter path already uses. When the widened pool still comes back full, the response carries date_filter_pool_truncated: true instead of passing a thin result off as complete; the BM25 path sets the same flag when its candidate page hits max_candidates.
  • On the BM25 sqlite path the window is also pushed into SQL as a narrowing prefilter at whole-day granularity, which keeps it provably wider than the window for every ISO spelling that shares the date prefix (bare date, space separator, minute precision, Z/offset suffixes); the Python check stays authoritative, mirroring how wing/room are double-checked there.
  • Parameter names are since/before rather than the issue's after: list_drawers already shipped since/before with these exact semantics in feat: add since/before date filter to list_drawers and search #1128, and one vocabulary across the read tools seemed more valuable than a second name for the same bound. Relative forms (--last 7d) and a --between shorthand from the issue discussion are intentionally left for a follow-up; they layer cleanly on top of this window.
  • Known edges, called out rather than hidden: when ChromaDB fails a filtered query and search falls back to the unfiltered-plus-post-filter recovery path (fix: filtered search fallback + diary_write content alias #1245), total_before_filter reflects the post-filtered count, so the truncation flag can under-report there; union mode's lexical fetch keeps its n_results * 3 size (widening it under a window is a possible follow-up, the merge itself already respects the window); a before at the calendar ceiling (9999-12-31) skips the SQL narrowing instead of overflowing past datetime.max; and a stored filed_at with leading whitespace would pass the Python filter but not the SQL prefilter (no in-tree writer produces one).

How to test

Automated (all green locally, ruff 0.15.14 clean):

uv run pytest tests/test_date_window.py tests/test_searcher.py tests/test_mcp_server.py -q

New coverage: 20 unit tests for the shared parser/window, 26 searcher tests (window/boundaries/invalid/inverted, undated and aware filed_at, pool widening recall, truncation flags on both paths, legacy filed_at spellings and the calendar-ceiling sentinel on the sqlite path, union side door, CLI ordering and errors), 6 MCP-surface tests (schema, threading, filters echo, clean error shapes).

Manual, against a real palace:

mempalace search "budget meeting" --since 2026-04-01 --before 2026-05-01
mempalace search "budget meeting" --since garbage   # clear error, exit 1

MCP: call mempalace_search with {"query": "budget meeting", "since": "2026-04-01"} over JSON-RPC; the tool schema now declares both properties, results stay within the window, filters echoes it, and an invalid or inverted window returns a structured error instead of -32000/-32602.

Checklist

  • Tests pass (python -m pytest tests/ -v)
  • No hardcoded paths
  • Linter passes (ruff check .)

@mvalentsev mvalentsev marked this pull request as ready for review July 13, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add date/time filtering to mempalace_search

1 participant