feat(backend,app): add /v1/conversations/{id}/trash endpoint with 30-day Trash UI (#7122)

[mvanhorn]

Summary

Adds a non-breaking POST /v1/conversations/{id}/trash endpoint with a Settings → Trash UI. The existing DELETE /v1/conversations/{id}?cascade=true is unchanged and remains the permanent-delete path used by the cascade cron and by the new "Delete forever" UI.

Why this matters

Issue #7122 asks for a Trash bin so accidentally deleted conversations can be recovered. For users who store months of life-log data, an accidental delete is currently unrecoverable.

Demo

Simulated demo (Remotion) of the move-to-trash flow and the Settings → Trash page. Backend can't run end-to-end without Firebase / Modal / Deepgram credentials in my workspace, so this is a programmatic mock against the real Omi color palette and the Trash page layout in this PR, not a live capture.

Design notes

Why a new trashed_at field instead of extending discarded. The existing discarded boolean is a system heuristic (set_conversation_as_discarded flags short/empty convos). Trash is user-initiated and needs a timestamp for the 30-day purge. Mixing both into one field would force every consumer (search, count, list, iter_all) to distinguish "auto-discarded short convo" from "user-trashed long convo" by inference. A separate explicit field matches the project's existing pattern.

Why the new endpoint is additive. The Flutter app at app/lib/backend/http/api/conversations.dart:92 always calls DELETE ...?cascade=true. Changing default DELETE semantics under that flag would orphan vector and audio state for current mobile clients. The trash endpoint is a new write path; existing call sites are untouched.

API

Method	Path	Behavior
POST	/v1/conversations/{id}/trash	flips trashed_at, revokes share if public
POST	/v1/conversations/{id}/restore	clears trashed_at. Restore does NOT auto re-share.
GET	/v1/conversations/trash	trashed conversations newest first
DELETE	/v1/conversations/{id}?cascade=true	unchanged - permanent delete

include_trashed=False is the default for get_conversations / get_conversations_count. iter_all_conversations defaults to include_trashed=True so /v1/users/export still streams the full stored set.

App

Settings → Trash lists trashed conversations with Restore and Delete-forever actions and a 30-day countdown indicator. Conversation list rows gain a "Move to Trash" overflow action on both the new and non-new card layouts. Existing Delete is unchanged.

Cron

backend/utils/other/purge_trashed.py runs daily at 03:00 UTC, finds trashed_at < now - 30d via Firestore collection_group, and calls the existing cascade-delete logic (conversation + photos + vectors + audio + memories + memory vectors + action items). Wired into utils/other/jobs.start_job. Logs use sanitize_pii().

Vector / search

Trashed conversations are filtered at query time in vector_db.query_vectors, vector_db.query_vectors_by_metadata, and utils/conversations/search.search_conversations via a new database.conversations.filter_visible_conversation_ids helper. Vectors stay in Pinecone until permanent delete, so restore is lossless.

Tests

• backend/tests/integration/test_conversation_trash.py (502 lines): trash, restore, listing, default exclusion, share-revoke-on-trash, missing-target 404, filter_visible_conversation_ids, Typesense search exclusion, over-fetch pagination.
• backend/tests/unit/test_purge_trashed_cron.py (139 lines): schedule predicate, 29-day retention, 31-day expired purge cleanup chain, error continuation.
• backend/tests/unit/test_lock_bypass_fixes.py::TestSearchRedaction updated to mock filter_visible_conversation_ids as a pass-through.

All 16 pass locally.

l10n

New keys are added to app_en.arb. Non-English locales fall back to English at runtime via TrashLocalizationExtension on AppLocalizations until flutter gen-l10n is rerun and omi-add-missing-language-keys-l10n propagates translations. The extension is then shadowed by the canonical class methods and can be removed.

Out of scope

• Status enum refactor consolidating discarded + trashed_at
• Translated l10n strings for the 48 non-English locales
• Backfill migration setting trashed_at = null on legacy docs (would let list/count use Firestore-native WHERE trashed_at == None filters instead of the current over-fetch and post-filter)

Fixes #7122

[greptile-apps]

Greptile Summary

Adds a soft-delete "Trash" layer to conversations: a POST /v1/conversations/{id}/trash endpoint sets trashed_at, a matching restore endpoint clears it, and a daily cron hard-deletes anything older than 30 days. The Flutter app gains a Settings → Trash page with restore/delete-forever actions and a "Move to Trash" overflow menu on conversation list items.

• Backend: New trashed_at field on the Conversation model; Firestore helpers for trash/restore/list/filter; filter_visible_conversation_ids is threaded into Pinecone vector queries and Typesense search so trashed conversations are excluded from AI retrieval without touching stored data; the purge cron reuses the existing cascade-delete path.
• Flutter app: TrashPage widget under Settings, overflow "Move to Trash" with confirmation dialog, ConversationProvider.trashConversation removes the item from all local lists; swipe-to-delete is intentionally left as permanent delete per design.
• l10n: New ARB keys added to app_en.arb; a temporary TrashLocalizationExtension provides the strings for non-English locales until flutter gen-l10n propagates them.

Confidence Score: 3/5

The app-facing endpoints and Flutter UI are solid, but the nightly purge cron will fail on its first run without a Firestore Collection Group index that is not included in the PR.

The collection-group query in list_expired_trashed requires a manually-created Firestore Collection Group index. Without it the query throws FAILED_PRECONDITION, escapes the per-conversation try/except, and crashes the entire purge — every trashed conversation older than 30 days accumulates indefinitely. Separately, iter_all_conversations checks len(batch) < batch_size against the filtered batch size when include_trashed=False, causing the generator to stop early and silently return an incomplete set.

backend/utils/other/purge_trashed.py and backend/database/conversations.py need attention before the purge cron goes live. All Flutter and API endpoint files look safe to merge.

Important Files Changed

Filename	Overview
backend/database/conversations.py	Adds trash/restore/list/filter DB helpers; iter_all_conversations has an early-exit bug when include_trashed=False, and get_conversations_count does a full O(n) scan for non-trashed counts.
backend/utils/other/purge_trashed.py	New daily cron to hard-delete 30-day expired trashed conversations; the collection-group Firestore query requires a manually-created index not included in the PR, causing the first cron run to fail completely.
backend/routers/conversations.py	Adds /trash, /restore, and GET /trash endpoints; route ordering is correct (static before parametric); auth and 404 handling are consistent with existing patterns.
backend/utils/conversations/search.py	Post-filters Typesense results through filter_visible_conversation_ids so trashed conversations are excluded by default.
backend/database/vector_db.py	Pipes vector query results through filter_visible_conversation_ids, correctly excluding trashed conversations from AI retrieval without touching Pinecone state.
app/lib/pages/settings/trash_page.dart	Clean Trash UI with restore, delete-forever, days-remaining countdown, and empty state; mounted guards are present on all async callbacks.
app/lib/utils/l10n_extensions.dart	Temporary extension providing new l10n strings; restoreConversation is a duplicate of an already-generated ARB key and is dead code.
app/lib/providers/conversation_provider.dart	Adds trashConversation to the provider; correctly removes the conversation from both lists and calls groupConversationsByDate() which notifies listeners.
app/lib/pages/conversations/widgets/conversation_list_item.dart	Adds 'Move to Trash' overflow menu item with confirmation dialog; swipe-to-delete intentionally left as permanent delete per PR design.
backend/models/conversation.py	Adds trashed_at: Optional[datetime] = None to the Conversation model; additive and non-breaking.
backend/utils/other/jobs.py	Wires purge cron into start_job; synchronous call is acceptable given it runs as the sole task under asyncio.run().

Sequence Diagram

sequenceDiagram
    participant App
    participant API as FastAPI Router
    participant DB as Firestore (conversations_db)
    participant Redis
    participant Pinecone
    participant Cron as Daily Cron (03:00 UTC)

    App->>API: "POST /v1/conversations/{id}/trash"
    API->>DB: trash_conversation(uid, id)
    DB->>DB: "set trashed_at = now()"
    DB->>Redis: remove public share if needed
    DB-->>API: updated conversation dict
    API-->>App: 200 Conversation

    App->>API: "POST /v1/conversations/{id}/restore"
    API->>DB: restore_conversation(uid, id)
    DB->>DB: DELETE trashed_at field
    API-->>App: 200 Conversation

    App->>API: GET /v1/conversations/trash
    API->>DB: get_trashed_conversations(uid)
    DB-->>API: "docs where trashed_at != null"
    API-->>App: 200 List[Conversation]

    App->>API: Vector/Search query
    API->>Pinecone: query_vectors
    Pinecone-->>API: raw conversation_ids
    API->>DB: filter_visible_conversation_ids(uid, ids)
    DB-->>API: ids where trashed_at is null
    API-->>App: filtered results

    Cron->>DB: "list_expired_trashed(cutoff = now - 30d)"
    DB-->>Cron: (uid, conversation_id) pairs
    Cron->>DB: delete_conversation(uid, id)
    Cron->>Pinecone: delete_vector(uid, id)
    Cron->>DB: delete_memories / action_items

Loading

Comments Outside Diff (1)

1. backend/database/conversations.py, line 314-327 (link)

   iter_all_conversations terminates early when include_trashed=False

   The break condition compares len(batch) (the filtered list) against batch_size. When include_trashed=False, trashed documents are skipped inside the loop, so len(batch) can be far smaller than the number of Firestore docs actually fetched. For example, if batch_size=400 and 200 of the fetched documents are trashed, len(batch) becomes 200, which is less than batch_size — triggering an early exit even though more non-trashed conversations remain. Any future caller passing include_trashed=False would silently receive an incomplete result set. The raw Firestore document count should be tracked separately to decide whether to break.

_{Reviews (1): Last reviewed commit: "fix: include trashed in data export and ..." | Re-trigger Greptile}

[greptile-apps]

Missing Firestore Collection Group index for the purge query

db.collection_group(conversations_collection).where(filter=FieldFilter('trashed_at', '<', cutoff_dt)) is a collection-group inequality query. Firestore does not auto-generate Collection Group indexes — one must be created manually in the Firestore console or via firestore.indexes.json before this query executes. Without it the first nightly cron run at 03:00 UTC will throw a 400 FAILED_PRECONDITION exception inside list_expired_trashed, which propagates out of the for uid, conversation_id in … loop in purge_expired_trashed_conversations (that call is outside the per-conversation try/except), crashing the entire purge function and leaving all expired-trashed conversations permanently un-purged — a data-retention failure. The PR should include a firestore.indexes.json that registers a Collection Group index on conversations / trashed_at ASC.

[greptile-apps]

Full-collection scan on get_conversations_count when include_trashed=False

When include_trashed=False, the function streams every document in the user's conversations subcollection and calls _is_not_trashed(doc.to_dict()) on each one, making it an O(n) client-side filter. For users with thousands of conversations this is slow and produces high read billing. The PR notes a backfill migration is out of scope, but once legacy documents have trashed_at=null consistently, a server-side WHERE trashed_at == NULL filter could replace the full scan. A comment acknowledging this known limitation would help future contributors avoid adding more callers that rely on an efficient count.

[greptile-apps]

restoreConversation already exists as a key in app_en.arb (line 112), so it is already present in the generated AppLocalizations class. Dart will silently prefer the class method over the extension, making this getter dead code. It should be removed from the extension to avoid confusion when the temporary extension is eventually cleaned up.

Suggested change

	extension TrashLocalizationExtension on AppLocalizations {
	String get trash => 'Trash';
	String get trashEmpty => 'Trash is empty';
	String get trashDescription => 'Conversations in Trash are permanently deleted after 30 days.';
	String get moveToTrash => 'Move to Trash';
	String get restoreConversation => 'Restore';
	String get deleteForever => 'Delete forever';
	extension TrashLocalizationExtension on AppLocalizations {
	String get trash => 'Trash';
	String get trashEmpty => 'Trash is empty';
	String get trashDescription => 'Conversations in Trash are permanently deleted after 30 days.';
	String get moveToTrash => 'Move to Trash';
	String get deleteForever => 'Delete forever';

[mvanhorn]

Addressed in 0f9565a:

• P1 (Firestore Collection Group index): added backend/firestore.indexes.json registering a Collection Group index on conversations / trashed_at ASC, so the purge_expired_trashed_conversations cron's collection_group(...).where(trashed_at < cutoff_dt) inequality query has the required index and won't 400 FAILED_PRECONDITION on the first nightly run.
• P2 (get_conversations_count O(n) scan): added a comment in backend/database/conversations.py documenting that the include_trashed=False branch is an O(n) client-side filter due to legacy docs without trashed_at, and that a server-side WHERE trashed_at == NULL filter becomes feasible once those are backfilled.
• P2 (dead restoreConversation getter): removed it from TrashLocalizationExtension; the key exists in app_en.arb and is already on the generated AppLocalizations class.

Lint check failure was on dart format reporting changes to 6 files - locally dart format --line-length 120 --set-exit-if-changed on those 6 files exits 0, so it should pass on this push. Will re-check after CI runs.

[dluffy56]

Hi @mvanhorn iirc Firestore indexes here are created manually (by maintainer ig) so firestore.indexes.json is redundant - you can remove it