Conversation
ajslater
commented
Jun 15, 2026
Owner
- Fixes
- Fix read/unread filter leaking one user's read state to other users.gi
Fix clear settings null bug, add global settings clear button
- clearComicSettings was setting book.settings to null, causing
Object.entries() to throw TypeError downstream in getBookSettings
- Add null guard in getBookSettings as defense-in-depth
- Add null guard in isClearDisabled computed
- Add clearGlobalSettings action and clear button to Default Settings panel
- Compare against READER_DEFAULTS to determine if global clear is disabled
vite-plugin-dynamic-base 1.4.1 dropped the leading `/` it used to add inside template-element replacements. The codex publicPath stripped APP_PATH's leading slash with `.substring(1)` to avoid a double `/` with the old behavior; under 1.4.1 the strip made chunk URLs relative, so deep routes like /admin/libraries fetched chunks from /admin/static/... and got HTML from the Django catch-all, breaking the admin panel and any lazy-loaded route. Bump version to 1.12.6 and add news entry. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
a709a74 added a "." prefix to _IGNORED_BASENAME_PREFIXES so the poller's walker, the watchfiles filter, and expand_dir_added all skip hidden files before any cover predicate runs. User-supplied folder covers named .codex-cover.jpg / .png / .webp / ... are dotfiles by name but legitimate covers, so the dotfile filter swallowed them and broke custom folder covers from v1.12.5 on. is_ignored_basename now exempts a basename when match_folder_cover claims it. .DS_Store, .git, .Trashes, .codex-cover (no extension), .codex-cover.txt etc. still get filtered. Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
DRF's ``SessionAuthentication`` enforces CSRF on unsafe methods, which rejects proxy-authenticated API clients that legitimately carry no CSRF token. Add ``HttpRemoteUserAuthentication`` (subclass of DRF's stock ``RemoteUserAuthentication`` reading ``HTTP_REMOTE_USER`` instead of ``REMOTE_USER``) and conditionally prepend it to ``DEFAULT_AUTHENTICATION_CLASSES`` when ``CODEX_AUTH_REMOTE_USER`` is enabled, so proxy-forwarded ``Remote-User`` API requests authenticate through the existing ``RemoteUserBackend`` without hitting CSRF. Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
…vers Vuetify 4's v-dialog defaults to location 'center center', which activates the connected overlay location strategy. That strategy caps width with parseFloat(props.maxWidth), so "20em" parses to 20 and the em is dropped, collapsing dialogs to a ~20px sliver. Convert every em-based max-width/ min-width on a v-dialog (or forwarded via $attrs) to unitless px at 16px/em. Co-Authored-By: Claude Opus 4.8 <[email protected]>
The UNREAD filter used Q(bookmark=None), a multi-valued relationship test for "comic has no bookmark from anyone" rather than "no bookmark for me". Once any user finished a comic it dropped out of every other user's (and anonymous visitors') unread view. UNREAD now negates the per-user finished predicate (~(my_filter & finished=True)), compiling to a per-user NOT-EXISTS subquery. Also harden get_my_bookmark_filter: an anonymous visitor with no established session_key resolved to `session_id IS NULL`, which matches every authenticated user's bookmarks. Return a match-nothing filter instead; a session key is minted only when a bookmark is written. Adds tests/test_bookmark_filter_isolation.py covering cross-user and anonymous isolation for READ/UNREAD/IN_PROGRESS. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replace a stale `ty: ignore[no-matching-overload]` with an isinstance guard so `settings` is a Mapping before `_copy_params_into`, clearing the `invalid-argument-type` diagnostic from `make ty`. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Skipping super() left the class-level atomic from setUpTestData uncommitted-but-unrolled-back, leaking the codex_init() "admin" superuser into later test classes and causing UNIQUE constraint failures (e.g. test_bookmark_filter_isolation) in full-suite runs. Co-Authored-By: Claude Opus 4.8 <[email protected]>
ajslater
added a commit
that referenced
this pull request
Jun 15, 2026
commit 5e6f4dfc55ee4ad3f8fab48b7b33ffe02f17d8a4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:21 2026 -0700 update version 2.0.0a13 commit 59d607d6c886cccb093fbc07dea5a2ef97ab1dc0 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:25:51 2026 -0700 test(onlinetag): exercise replay path in unmatched-write test The candidate carried issue_id 123, routing the resolve through the explicit-id fetch (real fetch_tags_by_explicit_id) instead of the replay session the test sets up and claims to verify. That crashed on the empty touched .cbz before any assertion ran. Drop the issue_id so the candidate falls back to the replay path guarded by _first_tags. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit dfa2349e8b821fbaa93b53fb4639bd2d66db8522 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:19:14 2026 -0700 fix(mail): ASCII hyphen in noqa comments to dodge complexipy panic complexipy's extract_comment_marker slices noqa comment text by a fixed byte offset (16); a multi-byte em-dash straddling that offset lands mid-char and panics the Rust core, aborting the whole run. Swap the two em-dashes in the noqa comments for ASCII hyphens. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 1799ce937559d62a667a0bf66f819f1593c93c5e Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:16:34 2026 -0700 decomplexify test for radon commit 6213c0afe641942e9e1d28f090970c6da524ceb4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:14:19 2026 -0700 remove unused ty ignore commit dc59f1dc9560e177f9e0c36ea4552208014746d6 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:13:38 2026 -0700 remove redundant definition commit 5d7a204d5069783c8638bc8f76f84bf15840239a Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:10:36 2026 -0700 Squashed commit of the following: commit 1274bad663bf382d76453e65a6fc935b8b3d4559 Merge: a8aca7b33 72fad6861 Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:57:33 2026 -0700 Merge branch 'main' into develop commit 72fad6861b0af14731847b5444f097fdec699fda Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:06:27 2026 -0700 v1.12.8 (#781) commit a8aca7b3341e7afdee8e99829be88d77e4d988c5 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:50:21 2026 -0700 fix(test): call super().tearDownClass() in BaseTestImporter Skipping super() left the class-level atomic from setUpTestData uncommitted-but-unrolled-back, leaking the codex_init() "admin" superuser into later test classes and causing UNIQUE constraint failures (e.g. test_bookmark_filter_isolation) in full-suite runs. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit d7570ecf99e1a15449dec2f7a0550122b9d405df Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:16:59 2026 -0700 remove gitignore commit be211c5724d5d7cc25ca367813fa34244097f372 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:12:03 2026 -0700 fix(types): narrow OPDS redirect settings to Mapping for ty Replace a stale `ty: ignore[no-matching-overload]` with an isinstance guard so `settings` is a Mapping before `_copy_params_into`, clearing the `invalid-argument-type` diagnostic from `make ty`. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5dd3a0dd9feb18aa4b13c15b000c048befdeceef Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:09:05 2026 -0700 update devenv commit e5a2a41881d35e2d8716bd6dfefdd64fa3792b5c Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:07:34 2026 -0700 shfmt commit 1f867c1497e4edc14315d63c7eeb8c3e2b0f07bd Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:05:38 2026 -0700 shfmt commit c5366787b56212e12a69c5aaab1776474962a31b Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:02:56 2026 -0700 fix(bookmark): scope read/unread filters per user (v1.12.8) The UNREAD filter used Q(bookmark=None), a multi-valued relationship test for "comic has no bookmark from anyone" rather than "no bookmark for me". Once any user finished a comic it dropped out of every other user's (and anonymous visitors') unread view. UNREAD now negates the per-user finished predicate (~(my_filter & finished=True)), compiling to a per-user NOT-EXISTS subquery. Also harden get_my_bookmark_filter: an anonymous visitor with no established session_key resolved to `session_id IS NULL`, which matches every authenticated user's bookmarks. Return a match-nothing filter instead; a session key is minted only when a bookmark is written. Adds tests/test_bookmark_filter_isolation.py covering cross-user and anonymous isolation for READ/UNREAD/IN_PROGRESS. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 541907b6708dfbbde197432960c6adc6dcf44b02 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:44:58 2026 -0700 fix(frontend): convert v-dialog em widths to px so dialogs aren't slivers Vuetify 4's v-dialog defaults to location 'center center', which activates the connected overlay location strategy. That strategy caps width with parseFloat(props.maxWidth), so "20em" parses to 20 and the em is dropped, collapsing dialogs to a ~20px sliver. Convert every em-based max-width/ min-width on a v-dialog (or forwarded via $attrs) to unitless px at 16px/em. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5918d0b621cd635f3de32f142d4dff7ec9c45242 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:41:25 2026 -0700 move VPullToRefersh out of labs commit 57d5cf3888c2dc0aab55e46447b41a459d37003a Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:32:39 2026 -0700 update devenv and deps commit cd944f0e4cf7044ba8e72cb6f90539ca892adb72 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:25:16 2026 -0700 Squashed commit of the following: commit b5874b67774af29f43603f01690a0ca97b43c46a Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:24:54 2026 -0700 update deps commit e0a16dff8e66c357b2d2aa5e4ff3ea7decce4572 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:23:51 2026 -0700 remove pytest-gitignore commit 35b84324300352499f3c455aa43201c3f475cd21 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:20:46 2026 -0700 update version to 2.0.0a12 commit e3e1167851960a42fef9a5ceba3b3b5470f7a56c Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:19:54 2026 -0700 Fix basedpyright error on Folder FK access in moved-parents test The Folder model annotates parent_folder as ForeignKey explicitly, so django-stubs doesn't synthesize the implicit parent_folder_id attribute. Compare the FK object instead (Django compares models by pk). Pre-existing since ce1fd81ad; behavior unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 0ee9c966f9d1dcfd84d36e6077778fd9354d9574 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:45 2026 -0700 update deps commit 38f527b57ce30be0f162d7aae2bf51d938e3141b Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:39 2026 -0700 ignore benchmark dir commit 7007c109accb1654c5b61580336d71cedfb5ce3f Author: AJ Slater <[email protected]> Date: Sun Jun 14 17:56:39 2026 -0700 Add online tagging status table with live progress and session controls Admin Tagging tab now shows a live status table for the online tagging batch: per-comic state, per-source rate-limit countdowns, batch ETA, and match-review status. Backed by a daemon-published session snapshot in the tagging cache, exposed via GET /api/v4/admin/tag-sessions/snapshot and refreshed over the existing task.progress websocket. - Snapshot: session_snapshot.py builds a capped, JSON-safe per-comic snapshot from OnlineTagOutcomeStats + pending prompts; published throttled from the runner, frozen inactive on finish, cleared at startup / janitor. - Session lifecycle: pause / resume / abort / dismiss controls in the table, wired to the online-tag store and admin endpoints. - Match-review fixes: - direct-id fetch for explicit picks (no rate-limit-prone re-search drift) - record user_matched / user_skipped outcomes, overlaid on the snapshot - client guard against resurrecting just-answered prompts in the dialog - optimistic local overlay so resolutions show during rate-limit stalls Tests: backend snapshot / resume / session-manager / janitor suites; frontend store + status-table specs. ruff/ty/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit ce1fd81adb4823ac454950ab66905a38e3f6d9e1 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:56:39 2026 -0700 Fix ScribeThread crash on folder move under untracked parent _get_move_create_folders_one_layer returned bare path strings, but bulk_folders_create expects key tuples (MODEL_REL_MAP[Folder] == (("path",), ...)) and iterates each entry to extract the path. Bare strings were iterated character by character, producing single-char paths like PosixPath('K') whose stat() raised FileNotFoundError and crashed the ScribeThread during import. Wrap the path in a tuple to match the contract, and add a regression test covering a folder move whose destination parent isn't yet a tracked Folder. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 265988a4b44ba7ebe9c810bedc11a42796853d90 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:30:44 2026 -0700 Delete duplicate migration 0044 that broke test-DB setup The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e6), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c31c61e6175d24d9e1dde1835c7235c367c251c9 Author: AJ Slater <[email protected]> Date: Sat Jun 13 02:00:11 2026 -0700 fix some kind of merge issue commit 92e526940e96b6ea9679363159395270233e75f2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:55 2026 -0700 Squashed commit of the following: commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:29 2026 -0700 refactor(complexity): clear radon cc rank-C findings bin/lint-complexity.sh reported three rank-C functions; extract helpers to drop each below the gate (radon cc --min C now empty). - AdminTagByIdView.post: pull primary-identifier parsing and merge extra-id resolution into _resolve_primary / _resolve_extra_ids. - OnlineTagSessionManager._apply_resolution: extract _handle_unresolved and _enqueue_resolved_write. - test_update_protagonist: extract repeated protagonist assertions into a helper (radon counts each assert as a branch). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 55bb659fcc8c02e73c20a34823652a4f81be1d56 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:46:04 2026 -0700 refactor(migrations): squash 0044 into 0043 Fold the merge_all_sources field into the ComicboxTaggingDefaults CreateModel in 0043 and drop the standalone 0044. makemigrations --check reports no drift. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 7a988737a63286cc85396346579e124e5fff8e5b Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:44:59 2026 -0700 feat(onlinetag): merge metadata from all sources Add an opt-in "merge all sources" mode that queries every enabled online source per comic and merges the results (comicbox first_wins=False) for maximum tag completeness, instead of stopping at the first match. - Admin tagging defaults: merge_all_sources boolean (default off), settable as the default and overridable per scan. - Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate + rate-limit warning multiply calls by source count (kept in sync with estimate.py) so the doubled API cost is visible. - By-ID: chips input accepting multiple URLs/ids; merging fetches two explicit ids (one per source) and merges them. Source select shows only when a token can't be auto-identified (bare number). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:34:56 2026 -0700 update deps commit 473221d7a09403b1848f202d55902b71e31319c2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:22:58 2026 -0700 fix(lint): clear ruff/pyright findings; fix benchmark import path - tests: add docstrings, name magic estimate values, use pytest.raises and out-of-class exception message - onlinetag: collapse implicit f-string concatenation in log call - benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*) which was a real runtime ImportError in library generation Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 45a9fde6076a5b0b15be889769e9a55af4b0678a Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:13:49 2026 -0700 refactor(migrations): squash 0044 + 0045 into 0043 Fold the comic-leading composite Bookmark indexes (former 0044) and the LibrarianStatus eta/retry_at fields + JFR status-type choice (former 0045) into the consolidated 0043 migration; delete both files. makemigrations --check reports no changes against the model state. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 22da6b9cd85615f3d7b8e7043818babac313d24e Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:35 2026 -0700 update deps commit 6f27454623e43cccd95253d5dfe66471820c9349 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:15 2026 -0700 fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns Two online-tagging bugs plus a status-UX overhaul. Prompt answers given while a scan was running were lost on refresh: the response task sat in the queue behind the busy OnlineTagThread, so the cache kept the prompt. Now the drain loop removes answered prompts from the cache inline (race-free, single-threaded) and marks the fingerprint so the running scan stops re-persisting it; the network re-fetch + write is deferred until the scan releases the thread. A rate-limited lookup looked hung: collect_results could raise mid-pass (budget exhausted, network error) without ever calling finish(), leaving the status row frozen forever. finish() now runs in a finally. The rate-limit status was a static "rate limited ~57s" — a second subtitle writer clobbered the parseable one, and nothing refreshed it during comicbox's blocking sleep. Replaced with two absolute-timestamp countdowns the admin UI ticks down to live: - retry_at: time until the next online request, re-anchored per attempt. - eta: total time remaining, carrying forward the launcher dialog's estimate (ported verbatim to estimate.py), re-estimated on each comic completion and pushed out by rate-limit waits. Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in the pending status_type choices drift), plumbed through Status and StatusController. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit eccf41bc603b118bdbd63f4e4918a30da87140ae Author: AJ Slater <[email protected]> Date: Fri Jun 12 22:39:15 2026 -0700 feat(tagging): admin-orderable online source priority defaultSources order is now run priority end to end. comicbox runs sources in the given order under first-wins, so the admin can choose which source is primary and which is fallback: - Tagging tab renders the source rows in defaultSources order with per-row up/down arrows; enabling a source appends it at lowest priority. A hint explains that the first matching source wins. - The launcher dialog normalizes its checkbox selection to the admin's priority order before starting a session (checkbox v-model records click order, not priority). - Serializers validate defaultSources / sources: known names only, deduped, order preserved (unknown source → 400). - explicit_id / session settings pass ordered tuples to comicbox's retyped OnlineLookupSettings.sources. Co-Authored-By: Claude Fable 5 <[email protected]> commit 02869ddf4d027c325637dd41081199c59f1141ad Author: AJ Slater <[email protected]> Date: Fri Jun 12 17:06:25 2026 -0700 add METRON_INFO to default formats to write commit a214c9c93cf679194beeb5c42eb66deea933a3da Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:36 2026 -0700 update deps commit 65a294362171bded71547430dd51b322ea83dcef Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:08 2026 -0700 don't make abort button red commit de49b5dd9529446ab34a7468bda694fae39c7a00 Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:52:14 2026 -0700 fix(frontend): remove duplicated Tag Write Errors sidebar item admin-menu.vue contained the same tag-write-errors CodexListItem twice (re-added in a later squash while the original stayed), so any error rendered two identical sidebar entries. One item per error kind; the counts live in the admin tables. Co-Authored-By: Claude Fable 5 <[email protected]> commit e0bd853d9ed1991859849e11d85d487409dc1194 Author: AJ Slater <[email protected]> Date: Fri Jun 12 13:43:19 2026 -0700 fix(onlinetag): stop spurious rewrites and prompt loss in online tagging Three interacting bugs broke a single-comic tagging run: - Pass-1 and prompt-resolution writes now require result.matched (new comicbox flag): unmatched results carry the comic's merged existing metadata, so gating on truthy tags re-wrote files with no new information — which also triggered a re-import mid-session. - Pending prompts and tag-write errors move to a dedicated 'tagging' cache (codex.cache.tagging_cache). The session_cache docstring claimed key namespacing protected them from cache.clear(), but Django's file-based clear deletes everything: every import that changed anything (importer finish), Library/Group CRUD, and startup wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'. - Prompt resolution builds its replay session with defer_prompts=True so the preloaded resolution is actually consulted (comicbox only installs its cache-reading selector with defer or a handler; without it an ambiguous re-search fell through to comicbox's interactive CLI prompt inside the daemon). A fingerprint miss — the re-search returned different candidates — now re-queues the fresh deferred prompt instead of dying silently. Co-Authored-By: Claude Fable 5 <[email protected]> commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:45:22 2026 -0700 feat(admin): auto-enable a tagging source when its credentials are first saved Saving credentials for a source that previously had none now adds it to default_sources via the auto-saving draft, so a freshly configured source works without a second checkbox click. Re-saving credentials for an already-configured source leaves the enable checkbox untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:38:11 2026 -0700 Squashed commit of the following: commit 9543008f24441b6b2992f0c818d019f42135f7df Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:21:24 2026 -0700 dockerignore benchmark dir commit 00134a40fe1c37d96e64066735cb42cc03f463c8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:20:24 2026 -0700 mv fixutres to benchmark dir commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:54 2026 -0700 change import path for mock comics commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:05 2026 -0700 ignore fixtures dir commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:11:25 2026 -0700 common subdir for mock & benchmark commit cd768965a544069fb623f9798a2ef7f472016586 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:01:01 2026 -0700 move benchmark import out of ci commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:52:32 2026 -0700 v2.0.0a10 commit 43b6042bca9c546d49268a52002efa4db69651c4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:51:47 2026 -0700 update deps commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:39:57 2026 -0700 fix typecheck and ty warnings Merge implicit f-string concatenations, add missing @override on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:28:58 2026 -0700 remove bad news line commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:18:54 2026 -0700 adapt to comicbox 4.0.0a3 API Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]> commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251 Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:55:25 2026 -0700 update deps commit 919fad63cf5790b82275e5f20207e38f65934a6c Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:48:25 2026 -0700 update deps commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e Author: AJ Slater <[email protected]> Date: Thu Jun 11 14:00:48 2026 -0700 poller: apply discarded .only() projection in manual poll path _handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]> commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 07:52:01 2026 -0700 admin: fix stale Last Poll column after websocket table refetch DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <[email protected]> commit 188dd22775d6beb284b6ddafb3629dcb0affbccb Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:18:16 2026 -0700 edit news for brevity commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:17:05 2026 -0700 NEWS: browser performance entry for 2.0.0 Co-Authored-By: Claude Fable 5 <[email protected]> commit 994d6a586c249376827c1d72494071b4281528fa Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:15:27 2026 -0700 news: v2.0.0 performance section for the import speedups Co-Authored-By: Claude Fable 5 <[email protected]> commit cbb07c980bd5bb317e7a506bec81ba8f66415163 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 cards: Max bookmark timestamp aggregate instead of JSON array + parse loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <[email protected]> commit 01a9394d8ecaaa22e72885b4257545a82220b3d4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 breadcrumbs: batch the folder ancestor chain in one query The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <[email protected]> commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <[email protected]> commit 578807ab0294391095e2f6b6fe3d1b1540f643a4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:55:07 2026 -0700 search: materialize the FTS match set once per browse request The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <[email protected]> commit e56b7eb2dd54cab402041de418832f865269540c Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 lint: format follow-ups for the order/cover perf commits Co-Authored-By: Claude Fable 5 <[email protected]> commit ca67645e4bf757568021e19781d9179ae599bda2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 browser: move the last-route write off the read path; settings row fetched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <[email protected]> commit 94ef67e41554869cfe4c19494cea7e422ff5d30b Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:10:00 2026 -0700 covers: batch cover mtimes post-pagination instead of a second subquery cover_mtime was a .values('updated_at') Subquery over the identical correlated cover queryset that already computes cover_pk — two full executions of the most expensive expression per card (measured 40-49% of the card query across publisher/folder/search pages). Drop the annotation; resolve the representative comics' updated_at after pagination with one indexed pk__in batch over the page's cover_pks and attach it to the cached instances (the serializer iterates the same objects). Custom-cover mtimes keep their cheap rowid subquery and still win the coalesce. _CoverMtimeCoalesce is gone with the annotation that needed it. Responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 browser: zero-pad from page rows; skip dead card serialization in table mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <[email protected]> commit d1789eda2ac5a4351c3039ee42d4677db1c669c6 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 table mode: evaluate the collection page query once, not twice compute_collection_intersections read the page pks via values_list('pk'), a clone that re-executed the entire annotated/ grouped/sorted collection query; the serializer then ran the original queryset again. Iterating the queryset itself primes its result cache, which the serializer reuses — the heaviest table-mode query now runs once per request. Co-Authored-By: Claude Fable 5 <[email protected]> commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 comic listings: stop aggregating order aliases; GROUP BY pk only Two GROUP BY-width fixes for Comic-model querysets: - _alias_filename aggregated Min/Max over the comic's own single path, which turned the alias into an aggregate and dragged every selected column into GROUP BY inside the listing and both cover subqueries. A comic is its own file; use the bare expression (as annotate_comic_extra_specials already did). A filename-ordered folder browse measured 135s before, 0.6s after, identical rows. - The ids JsonGroupArray made Django compute GROUP BY over all ~50 selected Comic columns including TEXT blobs. Force GROUP BY id — every other column is functionally dependent on the pk, so groups are identical and SQLite sorts a one-column key (25-34% off the books query). Not applied to the cover path, where the forced literal column doesn't survive nested-subquery aliasing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 browser: UNREAD filter must not hide comics other users finished The UNREAD arm was Q(bookmark=None) | (mine & unfinished): bookmark=None matches only comics with no bookmark from ANY user or session, so a comic finished by someone else (or a stale anonymous session) matched neither arm and silently vanished from my unread listing (reproduced: 50 foreign finished bookmarks shrank my unread count by 50). UNREAD now probes 'I have no finished bookmark on this comic' with a correlated NOT EXISTS scoped to the requesting user/session. It correlates on the queryset's comic pk, so on collection querysets it binds to the same joined comic row as the other filters in the combined .filter() call — group rows still require ONE comic that satisfies every condition (pinned by test). New comic-leading composite indexes (comic,user)/(comic,session) keep the probe off the user index: under stale sqlite_stat1 the planner flipped a user-scoped probe onto it — a measured 17s vs 27ms plan. EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user (comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged; single-user responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit cad52d79e8242e3b69e4cbd604757f2c7faab728 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 metadata: stop GET writing m2m through tables and corrupting tags _copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True) for every real m2m field on the Comic path — obj is a live saved instance (the 'values dicts' comment was stale), so a GET /metadata DELETE+INSERTed up to 12 through tables, and a multi-comic selection permanently rewrote the first comic's tags to the selection's intersection (reproduced: 339 characters silently became 43). Serve the intersection queryset through the instance's prefetch cache instead: RelatedManager.get_queryset consults it keyed by field name, so the serializer reads the already-optimized queryset query-free. Response verified byte-identical; the 38-query identifier N+1 (the post-write re-read discarded select_related) disappears with it. Co-Authored-By: Claude Fable 5 <[email protected]> commit 71b52749a3b24a8493c6803f0c3cda04466280d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 browse: bind favorite collection code as str so cachalot can cache Binding the Collection StrEnum member itself made cachalot's parameter-type check raise UncachableQuery for every query the favorite subquery landed in — silently uncaching all browse queries. Repeat table-mode browses re-paid ~10s of SQL every time; with .value they serve from cache (16-22ms, 2-3 queries). The bound SQL value is identical either way. Co-Authored-By: Claude Fable 5 <[email protected]> commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 metadata: pk subquery instead of inlined IN lists; count the filtered relation Two fixes to the intersection queries behind GET /metadata: 1. _get_comic_pks returned a materialized frozenset, inlining one bound SQL variable per comic per union arm. With 12 m2m arms, any collection over ~2,730 comics exceeded SQLite's 32,766-variable limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and 17.8k comics). Passing the distinct .values(pk) queryset keeps the statement constant-size (36 params); the same union measured 101ms at 8,576 comics. num_comics is computed once via COUNT and threaded to both consumers. 2. _get_fk_intersection_query counted the hardcoded 'comic' reverse m2m even when the filter traversed the main_*_in_comics FK back-relations — a semantically wrong HAVING over the unfiltered relation that also forced a separate unindexed join (main_team measured 2,069ms -> 12ms counting the filtered relation). Co-Authored-By: Claude Fable 5 <[email protected]> commit 5beb474835c7375f4b726eca63aa05f2ef8b9966 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 arc browse: demote story-arc chain joins to INNER for StoryArc queries With only codex_comic demoted, SQLite planned arc search browses from the arc side and built a cartesian product against the FTS match set: arcs root with a common search term measured 156s (main query 150.8s). Demoting codex_storyarcnumber and the M2M through table lets the plan start from the FTS/comic side (36ms). Scoped to StoryArc-model querysets: there the chain IS the collection relation, so NULL-extended rows are exactly the empty collections force_inner_joins exists to drop. On other models these tables appear as table-view annotation LEFT JOINs, where demotion would drop rows whose comics have no arcs. Co-Authored-By: Claude Fable 5 <[email protected]> commit 1434a5a771fc7445472cb52726f4e86c645df999 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 cover subqueries: join FTS rank only when ordering by search_score The direct fts_q join exists solely to populate codex_comicfts.rank for the cover subquery's search_score ORDER BY, but it was applied on every search regardless of order key — and the MATCH re-executed once per correlated cover evaluation. The pre-materialized fts_sq membership test already filters correctly. A name-sorted search browse measured 135s before, 6.5s after (byte-identical rows); rank-ordered searches keep the join and pick the same top-ranked cover as before. Co-Authored-By: Claude Fable 5 <[email protected]> commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:40 2026 -0700 table mode: keep intersection-sort subqueries out of GROUP BY RawSQL.get_group_by_cols() returns [self], so the correlated intersection-sort subqueries landed in the GROUP BY of aggregated collection queries. Folder is the one collection model without a forced-GROUP-BY entry, so SQLite evaluated the subquery once per pre-aggregation joined row — at folders root the ancestor M2M fans out to every descendant comic (366s request; each of the two queries measured ~256s alone, 85ms after). Same mechanism as the _CoverMtimeCoalesce fix, recurring through a different expression type. _IntersectionSortRawSQL overrides get_group_by_cols() to return []: the subquery's only outer reference is the collection pk, which is always grouped, so grouping is unchanged (verified row-identical). Co-Authored-By: Claude Fable 5 <[email protected]> commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:14:18 2026 -0700 fix librarian status feed order: cursor pagination discarded the order_by The sidebar progress feed showed import phases scrambled — read next to the search statii, query below create. The viewset ordered active rows by (preactive, active, pk), but DRF CursorPagination re-orders the queryset with its own ordering, so the v4 admin refactor's plain pk default (545fc2aa9) silently replaced the registration-stamp order with row-creation order. LibrarianStatusCursorPagination orders the active poll by (preactive, active, pk) — matching start_many's padded registration stamps, with NULL preactive sorting first for rows started without pre-registration (the cover thread case). The Jobs tab history view keeps the pk default. The active feed is ~20 rows against a 200-row page, so the cursor filter never engages the nullable fields. Regression test creates status rows with pk order deliberately opposite the registration stamps and asserts both routes. Co-Authored-By: Claude Fable 5 <[email protected]> commit 396312c6fe9964b36f895dd6a2c8bb84757191c3 Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:55:39 2026 -0700 import timing: don't double-count sub-steps in the phase share total _run_phases reuses the timed_step helper instead of an inline perf_counter copy, and _log_phase_times sums only top-level phases — dotted "phase.step" entries already accrue inside their parents, so including them inflated the total and shrank every phase's share. Co-Authored-By: Claude Fable 5 <[email protected]> commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:47:39 2026 -0700 metadata edit: full-size Protagonist label, live menu, stale-pick clear The Protagonist row inherited the old Main sub-row styling (smaller label, dimmed row) — it is a first-class row now, styled like its tag siblings. The choice menu already tracked the entered character/team names live (it is name-based, no pks involved); what was missing is that removing the picked name left the selection dangling — a watcher now clears it. A protagonist seeded from the DB but absent from the entered lists still survives load untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:28 2026 -0700 metadata: unify main_character/main_team as a single Protagonist field Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is replaced by one "Protagonist" row above the tag table showing the mainCharacter and/or mainTeam chips — both only in the should-never-happen case where both are set. Each chip carries its own browser filter key (characters vs teams) via a per-item filter override in MetadataTags. En route this fixes mapTag's sticky filter: the loop reassigned the filter param, so every row after the first reused the first row's filter key. Edit: the two Main Character / Main Team sub-selects collapse into one Protagonist select offering the entered characters and teams; the pick is sent as the comicbox "protagonist" patch field on write, and the importer resolves it back to main_character or main_team (nulling the other) on re-import. Co-Authored-By: Claude Fable 5 <[email protected]> commit 632a238590f188ee249c4a8164a6936bcdfc779c Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:16 2026 -0700 importer: stop clearing an unchanged protagonist on update imports The fk link phase set main_character/main_team to None whenever a comic's remaining LINK_FKS dict lacked a "protagonist" key. The query prune pops that key when the protagonist already matches the DB, so re-importing a comic with any *other* FK change silently NULLed its protagonist. Updates now only touch the protagonist fields when the metadata explicitly carried one (set-or-clear), matching the absent-means-untouched semantics of every other simple FK tag; creates keep both fields defaulted. Also: a name that is both a Character and a Team now links only main_character — both fields filled is invalid. Regression test drives query → prune → link with only scan_info changed and asserts the protagonist survives (it NULLed before). Co-Authored-By: Claude Fable 5 <[email protected]> commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead Author: AJ Slater <[email protected]> Date: Wed Jun 10 16:16:30 2026 -0700 import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s) TimestampUpdater filtered every collection model through a Count(DISTINCT comic) alias to exclude empty collections — a per-row GROUP BY join aggregate costing ~1.5s per import even with zero deletions, in every import scenario. The count was redundant for two of the three OR branches: a comic__updated_at join match implies the collection has children, and force-updated pks were exempted from the child filter anyway. Only the custom-cover branch can match an empty collection, so it now carries its own join-presence test (comic__isnull=False) instead. Also replaces the fetch-instances + CASE-WHEN bulk_update round trip with a batched .update(updated_at=Now()) over the distinct pks. Also corrects the stale IMPORTER_FILTER_BATCH_SIZE comment: the binding constraint on OR chains is SQLITE_LIMIT_EXPR_DEPTH (1000), not the old 999-variable cap (modern SQLite allows 32766), so 900 remains correct for the remaining chain users. Benchmark, 1000 mock comics: delete phase 1.46s -> 0.10s on fresh and below 0.1s on reimport (5.9s total). Re-stamp semantics covered by tests/importer/test_move_timestamps.py. Co-Authored-By: Claude Fable 5 <[email protected]> commit bac8ee8af974bec78781f5c764e543f8429540a1 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:51:29 2026 -0700 import perf: bulk_update only changed comic fields (update step 8x) update_comics passed all ~40 BULK_UPDATE_COMIC_FIELDS to bulk_update regardless of what changed — CASE-WHEN compilation and parse scale with rows x fields, so a force-reimport whose comics needed nothing but a stat refresh paid the full-width rewrite: 4.4s of the 7s remaining reimport time, almost all inside the two bulk_update manager calls. Collect the union of fields actually applied per batch (proposed md diffs + FK link fields) and update that union plus the presave-derived/timestamp set (stat, size, date, decade, age_rating_metron_index, updated_at). Fields outside BULK_UPDATE_COMIC_FIELDS are dropped as before. Benchmark, 1000 mock comics: forced reimport 10.0s -> 7.0s, update_comics sub-step 4.4s -> 0.55s. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55401e6d372da3a9bf1c8b1cedeecbfd6fa06a27 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:46:01 2026 -0700 import perf: narrow existing-row queries with a selector IN, not Q-OR chains Multi-key existing-row lookups (query phase query_existing_mds, link and create pk-map builders) built per-key AND/OR Q chains — one Django filter resolution per leaf and a giant OR for SQLite to parse, per 500-900-key batch. Profiling a fresh import showed ~40% of its time in this filter machinery. Both consumers already match exact key tuples in Python from the fetched values, so the chain only ever bounded the fetch. Replace it with a single indexed IN on the model's most selective non-null key rel (MODEL_SELECTOR_REL_MAP: imprint/series name, volume series name, credit person, identifier key, story-arc name, folder path). Superset rows — the same selector value under a different parent — are harmless extra map entries. Keys with a None selector value fall back to the old chain (selector columns are non-null, so normally none). Also: read phase sub-instrumented (extract vs aggregate) and the benchmark gained --profile (cProfile per scenario). Benchmark, 1000 mock comics: fresh 12.1s -> 9.4s (query phase 2.6s -> 1.0s, link 2.5s -> 1.7s); reimport 11.8s -> 10.0s (query 3.4s -> 1.6s). Tag/link counts byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 610c51b67580006567b168e0e260c6c6097433d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:11:28 2026 -0700 import perf: skip archive opens when the fs stat snapshot is unchanged The fs prefilter compared disk mtime against the stored *embedded* metadata mtime, which is older than the file's mtime for any archive copied into the library after tagging — so the common at-rest comic never skipped and every modified-event re-poll opened its archive (an unrar spawn per CBR) just for comicbox to no-op. Reuse the stat call to also compare disk (mtime, size) against the stored Comic.stat snapshot — the poller's own modified test — and skip the worker entirely on a match. Paths with no stored embedded mtime still always survive, so a library imported with the import-metadata flag off picks up tags after the flag turns on. Benchmark, 1000 mock comics: noop poll 1.0s -> 0.2s (7.6s at baseline); zero archive opens. Reimport unaffected (force path never populates the prefilter's mtime map). Co-Authored-By: Claude Fable 5 <[email protected]> commit 06bec468cd68b423efb8f2b9da01be0f250c9d34 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:06:59 2026 -0700 import perf: skip true no-op stat-only updates (noop poll 7.6s -> 1s) _envelope_deltas included metadata_mtime unconditionally, so every file whose tags comicbox skipped still routed through the stat-only update: the comic row rewritten, its covers purged and re-queued for generation, and the import counted as changed -> cache.clear() + LIBRARY_CHANGED broadcast. A modified-event burst over an unchanged library rewrote every row and regenerated every cover. Compare metadata_mtime against the stored value, and skip the stat-only routing entirely when the envelope is unchanged AND the on-disk (mtime, size) still matches Comic.stat — the same equality the poller's SnapshotDiff uses, so a skipped file is exactly one the poller would not re-emit (no stale-stat re-poll loop). Touched files (stat moved, content same) keep today's full stat-only path, which deliberately refreshes Comic.stat via presave. Benchmark, 1000 mock comics: noop poll 7.6s -> 1.0s with "No updates necessary" (the remaining ~0.9s is archive opens for the comicbox mtime check). Fresh and reimport unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 488af595b62d643f130098f38eb2d0d8760230cd Author: AJ Slater <[email protected]> Date: Wed Jun 10 13:40:39 2026 -0700 import perf: batch-resolve comic fk links (fresh import 3x) Comic creation resolved every FK link with objects.get() per (comic, field) — folder, two protagonist lookups and ~one per FK field, ~14 single-row SELECTs per created comic. create_comics was 59% of a fresh import. prepare_fk_link_instance_maps now runs once per chunk after the FK create/update steps: one batched query per referenced field over the distinct key tuples (reusing the link phase's pk-map builders), plus in_bulk for instances. get_comic_fk_links stitches from the maps. Instances, not pks, so Comic(**md), the update path's setattr/default handling and presave's age_rating dereference are unchanged. Missing rows still raise DoesNotExist and skip the comic. The parent-folder map is also library-scoped now; the old Folder.objects.get(path=...) could MultipleObjectsReturned when two libraries shared a directory path. Also fixes a latent crash exposed by the new path: _build_pk_map_multi_key sorted key tuples that mix None with values (volume year/number_to, story-arc numbers) — TypeError when two keys first differ at a None position. Sort key now orders None last. Benchmark, 1000 mock comics: fresh 35.1s -> 12.1s; create_comics sub-step 21.7s -> 0.25s (map building 0.5s). Cumulative since baseline: fresh 2.9x, forced reimport 131.6s -> 11.8s (11x). Sub-step timing for create_and_update phases included (timed_step helper; dotted names excluded from share totals). Co-Authored-By: Claude Fable 5 <[email protected]> commit 94dd32a4ab9a74e2fd250aea22340c41ff35b5e5 Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:59:44 2026 -0700 import perf: compare fk prune links as values tuples (query 12s -> 3.7s) The fk prune select_related'd only the directly referenced FK fields, then walked each key rel via getattr chains — every hop past depth one (volume -> series -> imprint -> publisher) lazy-loaded a single-row SELECT per comic, and the protagonist check lazy-loaded main_character and main_team likewise. Query every referenced field's key rels in one batched values_list and compare row slices against the proposed key tuples. Also makes the protagonist prunable when it's the only referenced field (it was skipped before because it isn't a concrete FK for select_related). Benchmark, 1000 mock comics: reimport 23.9s -> 13.2s, query phase 12.2s -> 3.7s. With the m2m prune rewrite, forced reimport is 10x faster than the 131.6s baseline. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55ceae11efd87c421f59b616ec65bf64882d4a8e Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:53:01 2026 -0700 import perf: scan m2m prune off the through tables (10x) The m2m prune walked prefetched related objects per comic and read key attrs through instance getattr chains — for credits, identifiers and story-arc-numbers each FK in the key tuple lazy-loaded with a single-row SELECT per through row. A forced reimport of 1000 comics spent 121s (92%) in this phase. Scan the through tables directly instead: one values_list query per (field, batch) with the key attrs joined in SQL, pruned via set ops against the proposed links. No comic or related-model instances, no lazy FK loads. get_through_model moves from the link phase to const so the query phase can share it. Same pruning semantics (kept/deleted/FTS-existing/moved…
ajslater
added a commit
that referenced
this pull request
Jun 16, 2026
commit 15a6b8c58bd31e2d3ce558db9a35c3bf3d4be623 Author: AJ Slater <[email protected]> Date: Mon Jun 15 21:15:28 2026 -0700 version 2.0.0a14 commit 3ed935b45d978fe81118996285111a57fc340067 Author: AJ Slater <[email protected]> Date: Mon Jun 15 21:14:52 2026 -0700 update deps commit 4201d0747a3f43ee7aa5298ce229c07ab5b02eac Author: AJ Slater <[email protected]> Date: Mon Jun 15 20:03:44 2026 -0700 fix(lazy-import): stop the tag-icon hover loop on no-metadata comics Hovering a browser card's tag icon triggers a lazy metadata import gated on whether the comic "has metadata". Both the frontend hover gate and the lazy-import worker keyed off Comic.metadata_mtime — but that field is comicbox's *embedded* metadata mtime, which stays NULL forever for an archive with no metadata file. So a no-metadata comic always reported hasMetadata=false; after each import the browser refreshed, the card remounted, the cursor was still over the icon, and the import re-fired endlessly. Add a dedicated Comic.metadata_imported_at marker, stamped by a single path-scoped bulk UPDATE at import finish for every comic a forced/lazy pass touched (covers the SKIPPED no-metadata comics that never reach the per-comic write path), gated on force_import_metadata so ordinary watcher imports don't rewrite it. The worker gate and the has_metadata annotations (browser + reader) now read the marker, with metadata_mtime kept as a fallback. The field + backfill (COALESCE(metadata_mtime, updated_at), so existing comics don't regress to un-imported) is folded into the consolidated v2 migration 0043. metadata_mtime is left untouched — it is fed back to comicbox as the skip-re-extraction baseline, so overloading it would corrupt that protocol. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit a665bd183640a9abe33c6b083a784c3dcdfa4533 Author: AJ Slater <[email protected]> Date: Mon Jun 15 19:45:54 2026 -0700 rename batch to session commit 00f29ab33681432f6d3bb25e4bea46a5328dbd03 Author: AJ Slater <[email protected]> Date: Mon Jun 15 19:44:28 2026 -0700 feat(onlinetag): refine match review dialog for pause/resume workflow - Expand the first match panel by default so the admin can act at once. - Replace the dead "Abort Session" button (abortSession never existed) with a "Pause" button wired to pauseSession, matching the Tagging tab. - Show "Cancel" instead of "Dismiss" while the session is unfinished (active or resumable), derived from the store snapshot. - Add a tooltip to the sidebar "Matches to Review" item. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c1beab7ecbef3a5ce6fa765551d06141e6b03d39 Author: AJ Slater <[email protected]> Date: Mon Jun 15 02:06:47 2026 -0700 feat(onlinetag): default delete_original to True Flip the ComicboxTaggingDefaults.delete_original field default and the 0043 seed so newly tagged comics replace their originals by default. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 910fa88989256d8ef6f01b2a501c9e4f81e25cfa Author: AJ Slater <[email protected]> Date: Mon Jun 15 02:00:01 2026 -0700 fix(tagging): scroll long comic names instead of truncating The Online Tagging Status table truncated long comic names with an ellipsis, putting them out of reach. Replace the path cell's overflow/text-overflow ellipsis with overflow-x: auto so a long name scrolls horizontally within its existing width. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 05d44ba3b0f808cdb8a780da05f147333a486625 Author: AJ Slater <[email protected]> Date: Mon Jun 15 01:59:23 2026 -0700 fix(onlinetag): enable tag-by-id from reader metadata screen The reader builds its metadata book from books.current, which carries a pk but no ids array. The online-tag dialog's idTaggable gate requires book.ids.length === 1, so the tag-by-id tab never appeared for a single comic opened from the reader, while browser card items (which include an ids array) worked. Derive ids from the pk to match the card shape. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8afde179d0dbc5a9e35d3523afb88e1c4daf9ad0 Author: AJ Slater <[email protected]> Date: Mon Jun 15 01:45:28 2026 -0700 Squashed commit of the following: commit a6c315e224b2b6170d437099736afcbbf97e5479 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:31:51 2026 -0700 update deps commit 98f55c7bf96be3d4c56c3fb12b10e95270a39cbc Merge: 5ad35f1cd 1274bad66 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:31:20 2026 -0700 Merge branch 'develop' into pre-release commit 5ad35f1cd4d7b40935e4ca19c018f312bdef887b Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:35 2026 -0700 Squashed commit of the following: commit 5e6f4dfc55ee4ad3f8fab48b7b33ffe02f17d8a4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:21 2026 -0700 update version 2.0.0a13 commit 59d607d6c886cccb093fbc07dea5a2ef97ab1dc0 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:25:51 2026 -0700 test(onlinetag): exercise replay path in unmatched-write test The candidate carried issue_id 123, routing the resolve through the explicit-id fetch (real fetch_tags_by_explicit_id) instead of the replay session the test sets up and claims to verify. That crashed on the empty touched .cbz before any assertion ran. Drop the issue_id so the candidate falls back to the replay path guarded by _first_tags. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit dfa2349e8b821fbaa93b53fb4639bd2d66db8522 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:19:14 2026 -0700 fix(mail): ASCII hyphen in noqa comments to dodge complexipy panic complexipy's extract_comment_marker slices noqa comment text by a fixed byte offset (16); a multi-byte em-dash straddling that offset lands mid-char and panics the Rust core, aborting the whole run. Swap the two em-dashes in the noqa comments for ASCII hyphens. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 1799ce937559d62a667a0bf66f819f1593c93c5e Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:16:34 2026 -0700 decomplexify test for radon commit 6213c0afe641942e9e1d28f090970c6da524ceb4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:14:19 2026 -0700 remove unused ty ignore commit dc59f1dc9560e177f9e0c36ea4552208014746d6 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:13:38 2026 -0700 remove redundant definition commit 5d7a204d5069783c8638bc8f76f84bf15840239a Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:10:36 2026 -0700 Squashed commit of the following: commit 1274bad663bf382d76453e65a6fc935b8b3d4559 Merge: a8aca7b33 72fad6861 Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:57:33 2026 -0700 Merge branch 'main' into develop commit 72fad6861b0af14731847b5444f097fdec699fda Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:06:27 2026 -0700 v1.12.8 (#781) commit a8aca7b3341e7afdee8e99829be88d77e4d988c5 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:50:21 2026 -0700 fix(test): call super().tearDownClass() in BaseTestImporter Skipping super() left the class-level atomic from setUpTestData uncommitted-but-unrolled-back, leaking the codex_init() "admin" superuser into later test classes and causing UNIQUE constraint failures (e.g. test_bookmark_filter_isolation) in full-suite runs. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit d7570ecf99e1a15449dec2f7a0550122b9d405df Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:16:59 2026 -0700 remove gitignore commit be211c5724d5d7cc25ca367813fa34244097f372 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:12:03 2026 -0700 fix(types): narrow OPDS redirect settings to Mapping for ty Replace a stale `ty: ignore[no-matching-overload]` with an isinstance guard so `settings` is a Mapping before `_copy_params_into`, clearing the `invalid-argument-type` diagnostic from `make ty`. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5dd3a0dd9feb18aa4b13c15b000c048befdeceef Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:09:05 2026 -0700 update devenv commit e5a2a41881d35e2d8716bd6dfefdd64fa3792b5c Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:07:34 2026 -0700 shfmt commit 1f867c1497e4edc14315d63c7eeb8c3e2b0f07bd Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:05:38 2026 -0700 shfmt commit c5366787b56212e12a69c5aaab1776474962a31b Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:02:56 2026 -0700 fix(bookmark): scope read/unread filters per user (v1.12.8) The UNREAD filter used Q(bookmark=None), a multi-valued relationship test for "comic has no bookmark from anyone" rather than "no bookmark for me". Once any user finished a comic it dropped out of every other user's (and anonymous visitors') unread view. UNREAD now negates the per-user finished predicate (~(my_filter & finished=True)), compiling to a per-user NOT-EXISTS subquery. Also harden get_my_bookmark_filter: an anonymous visitor with no established session_key resolved to `session_id IS NULL`, which matches every authenticated user's bookmarks. Return a match-nothing filter instead; a session key is minted only when a bookmark is written. Adds tests/test_bookmark_filter_isolation.py covering cross-user and anonymous isolation for READ/UNREAD/IN_PROGRESS. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 541907b6708dfbbde197432960c6adc6dcf44b02 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:44:58 2026 -0700 fix(frontend): convert v-dialog em widths to px so dialogs aren't slivers Vuetify 4's v-dialog defaults to location 'center center', which activates the connected overlay location strategy. That strategy caps width with parseFloat(props.maxWidth), so "20em" parses to 20 and the em is dropped, collapsing dialogs to a ~20px sliver. Convert every em-based max-width/ min-width on a v-dialog (or forwarded via $attrs) to unitless px at 16px/em. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5918d0b621cd635f3de32f142d4dff7ec9c45242 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:41:25 2026 -0700 move VPullToRefersh out of labs commit 57d5cf3888c2dc0aab55e46447b41a459d37003a Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:32:39 2026 -0700 update devenv and deps commit cd944f0e4cf7044ba8e72cb6f90539ca892adb72 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:25:16 2026 -0700 Squashed commit of the following: commit b5874b67774af29f43603f01690a0ca97b43c46a Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:24:54 2026 -0700 update deps commit e0a16dff8e66c357b2d2aa5e4ff3ea7decce4572 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:23:51 2026 -0700 remove pytest-gitignore commit 35b84324300352499f3c455aa43201c3f475cd21 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:20:46 2026 -0700 update version to 2.0.0a12 commit e3e1167851960a42fef9a5ceba3b3b5470f7a56c Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:19:54 2026 -0700 Fix basedpyright error on Folder FK access in moved-parents test The Folder model annotates parent_folder as ForeignKey explicitly, so django-stubs doesn't synthesize the implicit parent_folder_id attribute. Compare the FK object instead (Django compares models by pk). Pre-existing since ce1fd81ad; behavior unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 0ee9c966f9d1dcfd84d36e6077778fd9354d9574 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:45 2026 -0700 update deps commit 38f527b57ce30be0f162d7aae2bf51d938e3141b Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:39 2026 -0700 ignore benchmark dir commit 7007c109accb1654c5b61580336d71cedfb5ce3f Author: AJ Slater <[email protected]> Date: Sun Jun 14 17:56:39 2026 -0700 Add online tagging status table with live progress and session controls Admin Tagging tab now shows a live status table for the online tagging batch: per-comic state, per-source rate-limit countdowns, batch ETA, and match-review status. Backed by a daemon-published session snapshot in the tagging cache, exposed via GET /api/v4/admin/tag-sessions/snapshot and refreshed over the existing task.progress websocket. - Snapshot: session_snapshot.py builds a capped, JSON-safe per-comic snapshot from OnlineTagOutcomeStats + pending prompts; published throttled from the runner, frozen inactive on finish, cleared at startup / janitor. - Session lifecycle: pause / resume / abort / dismiss controls in the table, wired to the online-tag store and admin endpoints. - Match-review fixes: - direct-id fetch for explicit picks (no rate-limit-prone re-search drift) - record user_matched / user_skipped outcomes, overlaid on the snapshot - client guard against resurrecting just-answered prompts in the dialog - optimistic local overlay so resolutions show during rate-limit stalls Tests: backend snapshot / resume / session-manager / janitor suites; frontend store + status-table specs. ruff/ty/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit ce1fd81adb4823ac454950ab66905a38e3f6d9e1 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:56:39 2026 -0700 Fix ScribeThread crash on folder move under untracked parent _get_move_create_folders_one_layer returned bare path strings, but bulk_folders_create expects key tuples (MODEL_REL_MAP[Folder] == (("path",), ...)) and iterates each entry to extract the path. Bare strings were iterated character by character, producing single-char paths like PosixPath('K') whose stat() raised FileNotFoundError and crashed the ScribeThread during import. Wrap the path in a tuple to match the contract, and add a regression test covering a folder move whose destination parent isn't yet a tracked Folder. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 265988a4b44ba7ebe9c810bedc11a42796853d90 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:30:44 2026 -0700 Delete duplicate migration 0044 that broke test-DB setup The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e6), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c31c61e6175d24d9e1dde1835c7235c367c251c9 Author: AJ Slater <[email protected]> Date: Sat Jun 13 02:00:11 2026 -0700 fix some kind of merge issue commit 92e526940e96b6ea9679363159395270233e75f2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:55 2026 -0700 Squashed commit of the following: commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:29 2026 -0700 refactor(complexity): clear radon cc rank-C findings bin/lint-complexity.sh reported three rank-C functions; extract helpers to drop each below the gate (radon cc --min C now empty). - AdminTagByIdView.post: pull primary-identifier parsing and merge extra-id resolution into _resolve_primary / _resolve_extra_ids. - OnlineTagSessionManager._apply_resolution: extract _handle_unresolved and _enqueue_resolved_write. - test_update_protagonist: extract repeated protagonist assertions into a helper (radon counts each assert as a branch). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 55bb659fcc8c02e73c20a34823652a4f81be1d56 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:46:04 2026 -0700 refactor(migrations): squash 0044 into 0043 Fold the merge_all_sources field into the ComicboxTaggingDefaults CreateModel in 0043 and drop the standalone 0044. makemigrations --check reports no drift. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 7a988737a63286cc85396346579e124e5fff8e5b Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:44:59 2026 -0700 feat(onlinetag): merge metadata from all sources Add an opt-in "merge all sources" mode that queries every enabled online source per comic and merges the results (comicbox first_wins=False) for maximum tag completeness, instead of stopping at the first match. - Admin tagging defaults: merge_all_sources boolean (default off), settable as the default and overridable per scan. - Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate + rate-limit warning multiply calls by source count (kept in sync with estimate.py) so the doubled API cost is visible. - By-ID: chips input accepting multiple URLs/ids; merging fetches two explicit ids (one per source) and merges them. Source select shows only when a token can't be auto-identified (bare number). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:34:56 2026 -0700 update deps commit 473221d7a09403b1848f202d55902b71e31319c2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:22:58 2026 -0700 fix(lint): clear ruff/pyright findings; fix benchmark import path - tests: add docstrings, name magic estimate values, use pytest.raises and out-of-class exception message - onlinetag: collapse implicit f-string concatenation in log call - benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*) which was a real runtime ImportError in library generation Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 45a9fde6076a5b0b15be889769e9a55af4b0678a Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:13:49 2026 -0700 refactor(migrations): squash 0044 + 0045 into 0043 Fold the comic-leading composite Bookmark indexes (former 0044) and the LibrarianStatus eta/retry_at fields + JFR status-type choice (former 0045) into the consolidated 0043 migration; delete both files. makemigrations --check reports no changes against the model state. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 22da6b9cd85615f3d7b8e7043818babac313d24e Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:35 2026 -0700 update deps commit 6f27454623e43cccd95253d5dfe66471820c9349 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:15 2026 -0700 fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns Two online-tagging bugs plus a status-UX overhaul. Prompt answers given while a scan was running were lost on refresh: the response task sat in the queue behind the busy OnlineTagThread, so the cache kept the prompt. Now the drain loop removes answered prompts from the cache inline (race-free, single-threaded) and marks the fingerprint so the running scan stops re-persisting it; the network re-fetch + write is deferred until the scan releases the thread. A rate-limited lookup looked hung: collect_results could raise mid-pass (budget exhausted, network error) without ever calling finish(), leaving the status row frozen forever. finish() now runs in a finally. The rate-limit status was a static "rate limited ~57s" — a second subtitle writer clobbered the parseable one, and nothing refreshed it during comicbox's blocking sleep. Replaced with two absolute-timestamp countdowns the admin UI ticks down to live: - retry_at: time until the next online request, re-anchored per attempt. - eta: total time remaining, carrying forward the launcher dialog's estimate (ported verbatim to estimate.py), re-estimated on each comic completion and pushed out by rate-limit waits. Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in the pending status_type choices drift), plumbed through Status and StatusController. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit eccf41bc603b118bdbd63f4e4918a30da87140ae Author: AJ Slater <[email protected]> Date: Fri Jun 12 22:39:15 2026 -0700 feat(tagging): admin-orderable online source priority defaultSources order is now run priority end to end. comicbox runs sources in the given order under first-wins, so the admin can choose which source is primary and which is fallback: - Tagging tab renders the source rows in defaultSources order with per-row up/down arrows; enabling a source appends it at lowest priority. A hint explains that the first matching source wins. - The launcher dialog normalizes its checkbox selection to the admin's priority order before starting a session (checkbox v-model records click order, not priority). - Serializers validate defaultSources / sources: known names only, deduped, order preserved (unknown source → 400). - explicit_id / session settings pass ordered tuples to comicbox's retyped OnlineLookupSettings.sources. Co-Authored-By: Claude Fable 5 <[email protected]> commit 02869ddf4d027c325637dd41081199c59f1141ad Author: AJ Slater <[email protected]> Date: Fri Jun 12 17:06:25 2026 -0700 add METRON_INFO to default formats to write commit a214c9c93cf679194beeb5c42eb66deea933a3da Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:36 2026 -0700 update deps commit 65a294362171bded71547430dd51b322ea83dcef Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:08 2026 -0700 don't make abort button red commit de49b5dd9529446ab34a7468bda694fae39c7a00 Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:52:14 2026 -0700 fix(frontend): remove duplicated Tag Write Errors sidebar item admin-menu.vue contained the same tag-write-errors CodexListItem twice (re-added in a later squash while the original stayed), so any error rendered two identical sidebar entries. One item per error kind; the counts live in the admin tables. Co-Authored-By: Claude Fable 5 <[email protected]> commit e0bd853d9ed1991859849e11d85d487409dc1194 Author: AJ Slater <[email protected]> Date: Fri Jun 12 13:43:19 2026 -0700 fix(onlinetag): stop spurious rewrites and prompt loss in online tagging Three interacting bugs broke a single-comic tagging run: - Pass-1 and prompt-resolution writes now require result.matched (new comicbox flag): unmatched results carry the comic's merged existing metadata, so gating on truthy tags re-wrote files with no new information — which also triggered a re-import mid-session. - Pending prompts and tag-write errors move to a dedicated 'tagging' cache (codex.cache.tagging_cache). The session_cache docstring claimed key namespacing protected them from cache.clear(), but Django's file-based clear deletes everything: every import that changed anything (importer finish), Library/Group CRUD, and startup wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'. - Prompt resolution builds its replay session with defer_prompts=True so the preloaded resolution is actually consulted (comicbox only installs its cache-reading selector with defer or a handler; without it an ambiguous re-search fell through to comicbox's interactive CLI prompt inside the daemon). A fingerprint miss — the re-search returned different candidates — now re-queues the fresh deferred prompt instead of dying silently. Co-Authored-By: Claude Fable 5 <[email protected]> commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:45:22 2026 -0700 feat(admin): auto-enable a tagging source when its credentials are first saved Saving credentials for a source that previously had none now adds it to default_sources via the auto-saving draft, so a freshly configured source works without a second checkbox click. Re-saving credentials for an already-configured source leaves the enable checkbox untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:38:11 2026 -0700 Squashed commit of the following: commit 9543008f24441b6b2992f0c818d019f42135f7df Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:21:24 2026 -0700 dockerignore benchmark dir commit 00134a40fe1c37d96e64066735cb42cc03f463c8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:20:24 2026 -0700 mv fixutres to benchmark dir commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:54 2026 -0700 change import path for mock comics commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:05 2026 -0700 ignore fixtures dir commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:11:25 2026 -0700 common subdir for mock & benchmark commit cd768965a544069fb623f9798a2ef7f472016586 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:01:01 2026 -0700 move benchmark import out of ci commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:52:32 2026 -0700 v2.0.0a10 commit 43b6042bca9c546d49268a52002efa4db69651c4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:51:47 2026 -0700 update deps commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:39:57 2026 -0700 fix typecheck and ty warnings Merge implicit f-string concatenations, add missing @override on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:28:58 2026 -0700 remove bad news line commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:18:54 2026 -0700 adapt to comicbox 4.0.0a3 API Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]> commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251 Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:55:25 2026 -0700 update deps commit 919fad63cf5790b82275e5f20207e38f65934a6c Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:48:25 2026 -0700 update deps commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e Author: AJ Slater <[email protected]> Date: Thu Jun 11 14:00:48 2026 -0700 poller: apply discarded .only() projection in manual poll path _handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]> commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 07:52:01 2026 -0700 admin: fix stale Last Poll column after websocket table refetch DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <[email protected]> commit 188dd22775d6beb284b6ddafb3629dcb0affbccb Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:18:16 2026 -0700 edit news for brevity commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:17:05 2026 -0700 NEWS: browser performance entry for 2.0.0 Co-Authored-By: Claude Fable 5 <[email protected]> commit 994d6a586c249376827c1d72494071b4281528fa Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:15:27 2026 -0700 news: v2.0.0 performance section for the import speedups Co-Authored-By: Claude Fable 5 <[email protected]> commit cbb07c980bd5bb317e7a506bec81ba8f66415163 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 cards: Max bookmark timestamp aggregate instead of JSON array + parse loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <[email protected]> commit 01a9394d8ecaaa22e72885b4257545a82220b3d4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 breadcrumbs: batch the folder ancestor chain in one query The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <[email protected]> commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <[email protected]> commit 578807ab0294391095e2f6b6fe3d1b1540f643a4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:55:07 2026 -0700 search: materialize the FTS match set once per browse request The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <[email protected]> commit e56b7eb2dd54cab402041de418832f865269540c Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 lint: format follow-ups for the order/cover perf commits Co-Authored-By: Claude Fable 5 <[email protected]> commit ca67645e4bf757568021e19781d9179ae599bda2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 browser: move the last-route write off the read path; settings row fetched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <[email protected]> commit 94ef67e41554869cfe4c19494cea7e422ff5d30b Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:10:00 2026 -0700 covers: batch cover mtimes post-pagination instead of a second subquery cover_mtime was a .values('updated_at') Subquery over the identical correlated cover queryset that already computes cover_pk — two full executions of the most expensive expression per card (measured 40-49% of the card query across publisher/folder/search pages). Drop the annotation; resolve the representative comics' updated_at after pagination with one indexed pk__in batch over the page's cover_pks and attach it to the cached instances (the serializer iterates the same objects). Custom-cover mtimes keep their cheap rowid subquery and still win the coalesce. _CoverMtimeCoalesce is gone with the annotation that needed it. Responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 browser: zero-pad from page rows; skip dead card serialization in table mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <[email protected]> commit d1789eda2ac5a4351c3039ee42d4677db1c669c6 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 table mode: evaluate the collection page query once, not twice compute_collection_intersections read the page pks via values_list('pk'), a clone that re-executed the entire annotated/ grouped/sorted collection query; the serializer then ran the original queryset again. Iterating the queryset itself primes its result cache, which the serializer reuses — the heaviest table-mode query now runs once per request. Co-Authored-By: Claude Fable 5 <[email protected]> commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 comic listings: stop aggregating order aliases; GROUP BY pk only Two GROUP BY-width fixes for Comic-model querysets: - _alias_filename aggregated Min/Max over the comic's own single path, which turned the alias into an aggregate and dragged every selected column into GROUP BY inside the listing and both cover subqueries. A comic is its own file; use the bare expression (as annotate_comic_extra_specials already did). A filename-ordered folder browse measured 135s before, 0.6s after, identical rows. - The ids JsonGroupArray made Django compute GROUP BY over all ~50 selected Comic columns including TEXT blobs. Force GROUP BY id — every other column is functionally dependent on the pk, so groups are identical and SQLite sorts a one-column key (25-34% off the books query). Not applied to the cover path, where the forced literal column doesn't survive nested-subquery aliasing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 browser: UNREAD filter must not hide comics other users finished The UNREAD arm was Q(bookmark=None) | (mine & unfinished): bookmark=None matches only comics with no bookmark from ANY user or session, so a comic finished by someone else (or a stale anonymous session) matched neither arm and silently vanished from my unread listing (reproduced: 50 foreign finished bookmarks shrank my unread count by 50). UNREAD now probes 'I have no finished bookmark on this comic' with a correlated NOT EXISTS scoped to the requesting user/session. It correlates on the queryset's comic pk, so on collection querysets it binds to the same joined comic row as the other filters in the combined .filter() call — group rows still require ONE comic that satisfies every condition (pinned by test). New comic-leading composite indexes (comic,user)/(comic,session) keep the probe off the user index: under stale sqlite_stat1 the planner flipped a user-scoped probe onto it — a measured 17s vs 27ms plan. EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user (comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged; single-user responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit cad52d79e8242e3b69e4cbd604757f2c7faab728 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 metadata: stop GET writing m2m through tables and corrupting tags _copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True) for every real m2m field on the Comic path — obj is a live saved instance (the 'values dicts' comment was stale), so a GET /metadata DELETE+INSERTed up to 12 through tables, and a multi-comic selection permanently rewrote the first comic's tags to the selection's intersection (reproduced: 339 characters silently became 43). Serve the intersection queryset through the instance's prefetch cache instead: RelatedManager.get_queryset consults it keyed by field name, so the serializer reads the already-optimized queryset query-free. Response verified byte-identical; the 38-query identifier N+1 (the post-write re-read discarded select_related) disappears with it. Co-Authored-By: Claude Fable 5 <[email protected]> commit 71b52749a3b24a8493c6803f0c3cda04466280d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 browse: bind favorite collection code as str so cachalot can cache Binding the Collection StrEnum member itself made cachalot's parameter-type check raise UncachableQuery for every query the favorite subquery landed in — silently uncaching all browse queries. Repeat table-mode browses re-paid ~10s of SQL every time; with .value they serve from cache (16-22ms, 2-3 queries). The bound SQL value is identical either way. Co-Authored-By: Claude Fable 5 <[email protected]> commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 metadata: pk subquery instead of inlined IN lists; count the filtered relation Two fixes to the intersection queries behind GET /metadata: 1. _get_comic_pks returned a materialized frozenset, inlining one bound SQL variable per comic per union arm. With 12 m2m arms, any collection over ~2,730 comics exceeded SQLite's 32,766-variable limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and 17.8k comics). Passing the distinct .values(pk) queryset keeps the statement constant-size (36 params); the same union measured 101ms at 8,576 comics. num_comics is computed once via COUNT and threaded to both consumers. 2. _get_fk_intersection_query counted the hardcoded 'comic' reverse m2m even when the filter traversed the main_*_in_comics FK back-relations — a semantically wrong HAVING over the unfiltered relation that also forced a separate unindexed join (main_team measured 2,069ms -> 12ms counting the filtered relation). Co-Authored-By: Claude Fable 5 <[email protected]> commit 5beb474835c7375f4b726eca63aa05f2ef8b9966 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 arc browse: demote story-arc chain joins to INNER for StoryArc queries With only codex_comic demoted, SQLite planned arc search browses from the arc side and built a cartesian product against the FTS match set: arcs root with a common search term measured 156s (main query 150.8s). Demoting codex_storyarcnumber and the M2M through table lets the plan start from the FTS/comic side (36ms). Scoped to StoryArc-model querysets: there the chain IS the collection relation, so NULL-extended rows are exactly the empty collections force_inner_joins exists to drop. On other models these tables appear as table-view annotation LEFT JOINs, where demotion would drop rows whose comics have no arcs. Co-Authored-By: Claude Fable 5 <[email protected]> commit 1434a5a771fc7445472cb52726f4e86c645df999 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 cover subqueries: join FTS rank only when ordering by search_score The direct fts_q join exists solely to populate codex_comicfts.rank for the cover subquery's search_score ORDER BY, but it was applied on every search regardless of order key — and the MATCH re-executed once per correlated cover evaluation. The pre-materialized fts_sq membership test already filters correctly. A name-sorted search browse measured 135s before, 6.5s after (byte-identical rows); rank-ordered searches keep the join and pick the same top-ranked cover as before. Co-Authored-By: Claude Fable 5 <[email protected]> commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:40 2026 -0700 table mode: keep intersection-sort subqueries out of GROUP BY RawSQL.get_group_by_cols() returns [self], so the correlated intersection-sort subqueries landed in the GROUP BY of aggregated collection queries. Folder is the one collection model without a forced-GROUP-BY entry, so SQLite evaluated the subquery once per pre-aggregation joined row — at folders root the ancestor M2M fans out to every descendant comic (366s request; each of the two queries measured ~256s alone, 85ms after). Same mechanism as the _CoverMtimeCoalesce fix, recurring through a different expression type. _IntersectionSortRawSQL overrides get_group_by_cols() to return []: the subquery's only outer reference is the collection pk, which is always grouped, so grouping is unchanged (verified row-identical). Co-Authored-By: Claude Fable 5 <[email protected]> commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:14:18 2026 -0700 fix librarian status feed order: cursor pagination discarded the order_by The sidebar progress feed showed import phases scrambled — read next to the search statii, query below create. The viewset ordered active rows by (preactive, active, pk), but DRF CursorPagination re-orders the queryset with its own ordering, so the v4 admin refactor's plain pk default (545fc2aa9) silently replaced the registration-stamp order with row-creation order. LibrarianStatusCursorPagination orders the active poll by (preactive, active, pk) — matching start_many's padded registration stamps, with NULL preactive sorting first for rows started without pre-registration (the cover thread case). The Jobs tab history view keeps the pk default. The active feed is ~20 rows against a 200-row page, so the cursor filter never engages the nullable fields. Regression test creates status rows with pk order deliberately opposite the registration stamps and asserts both routes. Co-Authored-By: Claude Fable 5 <[email protected]> commit 396312c6fe9964b36f895dd6a2c8bb84757191c3 Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:55:39 2026 -0700 import timing: don't double-count sub-steps in the phase share total _run_phases reuses the timed_step helper instead of an inline perf_counter copy, and _log_phase_times sums only top-level phases — dotted "phase.step" entries already accrue inside their parents, so including them inflated the total and shrank every phase's share. Co-Authored-By: Claude Fable 5 <[email protected]> commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:47:39 2026 -0700 metadata edit: full-size Protagonist label, live menu, stale-pick clear The Protagonist row inherited the old Main sub-row styling (smaller label, dimmed row) — it is a first-class row now, styled like its tag siblings. The choice menu already tracked the entered character/team names live (it is name-based, no pks involved); what was missing is that removing the picked name left the selection dangling — a watcher now clears it. A protagonist seeded from the DB but absent from the entered lists still survives load untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:28 2026 -0700 metadata: unify main_character/main_team as a single Protagonist field Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is replaced by one "Protagonist" row above the tag table showing the mainCharacter and/or mainTeam chips — both only in the should-never-happen case where both are set. Each chip carries its own browser filter key (characters vs teams) via a per-item filter override in MetadataTags. En route this fixes mapTag's sticky filter: the loop reassigned the filter param, so every row after the first reused the first row's filter key. Edit: the two Main Character / Main Team sub-selects collapse into one Protagonist select offering the entered characters and teams; the pick is sent as the comicbox "protagonist" patch field on write, and the importer resolves it back to main_character or main_team (nulling the other) on re-import. Co-Authored-By: Claude Fable 5 <[email protected]> commit 632a238590f188ee249c4a8164a6936bcdfc779c Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:16 2026 -0700 importer: stop clearing an unchanged protagonist on update imports The fk link phase set main_character/main_team to None whenever a comic's remaining LINK_FKS dict lacked a "protagonist" key. The query prune pops that key when the protagonist already matches the DB, so re-importing a comic with any *other* FK change silently NULLed its protagonist. Updates now only touch the protagonist fields when the metadata explicitly carried one (set-or-clear), matching the absent-means-untouched semantics of every other simple FK tag; creates keep both fields defaulted. Also: a name that is both a Character and a Team now links only main_character — both fields filled is invalid. Regression test drives query → prune → link with only scan_info changed and asserts the protagonist survives (it NULLed before). Co-Authored-By: Claude Fable 5 <[email protected]> commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead Author: AJ Slater <[email protected]> Date: Wed Jun 10 16:16:30 2026 -0700 import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s) TimestampUpdater filtered every collection model through a Count(DISTINCT comic) alias to exclude empty collections — a per-row GROUP BY join aggregate costing ~1.5s per import even with zero deletions, in every import scenario. The count was redundant for two of the three OR branches: a comic__u…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.