v2.0.0 alpha#779
Closed
ajslater wants to merge 347 commits into
Closed
Conversation
Follow-ups to the admin design-language refactor (e836e93): - Group-tab help: use margin-block so the `margin: 2em 0` shorthand no longer zeroes the inline margins and cancel .adminReadingColumn's margin-inline:auto centering on the same element. - Key/value tables: add an inter-column gap so a long label (e.g. "Authorization Groups") no longer butts into the right-aligned value, and stripe alternating rows for readability, replacing the per-row border. - Drop the stale stats-table.vue reference from status-helpers. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Pure behavior-preserving extractions so each flagged function/class lands under the gate thresholds. - vacuum.py: split backup_db into _resolve_backup_path / _write_backup / _prune_dated_backups (complexipy 16 -> <15) - restore.py: extract _build_user_defaults and _build_tagging_defaults; the tagging builder keeps the null-coalescing ors in one comprehension (radon cc C13/C11 -> A) - admin/user.py: split AdminUserBulkView.post into _parse_delete_ids and _delete_users; radon scores a class by its method-CC average, so this drops the class C12 -> A5 - browser/group_mtime.py: extract page-mtime cache get/set and the aggregate query out of get_group_mtime (radon cc C12 -> A5) - test_admin_custom_cover.py: extract _assert_covers_enqueued (C11 -> B9) - move the two self-contained unit classes out of test_browser_table_response.py into test_browser_table_columns.py (radon mi B 18.30 -> A 23.21) Verified: complexipy clean (pathless), radon cc --min C and mi --min B both empty, ruff/basedpyright/ty clean, 127 affected tests pass. Co-Authored-By: Claude Opus 4.8 <[email protected]>
The email tab's Test Send recipient field lived in the settings <v-form>, so pressing Save Settings ran form.validate() over it and warned "Recipient is required" on the usually-empty field. Move Test Send into its own form and validate the recipient in runTest() instead. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Save could stay disabled for a valid change because it was gated on the auth-form-mixin's async submitButtonEnabled flag, and the mixin also re-validated the whole form on every keystroke (flashing "required" on untouched fields). Opt out of the mixin's credentials watcher and gate Save on a synchronous isValid computed; fields keep their inline validate-on=input rules. Adds a focused unit test. Co-Authored-By: Claude Opus 4.8 <[email protected]>
- Serve the SPA for /auth/reset-password/ so Vue Router renders the reset screen; the catch-all was redirecting the emailed link to the home page. - Stop Django autoescaping the URL in the plain-text email (it turned the query separators into &, corrupting user_id/timestamp/signature). - Open the login dialog after a successful reset. - Show the username on the reset screen, carried by the link (display-only; the reset stays gated by the signed user_id/timestamp/signature). Adds backend routing + email tests and a frontend component test. Co-Authored-By: Claude Opus 4.8 <[email protected]>
- validate-on=submit on the login form so diverting to "Forgot password?" no longer trips the empty-username required rule on blur. - Close the reset-request dialog on send; the green "Reset link sent" banner already shown on the login screen is the only confirmation needed. Drops the redundant in-dialog message and the now-dead resetPasswordRequestSent store flag. Adds/updates unit tests. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Point RESET_PASSWORD_VERIFICATION_EMAIL_TEMPLATES at both a text_body and a new html_body so rest-registration sends a multipart message: the text template is the plain-text part (text-only readers fall back to it) and the HTML is attached as an alternative. The HTML renders the reset link as a styled codex-orange "Reset Password" button with a copy-paste fallback link, and carries the same &username= param so the reset screen still shows it. djlint H021 (no inline styles) is disabled for the template since email clients strip <style>/external CSS. Adds a multipart-assertion test. Co-Authored-By: Claude Opus 4.8 <[email protected]>
The edit tags panel could set Issue Count and Volume Count but had no
field for the comic's own issue number or suffix. Add Issue Number and
Issue Suffix inputs, regrouping the Publishing section to mirror the
read-only header (Volume + Volume Count, Issue + Issue Count).
buildPatch emits the comicbox `issue: {number, suffix}` object; comicbox
computes the combined `issue.name` written to <Number> for both
ComicInfo and MetronInfo. Enable both fields for the two formats in
format-field-support.json.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The SPA boots every visitor through GET /api/v4/session, but the view was gated behind IsAuthenticatedOrEnabledNonUsers. When a visitor was logged out AND the non_users flag was off, the request 401'd, so adminFlags never loaded, isAuthChecked stayed false, and the browser spun on the placeholder forever. Make SessionView AllowAny -- the payload is anonymous-safe by construction (user is null, permissions all-false). Split the admin flags into a public subset (registration, non_users, banner_text, register_verification, email_enabled) that the logged-out shell needs and a private subset (lazy_import_metadata, remote_user_enabled) that only the authenticated UI reads, so an unauthenticated boot discloses nothing beyond the logged-out screen. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Tag writes resolved a selected group (e.g. a publisher) to *every* comic under it via a bare relation filter, ignoring the active browser filters. Writing the "Image" publisher tag to a CBR+unread-filtered publisher leaked onto an unread CBZ (Monstress #62) the filters should have excluded. The shared `_resolve_comic_pks` helper backed all three archive-write entry points (bulk tag-write, its preflight, and online tag start), so every one of them ignored file_type, read/unread, ACL, favorite, and search filters alike. Replace the helper with a `FilteredComicPksView` mixin that resolves group+pks through `get_filtered_queryset(Comic, group, pks)` — the same browser filter pipeline `ForceUpdateView` and `BookmarkView` already use, reading the user's persisted filters via a load-only `params` override. The admin envelope renderer and browser base share `EnvelopeJSONRenderer`, so the wire format is unchanged and the Vue callers need no edits. Add a regression test covering the reported file_type case plus read/unread, with a no-filter control. Co-Authored-By: Claude Opus 4.8 <[email protected]>
The browser empty-state mapped adminFlags.registration into a computed but never referenced it in the template or any other computed. Dead mapping, removed. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two bugs left the hover-to-lazy-import feature dead code: - lazyImportEnabled read this.stateLazyImportMetadata, but the mapped auth-store state is named stateLazyImportEnabled, so the guard was always undefined (falsy). - the @mouseenter binding was a ternary (lazyImportEnabled ? onMouseEnter : null) which evaluates to a function reference but never invokes it. onMouseEnter already guards on lazyImportEnabled internally, so bind it directly. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Lock in the behavior that cd283fd restored. Mounts the real Vuetify button and dispatches a DOM mouseenter so the @mouseenter binding and the lazyImportEnabled guard are exercised end-to-end: - fires lazyImport({ group, ids }) only when the lazyImportMetadata admin flag is on and the book is an un-imported comic - stays inert when the flag is off, the comic already has metadata, or the group is not "c" - prefers book.ids over book.pk and imports at most once Reintroducing the stateLazyImportMetadata/stateLazyImportEnabled typo fails the two firing assertions, so this guards the exact regression. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 0 of unifying the group/collection vocabulary. Introduces codex/group.py: a Group StrEnum that is the single source of truth for the browse groups, with explicit char, collection, singular-cover-name and display-label maps plus from_char/from_collection helpers. The member value is the legacy single-char code for now. Because a StrEnum member compares and hashes equal to its value, this makes the enum a drop-in for the char strings still used by the engine, DB and URLConf: a dict re-keyed to Group members is still found by the old char keys, so each layer can adopt the enum incrementally with zero behavior change. A later phase flips the values to the collection names alongside the DB/URL/frontend cut; .char and .collection survive that flip because they read the explicit maps, not the member value. Purely additive — nothing imports it yet, so no behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 1 of the group/collection unification. const.py now sources its group constants (ROOT_GROUP, COMIC_GROUP, …) and maps (GROUP_MODEL_MAP, GROUP_RELATION, GROUP_ORDER, …) from the Group enum, re-keyed to Group members. Because a StrEnum member hashes and compares equal to its char value, every existing char-keyed lookup and `group == ROOT_GROUP` check keeps working unchanged. The remaining raw single-char group literals in the browse/reader engine (bookmark, browser, breadcrumbs, books, settings) are replaced with Group members, including _SHOW_KEYS so the show grid is enum-keyed ahead of the Phase 2 column rename. Behavior-preserving: full suite green (381 passed). The Route dataclass internals, _GROUP_PARENT_CHAIN, and the dummy-0-as-root sentinel are intentionally deferred to Phases 2-3, where the DB and URL representations actually change. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 3 (additive transition step). Adds the v4 collection SPA patterns to codex/urls/app.py — /<collection>, /<collection>/<parent_ids>, /read/<pk>, /read/<pk>/book.pdf — next to the legacy /<group>/<pks>/<page> and /c/<pk>/book.pdf routes. IndexView ignores the URL kwargs (it only serves the SPA shell), so deep links / refreshes on the new URLs now serve the app instead of hitting the catch-all redirect. The CollectionConverter regex keeps "read"/"admin"/"error" from matching the collection pattern, so the routes coexist with no ordering hazard. Both dialects are served until the frontend flips to collection URLs; the legacy routes are removed in the final cleanup. Page is a ?page= query param on the new routes. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 3 (additive transition step). RouteSerializer / SimpleRouteSerializer now include the v4 collection name and parent-id list in their output in addition to the legacy group/pks, so last_route, breadcrumbs, close_route, and redirect routes carry both dialects at once. The current frontend ignores the new keys; the collection frontend will consume them. Root (group "r") maps to the publishers collection with empty parentIds, matching the v4 API. Input is unchanged (still group/pks). Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 4 groundwork. Adds routeForGroup / groupForRoute / normalizeParentIds
to frontend/src/route.js, the single place that converts between the legacy
{group, pks} and the v4 {collection, parentIds} route shapes. Mirrors the
backend root rule (r <-> publishers with no parentIds) so a route round-trips
through either dialect. Inert until the router and route builders flip to
collection URLs; unit-tested in isolation (5 vitest cases).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 4 (reader half). The reader SPA route flips from /c/:pk/:page to
/read/:pk with the page as a ?page= query param, matching the v4 scheme
and the app.py route added earlier. The reader's internal {pk, page}
params model is unchanged; conversion happens only at the route push /
resolve (reader store) and the route page-reads (pager, pager-horizontal,
nav button), which now read ?page= with a 0 default. The browser route is
untouched here and flips in the next commit.
Verified: vitest 99 passed (nav-button href snapshot updated /c -> /read),
eslint clean, production build green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
…page=
Phase 4 (browser half). The browser SPA route becomes
/:collection(publishers|imprints|...)/:parentIds? with the page as a
?page= query param, matching the v4 scheme and the app.py routes.
The browser store keeps its intricate redirect/validate/dequal logic
untouched by working internally in the legacy {group, pks, page} string
shape: liveBrowseParams() reads the live collection route back into that
shape (parentIds -> pks, ?page= -> page string, default "1"), and
toBrowseRoute() converts an internal route to a pushable collection route
at the edges (idempotent; preserves the numeric-page force-redirect
trick). Route :to builders (card, browser-table, breadcrumbs, pagination
nav) convert via routeForGroup; metadata links flow through
routeWithSettings -> toBrowseRoute unchanged. The home redirect derives
the collection from last_route's group/pks.
Verified: vitest 99 passed (router regex uses [\d,]+ since path-to-regexp
rejects nested groups), eslint clean, production build green. The live
navigation flow (cards, breadcrumbs, pagination, top-group redirects)
still warrants a manual QA pass in a running app.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 4 follow-up. Five components still read the old path params directly: the search-limit page (main.vue), the pagination page (browser-toolbar-nav), the title's root check + nav group (browser-toolbar-title), the card subtitle's folder-view check, and the metadata link's already-here check (metadata-text). Page reads now use ?page= (default 1); root is "no parentIds"; the nav group is derived via groupForRoute (or a direct collection === "folders" check). Verified: vitest 99 passed, eslint clean, build green. Co-Authored-By: Claude Opus 4.8 <[email protected]>
There is no other functional OPDS test. This walks the catalog by following whatever nav / subsection / next / up links the server emits — asserting every one resolves (2xx, or a 3xx self-correcting redirect to a 2xx) for both the v1.2 Atom and v2.0 JSON feeds — plus pagination emits a followable `next`. Because it follows emitted links rather than hardcoding paths, it pins feed resolution invariantly across the upcoming group→collection URL flip (Phase 5). Co-Authored-By: Claude Opus 4.8 <[email protected]>
OPDS was the last URL surface still minting the legacy single-char
scheme (`<group>/<pks>/<page>` + the dummy `0`-as-root). Flip it to the
unified `/<collection>[/<parent_ids>][?page=]` scheme so the GroupConverter
and the `0` sentinel can be retired wholesale in the final value-flip phase.
Inbound is already handled: OPDS inherits `AuthMixin`, whose `initial()`
translates `{collection, parent_ids}` → engine `{group, pks}` (and reads
`?page=` once the feed views set `requires_page`). OPDS emits no bare
`publishers` listing — its `?topGroup=` facets resolve ROOT the same way —
so AuthMixin's publishers→ROOT collapse is exactly right here.
Outbound is the new `codex/views/opds/route.py:opds_feed_reverse`, the
single inverse edge: it maps engine `{group, pks, page}` back to the
collection URL (ROOT→publishers, pks→parent_ids dropping the dummy `0`,
page→`?page=`). Every feed-link reverse routes through it — v1 links /
facets / entry / metadata, the central v2 `href()`, and the OPDS
self-correcting redirect builder (`SeeOtherRedirectError.get_response`).
`OPDSURLsView` reverses `start` plainly. The v2 manifest keeps its literal
`c/` route (it never used the group converter); `position` moves to a
literal `comics/` prefix.
View bodies still speak `{group, pks, page}` internally — only the two
edges move — so feed logic is untouched. The functional suite (which
follows server-emitted links) stays green and now also asserts the
collection scheme: collection segments present, no `/0/`, no char groups.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 2 of the group→collection unification, staged: the engine, DB, and
URLConf now speak collection names end to end, while the still-char frontend
stays green via a translation edge at the serializers. The ~184-touchpoint
frontend flip + retiring the api/v4/browser.js shim + removing the remaining
scaffolding (GroupConverter, COLLECTION_TO_GROUP, Favorite migration, the
dual-dialect serializer, .char) is the deferred follow-up.
Core:
- Group enum member values char→collection ("publishers", … ROOT="root").
GROUP_CHARS + .char kept; new idempotent group_char()/group_value() helpers
bridge the two dialects.
- DB migration 0044: rename SettingsBrowserShow p/i/s/v → publishers/imprints/
series/volumes (+ constraint); widen + data-migrate SettingsBrowser.top_group
and SettingsBrowserLastRoute.group char→collection; strip the dummy 0 from
last_route pks. Reversible.
- AuthMixin drops the char translation — the collection segment IS the engine
group value now (publishers+no-ids → root). Aliased the import to BrowseGroup
so it no longer shadows django.contrib.auth Group.
Char compat (so the frontend wire is unchanged):
- The BrowseGroupField / BrowserRouteGroupField + the show serializer translate
char↔collection at the wire edge; everything internal (params, validation,
GROUP_ORDER, defaults) is collection-consistent.
- Per-row group annotation, table-column defaults, metadata parent groups, the
group filter relation, and the OPDS v2 preview were updated to the collection
vocabulary (or made dialect-tolerant). OPDS URL defaults r→root, c→comics.
Engine const maps GROUP_RELATION / FILTER_ONLY_GROUP_RELATION annotated
[str, str] (collection-value lookups). Verified: pytest 397, vitest 99,
ruff/ty/radon/complexipy clean. A char PATCH→GET round-trips while the rows
store the collection value (tests/test_settings_collection_flip.py).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
…r literals Remove the char compat layer added in the staged Phase 2a: the browser serializers now accept/emit the collection vocabulary directly (group fields, the show p/i/s/v → publishers/imprints/series/volumes flags, per-row group annotation, table_columns keys), and the OPDS feed-link builders + entry comparisons speak collection (topGroup/group literals, facet values, the special-group set, the position URL default). Backend tests move to the collection wire. The frontend flip + scaffolding removal follow. pytest 397 green. Co-Authored-By: Claude Opus 4.8 <[email protected]>
_handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]>
Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]>
Merge implicit f-string concatenations, add missing @OverRide on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]>
commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:29 2026 -0700 refactor(complexity): clear radon cc rank-C findings bin/lint-complexity.sh reported three rank-C functions; extract helpers to drop each below the gate (radon cc --min C now empty). - AdminTagByIdView.post: pull primary-identifier parsing and merge extra-id resolution into _resolve_primary / _resolve_extra_ids. - OnlineTagSessionManager._apply_resolution: extract _handle_unresolved and _enqueue_resolved_write. - test_update_protagonist: extract repeated protagonist assertions into a helper (radon counts each assert as a branch). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 55bb659fcc8c02e73c20a34823652a4f81be1d56 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:46:04 2026 -0700 refactor(migrations): squash 0044 into 0043 Fold the merge_all_sources field into the ComicboxTaggingDefaults CreateModel in 0043 and drop the standalone 0044. makemigrations --check reports no drift. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 7a988737a63286cc85396346579e124e5fff8e5b Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:44:59 2026 -0700 feat(onlinetag): merge metadata from all sources Add an opt-in "merge all sources" mode that queries every enabled online source per comic and merges the results (comicbox first_wins=False) for maximum tag completeness, instead of stopping at the first match. - Admin tagging defaults: merge_all_sources boolean (default off), settable as the default and overridable per scan. - Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate + rate-limit warning multiply calls by source count (kept in sync with estimate.py) so the doubled API cost is visible. - By-ID: chips input accepting multiple URLs/ids; merging fetches two explicit ids (one per source) and merges them. Source select shows only when a token can't be auto-identified (bare number). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:34:56 2026 -0700 update deps commit 473221d7a09403b1848f202d55902b71e31319c2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:22:58 2026 -0700 fix(lint): clear ruff/pyright findings; fix benchmark import path - tests: add docstrings, name magic estimate values, use pytest.raises and out-of-class exception message - onlinetag: collapse implicit f-string concatenation in log call - benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*) which was a real runtime ImportError in library generation Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 45a9fde6076a5b0b15be889769e9a55af4b0678a Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:13:49 2026 -0700 refactor(migrations): squash 0044 + 0045 into 0043 Fold the comic-leading composite Bookmark indexes (former 0044) and the LibrarianStatus eta/retry_at fields + JFR status-type choice (former 0045) into the consolidated 0043 migration; delete both files. makemigrations --check reports no changes against the model state. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 22da6b9cd85615f3d7b8e7043818babac313d24e Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:35 2026 -0700 update deps commit 6f27454623e43cccd95253d5dfe66471820c9349 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:15 2026 -0700 fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns Two online-tagging bugs plus a status-UX overhaul. Prompt answers given while a scan was running were lost on refresh: the response task sat in the queue behind the busy OnlineTagThread, so the cache kept the prompt. Now the drain loop removes answered prompts from the cache inline (race-free, single-threaded) and marks the fingerprint so the running scan stops re-persisting it; the network re-fetch + write is deferred until the scan releases the thread. A rate-limited lookup looked hung: collect_results could raise mid-pass (budget exhausted, network error) without ever calling finish(), leaving the status row frozen forever. finish() now runs in a finally. The rate-limit status was a static "rate limited ~57s" — a second subtitle writer clobbered the parseable one, and nothing refreshed it during comicbox's blocking sleep. Replaced with two absolute-timestamp countdowns the admin UI ticks down to live: - retry_at: time until the next online request, re-anchored per attempt. - eta: total time remaining, carrying forward the launcher dialog's estimate (ported verbatim to estimate.py), re-estimated on each comic completion and pushed out by rate-limit waits. Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in the pending status_type choices drift), plumbed through Status and StatusController. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit eccf41bc603b118bdbd63f4e4918a30da87140ae Author: AJ Slater <[email protected]> Date: Fri Jun 12 22:39:15 2026 -0700 feat(tagging): admin-orderable online source priority defaultSources order is now run priority end to end. comicbox runs sources in the given order under first-wins, so the admin can choose which source is primary and which is fallback: - Tagging tab renders the source rows in defaultSources order with per-row up/down arrows; enabling a source appends it at lowest priority. A hint explains that the first matching source wins. - The launcher dialog normalizes its checkbox selection to the admin's priority order before starting a session (checkbox v-model records click order, not priority). - Serializers validate defaultSources / sources: known names only, deduped, order preserved (unknown source → 400). - explicit_id / session settings pass ordered tuples to comicbox's retyped OnlineLookupSettings.sources. Co-Authored-By: Claude Fable 5 <[email protected]> commit 02869ddf4d027c325637dd41081199c59f1141ad Author: AJ Slater <[email protected]> Date: Fri Jun 12 17:06:25 2026 -0700 add METRON_INFO to default formats to write commit a214c9c93cf679194beeb5c42eb66deea933a3da Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:36 2026 -0700 update deps commit 65a294362171bded71547430dd51b322ea83dcef Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:08 2026 -0700 don't make abort button red commit de49b5dd9529446ab34a7468bda694fae39c7a00 Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:52:14 2026 -0700 fix(frontend): remove duplicated Tag Write Errors sidebar item admin-menu.vue contained the same tag-write-errors CodexListItem twice (re-added in a later squash while the original stayed), so any error rendered two identical sidebar entries. One item per error kind; the counts live in the admin tables. Co-Authored-By: Claude Fable 5 <[email protected]> commit e0bd853d9ed1991859849e11d85d487409dc1194 Author: AJ Slater <[email protected]> Date: Fri Jun 12 13:43:19 2026 -0700 fix(onlinetag): stop spurious rewrites and prompt loss in online tagging Three interacting bugs broke a single-comic tagging run: - Pass-1 and prompt-resolution writes now require result.matched (new comicbox flag): unmatched results carry the comic's merged existing metadata, so gating on truthy tags re-wrote files with no new information — which also triggered a re-import mid-session. - Pending prompts and tag-write errors move to a dedicated 'tagging' cache (codex.cache.tagging_cache). The session_cache docstring claimed key namespacing protected them from cache.clear(), but Django's file-based clear deletes everything: every import that changed anything (importer finish), Library/Group CRUD, and startup wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'. - Prompt resolution builds its replay session with defer_prompts=True so the preloaded resolution is actually consulted (comicbox only installs its cache-reading selector with defer or a handler; without it an ambiguous re-search fell through to comicbox's interactive CLI prompt inside the daemon). A fingerprint miss — the re-search returned different candidates — now re-queues the fresh deferred prompt instead of dying silently. Co-Authored-By: Claude Fable 5 <[email protected]> commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:45:22 2026 -0700 feat(admin): auto-enable a tagging source when its credentials are first saved Saving credentials for a source that previously had none now adds it to default_sources via the auto-saving draft, so a freshly configured source works without a second checkbox click. Re-saving credentials for an already-configured source leaves the enable checkbox untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:38:11 2026 -0700 Squashed commit of the following: commit 9543008f24441b6b2992f0c818d019f42135f7df Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:21:24 2026 -0700 dockerignore benchmark dir commit 00134a40fe1c37d96e64066735cb42cc03f463c8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:20:24 2026 -0700 mv fixutres to benchmark dir commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:54 2026 -0700 change import path for mock comics commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:05 2026 -0700 ignore fixtures dir commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:11:25 2026 -0700 common subdir for mock & benchmark commit cd768965a544069fb623f9798a2ef7f472016586 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:01:01 2026 -0700 move benchmark import out of ci commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:52:32 2026 -0700 v2.0.0a10 commit 43b6042bca9c546d49268a52002efa4db69651c4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:51:47 2026 -0700 update deps commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:39:57 2026 -0700 fix typecheck and ty warnings Merge implicit f-string concatenations, add missing @override on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:28:58 2026 -0700 remove bad news line commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:18:54 2026 -0700 adapt to comicbox 4.0.0a3 API Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]> commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251 Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:55:25 2026 -0700 update deps commit 919fad63cf5790b82275e5f20207e38f65934a6c Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:48:25 2026 -0700 update deps commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e Author: AJ Slater <[email protected]> Date: Thu Jun 11 14:00:48 2026 -0700 poller: apply discarded .only() projection in manual poll path _handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]> commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 07:52:01 2026 -0700 admin: fix stale Last Poll column after websocket table refetch DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <[email protected]> commit 188dd22775d6beb284b6ddafb3629dcb0affbccb Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:18:16 2026 -0700 edit news for brevity commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:17:05 2026 -0700 NEWS: browser performance entry for 2.0.0 Co-Authored-By: Claude Fable 5 <[email protected]> commit 994d6a586c249376827c1d72494071b4281528fa Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:15:27 2026 -0700 news: v2.0.0 performance section for the import speedups Co-Authored-By: Claude Fable 5 <[email protected]> commit cbb07c980bd5bb317e7a506bec81ba8f66415163 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 cards: Max bookmark timestamp aggregate instead of JSON array + parse loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <[email protected]> commit 01a9394d8ecaaa22e72885b4257545a82220b3d4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 breadcrumbs: batch the folder ancestor chain in one query The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <[email protected]> commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <[email protected]> commit 578807ab0294391095e2f6b6fe3d1b1540f643a4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:55:07 2026 -0700 search: materialize the FTS match set once per browse request The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <[email protected]> commit e56b7eb2dd54cab402041de418832f865269540c Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 lint: format follow-ups for the order/cover perf commits Co-Authored-By: Claude Fable 5 <[email protected]> commit ca67645e4bf757568021e19781d9179ae599bda2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 browser: move the last-route write off the read path; settings row fetched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <[email protected]> commit 94ef67e41554869cfe4c19494cea7e422ff5d30b Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:10:00 2026 -0700 covers: batch cover mtimes post-pagination instead of a second subquery cover_mtime was a .values('updated_at') Subquery over the identical correlated cover queryset that already computes cover_pk — two full executions of the most expensive expression per card (measured 40-49% of the card query across publisher/folder/search pages). Drop the annotation; resolve the representative comics' updated_at after pagination with one indexed pk__in batch over the page's cover_pks and attach it to the cached instances (the serializer iterates the same objects). Custom-cover mtimes keep their cheap rowid subquery and still win the coalesce. _CoverMtimeCoalesce is gone with the annotation that needed it. Responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 browser: zero-pad from page rows; skip dead card serialization in table mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <[email protected]> commit d1789eda2ac5a4351c3039ee42d4677db1c669c6 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 table mode: evaluate the collection page query once, not twice compute_collection_intersections read the page pks via values_list('pk'), a clone that re-executed the entire annotated/ grouped/sorted collection query; the serializer then ran the original queryset again. Iterating the queryset itself primes its result cache, which the serializer reuses — the heaviest table-mode query now runs once per request. Co-Authored-By: Claude Fable 5 <[email protected]> commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 comic listings: stop aggregating order aliases; GROUP BY pk only Two GROUP BY-width fixes for Comic-model querysets: - _alias_filename aggregated Min/Max over the comic's own single path, which turned the alias into an aggregate and dragged every selected column into GROUP BY inside the listing and both cover subqueries. A comic is its own file; use the bare expression (as annotate_comic_extra_specials already did). A filename-ordered folder browse measured 135s before, 0.6s after, identical rows. - The ids JsonGroupArray made Django compute GROUP BY over all ~50 selected Comic columns including TEXT blobs. Force GROUP BY id — every other column is functionally dependent on the pk, so groups are identical and SQLite sorts a one-column key (25-34% off the books query). Not applied to the cover path, where the forced literal column doesn't survive nested-subquery aliasing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 browser: UNREAD filter must not hide comics other users finished The UNREAD arm was Q(bookmark=None) | (mine & unfinished): bookmark=None matches only comics with no bookmark from ANY user or session, so a comic finished by someone else (or a stale anonymous session) matched neither arm and silently vanished from my unread listing (reproduced: 50 foreign finished bookmarks shrank my unread count by 50). UNREAD now probes 'I have no finished bookmark on this comic' with a correlated NOT EXISTS scoped to the requesting user/session. It correlates on the queryset's comic pk, so on collection querysets it binds to the same joined comic row as the other filters in the combined .filter() call — group rows still require ONE comic that satisfies every condition (pinned by test). New comic-leading composite indexes (comic,user)/(comic,session) keep the probe off the user index: under stale sqlite_stat1 the planner flipped a user-scoped probe onto it — a measured 17s vs 27ms plan. EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user (comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged; single-user responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit cad52d79e8242e3b69e4cbd604757f2c7faab728 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 metadata: stop GET writing m2m through tables and corrupting tags _copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True) for every real m2m field on the Comic path — obj is a live saved instance (the 'values dicts' comment was stale), so a GET /metadata DELETE+INSERTed up to 12 through tables, and a multi-comic selection permanently rewrote the first comic's tags to the selection's intersection (reproduced: 339 characters silently became 43). Serve the intersection queryset through the instance's prefetch cache instead: RelatedManager.get_queryset consults it keyed by field name, so the serializer reads the already-optimized queryset query-free. Response verified byte-identical; the 38-query identifier N+1 (the post-write re-read discarded select_related) disappears with it. Co-Authored-By: Claude Fable 5 <[email protected]> commit 71b52749a3b24a8493c6803f0c3cda04466280d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 browse: bind favorite collection code as str so cachalot can cache Binding the Collection StrEnum member itself made cachalot's parameter-type check raise UncachableQuery for every query the favorite subquery landed in — silently uncaching all browse queries. Repeat table-mode browses re-paid ~10s of SQL every time; with .value they serve from cache (16-22ms, 2-3 queries). The bound SQL value is identical either way. Co-Authored-By: Claude Fable 5 <[email protected]> commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 metadata: pk subquery instead of inlined IN lists; count the filtered relation Two fixes to the intersection queries behind GET /metadata: 1. _get_comic_pks returned a materialized frozenset, inlining one bound SQL variable per comic per union arm. With 12 m2m arms, any collection over ~2,730 comics exceeded SQLite's 32,766-variable limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and 17.8k comics). Passing the distinct .values(pk) queryset keeps the statement constant-size (36 params); the same union measured 101ms at 8,576 comics. num_comics is computed once via COUNT and threaded to both consumers. 2. _get_fk_intersection_query counted the hardcoded 'comic' reverse m2m even when the filter traversed the main_*_in_comics FK back-relations — a semantically wrong HAVING over the unfiltered relation that also forced a separate unindexed join (main_team measured 2,069ms -> 12ms counting the filtered relation). Co-Authored-By: Claude Fable 5 <[email protected]> commit 5beb474835c7375f4b726eca63aa05f2ef8b9966 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 arc browse: demote story-arc chain joins to INNER for StoryArc queries With only codex_comic demoted, SQLite planned arc search browses from the arc side and built a cartesian product against the FTS match set: arcs root with a common search term measured 156s (main query 150.8s). Demoting codex_storyarcnumber and the M2M through table lets the plan start from the FTS/comic side (36ms). Scoped to StoryArc-model querysets: there the chain IS the collection relation, so NULL-extended rows are exactly the empty collections force_inner_joins exists to drop. On other models these tables appear as table-view annotation LEFT JOINs, where demotion would drop rows whose comics have no arcs. Co-Authored-By: Claude Fable 5 <[email protected]> commit 1434a5a771fc7445472cb52726f4e86c645df999 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 cover subqueries: join FTS rank only when ordering by search_score The direct fts_q join exists solely to populate codex_comicfts.rank for the cover subquery's search_score ORDER BY, but it was applied on every search regardless of order key — and the MATCH re-executed once per correlated cover evaluation. The pre-materialized fts_sq membership test already filters correctly. A name-sorted search browse measured 135s before, 6.5s after (byte-identical rows); rank-ordered searches keep the join and pick the same top-ranked cover as before. Co-Authored-By: Claude Fable 5 <[email protected]> commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:40 2026 -0700 table mode: keep intersection-sort subqueries out of GROUP BY RawSQL.get_group_by_cols() returns [self], so the correlated intersection-sort subqueries landed in the GROUP BY of aggregated collection queries. Folder is the one collection model without a forced-GROUP-BY entry, so SQLite evaluated the subquery once per pre-aggregation joined row — at folders root the ancestor M2M fans out to every descendant comic (366s request; each of the two queries measured ~256s alone, 85ms after). Same mechanism as the _CoverMtimeCoalesce fix, recurring through a different expression type. _IntersectionSortRawSQL overrides get_group_by_cols() to return []: the subquery's only outer reference is the collection pk, which is always grouped, so grouping is unchanged (verified row-identical). Co-Authored-By: Claude Fable 5 <[email protected]> commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:14:18 2026 -0700 fix librarian status feed order: cursor pagination discarded the order_by The sidebar progress feed showed import phases scrambled — read next to the search statii, query below create. The viewset ordered active rows by (preactive, active, pk), but DRF CursorPagination re-orders the queryset with its own ordering, so the v4 admin refactor's plain pk default (545fc2aa9) silently replaced the registration-stamp order with row-creation order. LibrarianStatusCursorPagination orders the active poll by (preactive, active, pk) — matching start_many's padded registration stamps, with NULL preactive sorting first for rows started without pre-registration (the cover thread case). The Jobs tab history view keeps the pk default. The active feed is ~20 rows against a 200-row page, so the cursor filter never engages the nullable fields. Regression test creates status rows with pk order deliberately opposite the registration stamps and asserts both routes. Co-Authored-By: Claude Fable 5 <[email protected]> commit 396312c6fe9964b36f895dd6a2c8bb84757191c3 Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:55:39 2026 -0700 import timing: don't double-count sub-steps in the phase share total _run_phases reuses the timed_step helper instead of an inline perf_counter copy, and _log_phase_times sums only top-level phases — dotted "phase.step" entries already accrue inside their parents, so including them inflated the total and shrank every phase's share. Co-Authored-By: Claude Fable 5 <[email protected]> commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:47:39 2026 -0700 metadata edit: full-size Protagonist label, live menu, stale-pick clear The Protagonist row inherited the old Main sub-row styling (smaller label, dimmed row) — it is a first-class row now, styled like its tag siblings. The choice menu already tracked the entered character/team names live (it is name-based, no pks involved); what was missing is that removing the picked name left the selection dangling — a watcher now clears it. A protagonist seeded from the DB but absent from the entered lists still survives load untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:28 2026 -0700 metadata: unify main_character/main_team as a single Protagonist field Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is replaced by one "Protagonist" row above the tag table showing the mainCharacter and/or mainTeam chips — both only in the should-never-happen case where both are set. Each chip carries its own browser filter key (characters vs teams) via a per-item filter override in MetadataTags. En route this fixes mapTag's sticky filter: the loop reassigned the filter param, so every row after the first reused the first row's filter key. Edit: the two Main Character / Main Team sub-selects collapse into one Protagonist select offering the entered characters and teams; the pick is sent as the comicbox "protagonist" patch field on write, and the importer resolves it back to main_character or main_team (nulling the other) on re-import. Co-Authored-By: Claude Fable 5 <[email protected]> commit 632a238590f188ee249c4a8164a6936bcdfc779c Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:16 2026 -0700 importer: stop clearing an unchanged protagonist on update imports The fk link phase set main_character/main_team to None whenever a comic's remaining LINK_FKS dict lacked a "protagonist" key. The query prune pops that key when the protagonist already matches the DB, so re-importing a comic with any *other* FK change silently NULLed its protagonist. Updates now only touch the protagonist fields when the metadata explicitly carried one (set-or-clear), matching the absent-means-untouched semantics of every other simple FK tag; creates keep both fields defaulted. Also: a name that is both a Character and a Team now links only main_character — both fields filled is invalid. Regression test drives query → prune → link with only scan_info changed and asserts the protagonist survives (it NULLed before). Co-Authored-By: Claude Fable 5 <[email protected]> commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead Author: AJ Slater <[email protected]> Date: Wed Jun 10 16:16:30 2026 -0700 import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s) TimestampUpdater filtered every collection model through a Count(DISTINCT comic) alias to exclude empty collections — a per-row GROUP BY join aggregate costing ~1.5s per import even with zero deletions, in every import scenario. The count was redundant for two of the three OR branches: a comic__updated_at join match implies the collection has children, and force-updated pks were exempted from the child filter anyway. Only the custom-cover branch can match an empty collection, so it now carries its own join-presence test (comic__isnull=False) instead. Also replaces the fetch-instances + CASE-WHEN bulk_update round trip with a batched .update(updated_at=Now()) over the distinct pks. Also corrects the stale IMPORTER_FILTER_BATCH_SIZE comment: the binding constraint on OR chains is SQLITE_LIMIT_EXPR_DEPTH (1000), not the old 999-variable cap (modern SQLite allows 32766), so 900 remains correct for the remaining chain users. Benchmark, 1000 mock comics: delete phase 1.46s -> 0.10s on fresh and below 0.1s on reimport (5.9s total). Re-stamp semantics covered by tests/importer/test_move_timestamps.py. Co-Authored-By: Claude Fable 5 <[email protected]> commit bac8ee8af974bec78781f5c764e543f8429540a1 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:51:29 2026 -0700 import perf: bulk_update only changed comic fields (update step 8x) update_comics passed all ~40 BULK_UPDATE_COMIC_FIELDS to bulk_update regardless of what changed — CASE-WHEN compilation and parse scale with rows x fields, so a force-reimport whose comics needed nothing but a stat refresh paid the full-width rewrite: 4.4s of the 7s remaining reimport time, almost all inside the two bulk_update manager calls. Collect the union of fields actually applied per batch (proposed md diffs + FK link fields) and update that union plus the presave-derived/timestamp set (stat, size, date, decade, age_rating_metron_index, updated_at). Fields outside BULK_UPDATE_COMIC_FIELDS are dropped as before. Benchmark, 1000 mock comics: forced reimport 10.0s -> 7.0s, update_comics sub-step 4.4s -> 0.55s. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55401e6d372da3a9bf1c8b1cedeecbfd6fa06a27 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:46:01 2026 -0700 import perf: narrow existing-row queries with a selector IN, not Q-OR chains Multi-key existing-row lookups (query phase query_existing_mds, link and create pk-map builders) built per-key AND/OR Q chains — one Django filter resolution per leaf and a giant OR for SQLite to parse, per 500-900-key batch. Profiling a fresh import showed ~40% of its time in this filter machinery. Both consumers already match exact key tuples in Python from the fetched values, so the chain only ever bounded the fetch. Replace it with a single indexed IN on the model's most selective non-null key rel (MODEL_SELECTOR_REL_MAP: imprint/series name, volume series name, credit person, identifier key, story-arc name, folder path). Superset rows — the same selector value under a different parent — are harmless extra map entries. Keys with a None selector value fall back to the old chain (selector columns are non-null, so normally none). Also: read phase sub-instrumented (extract vs aggregate) and the benchmark gained --profile (cProfile per scenario). Benchmark, 1000 mock comics: fresh 12.1s -> 9.4s (query phase 2.6s -> 1.0s, link 2.5s -> 1.7s); reimport 11.8s -> 10.0s (query 3.4s -> 1.6s). Tag/link counts byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 610c51b67580006567b168e0e260c6c6097433d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:11:28 2026 -0700 import perf: skip archive opens when the fs stat snapshot is unchanged The fs prefilter compared disk mtime against the stored *embedded* metadata mtime, which is older than the file's mtime for any archive copied into the library after tagging — so the common at-rest comic never skipped and every modified-event re-poll opened its archive (an unrar spawn per CBR) just for comicbox to no-op. Reuse the stat call to also compare disk (mtime, size) against the stored Comic.stat snapshot — the poller's own modified test — and skip the worker entirely on a match. Paths with no stored embedded mtime still always survive, so a library imported with the import-metadata flag off picks up tags after the flag turns on. Benchmark, 1000 mock comics: noop poll 1.0s -> 0.2s (7.6s at baseline); zero archive opens. Reimport unaffected (force path never populates the prefilter's mtime map). Co-Authored-By: Claude Fable 5 <[email protected]> commit 06bec468cd68b423efb8f2b9da01be0f250c9d34 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:06:59 2026 -0700 import perf: skip true no-op stat-only updates (noop poll 7.6s -> 1s) _envelope_deltas included metadata_mtime unconditionally, so every file whose tags comicbox skipped still routed through the stat-only update: the comic row rewritten, its covers purged and re-queued for generation, and the import counted as changed -> cache.clear() + LIBRARY_CHANGED broadcast. A modified-event burst over an unchanged library rewrote every row and regenerated every cover. Compare metadata_mtime against the stored value, and skip the stat-only routing entirely when the envelope is unchanged AND the on-disk (mtime, size) still matches Comic.stat — the same equality the poller's SnapshotDiff uses, so a skipped file is exactly one the poller would not re-emit (no stale-stat re-poll loop). Touched files (stat moved, content same) keep today's full stat-only path, which deliberately refreshes Comic.stat via presave. Benchmark, 1000 mock comics: noop poll 7.6s -> 1.0s with "No updates necessary" (the remaining ~0.9s is archive opens for the comicbox mtime check). Fresh and reimport unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 488af595b62d643f130098f38eb2d0d8760230cd Author: AJ Slater <[email protected]> Date: Wed Jun 10 13:40:39 2026 -0700 import perf: batch-resolve comic fk links (fresh import 3x) Comic creation resolved every FK link with objects.get() per (comic, field) — folder, two protagonist lookups and ~one per FK field, ~14 single-row SELECTs per created comic. create_comics was 59% of a fresh import. prepare_fk_link_instance_maps now runs once per chunk after the FK create/update steps: one batched query per referenced field over the distinct key tuples (reusing the link phase's pk-map builders), plus in_bulk for instances. get_comic_fk_links stitches from the maps. Instances, not pks, so Comic(**md), the update path's setattr/default handling and presave's age_rating dereference are unchanged. Missing rows still raise DoesNotExist and skip the comic. The parent-folder map is also library-scoped now; the old Folder.objects.get(path=...) could MultipleObjectsReturned when two libraries shared a directory path. Also fixes a latent crash exposed by the new path: _build_pk_map_multi_key sorted key tuples that mix None with values (volume year/number_to, story-arc numbers) — TypeError when two keys first differ at a None position. Sort key now orders None last. Benchmark, 1000 mock comics: fresh 35.1s -> 12.1s; create_comics sub-step 21.7s -> 0.25s (map building 0.5s). Cumulative since baseline: fresh 2.9x, forced reimport 131.6s -> 11.8s (11x). Sub-step timing for create_and_update phases included (timed_step helper; dotted names excluded from share totals). Co-Authored-By: Claude Fable 5 <[email protected]> commit 94dd32a4ab9a74e2fd250aea22340c41ff35b5e5 Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:59:44 2026 -0700 import perf: compare fk prune links as values tuples (query 12s -> 3.7s) The fk prune select_related'd only the directly referenced FK fields, then walked each key rel via getattr chains — every hop past depth one (volume -> series -> imprint -> publisher) lazy-loaded a single-row SELECT per comic, and the protagonist check lazy-loaded main_character and main_team likewise. Query every referenced field's key rels in one batched values_list and compare row slices against the proposed key tuples. Also makes the protagonist prunable when it's the only referenced field (it was skipped before because it isn't a concrete FK for select_related). Benchmark, 1000 mock comics: reimport 23.9s -> 13.2s, query phase 12.2s -> 3.7s. With the m2m prune rewrite, forced reimport is 10x faster than the 131.6s baseline. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55ceae11efd87c421f59b616ec65bf64882d4a8e Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:53:01 2026 -0700 import perf: scan m2m prune off the through tables (10x) The m2m prune walked prefetched related objects per comic and read key attrs through instance getattr chains — for credits, identifiers and story-arc-numbers each FK in the key tuple lazy-loaded with a single-row SELECT per through row. A forced reimport of 1000 comics spent 121s (92%) in this phase. Scan the through tables directly instead: one values_list query per (field, batch) with the key attrs joined in SQL, pruned via set ops against the proposed links. No comic or related-model instances, no lazy FK loads. get_through_model moves from the link phase to const so the query phase can share it. Same pruning semantics (kept/deleted/FTS-existing/moved-source capture verified by tests/importer fixture diffs). Benchmark, 1000 mock comics: reimport 131.6s -> 23.9s; prune phase 121.0s -> 12.2s. Fresh and noop unchanged (nothing to prune). Co-Authored-By: Claude Fable 5 <[email protected]> commit 56144211fdeff41b084302bb2e7d566d9d2f16c0 Author: AJ Slater <[email protected]> Date: Wed Jun 10 02:21:36 2026 -0700 import perf phase 0: per-phase timing + repeatable import benchmark - Importer accumulates wall time per phase across chunks and logs a share table at finish (DEBUG). - bin/benchmark-import.py: deterministic mock library (seeded mock_comics) + scratch CODEX_CONFIG_DIR, times fresh / noop / force-reimport scenarios and prints per-phase tables. - mock_comics: _hex_path adler32 collisions silently overwrote ~2/3 of requested files; keep checksum dirs, name files by index. - ruff: bin/* scripts may print (replaces per-line noqas in roman.py). Baseline (1000 mock comics, ~77 tags each): reimport spends 92% in the query/prune phase (121s, superlinear); fresh spends 69% in create_and_update (24s, superlinear); noop rewrites every comic row via an unconditional metadata_mtime envelope delta. Plan re-ranked in tasks/todo.md; pipeline-overlap idea dropped (read is ~8% of fresh). Co-Authored-By: Claude Fable 5 <[email protected]> commit 3d7c290e0cba1bb4883a7a3bbdd5379d2e5fc820 Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:51:28 2026 -0700 remove unused css commit 9e7df00b27a2074563482f4d788e3a19017b2c59 Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:49:12 2026 -0700 age rating As-tagged tab: standardized column first, no parens Swap the two columns: the dim standardized equivalent now leads in a fixed-width left column (10ch, so the raw tags align), the raw tagged value follows it, and the parentheses are gone. The column renders empty for unmapped tags to preserve alignment. Co-Authored-By: Claude Fable 5 <[email protected]> commit a6b9de4e63192d1c13a264b8650a9cec3cb98c6e Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:48:00 2026 -0700 update comicbox commit 09237388499591adf5c044dcf2384cdaf693d27f Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:47:01 2026 -0700 0043: relink AgeRating.metron FKs with the current comicbox mapping AgeRating.metron is derived once, in presave at row creation, so mappings comicbox gains later (e.g. Rating Pending -> Unknown in 4.0.0a2) never reach existing rows. Add a data migration that recomputes every FK with the live comicbox lookup and heals the two denormalizations that derive from it: Comic.age_rating_metron_index, and the FTS age_rating_metron column by bumping affected comics' updated_at past the FTS watermark so the next search sync refreshes their entries. Co-Authored-By: Claude Fable 5 <[email protected]> commit 93475e7104e97b180202124353a9680856ba097f Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:44:49 2026 -0700 fix quadratic folder browse: keep cover_mtime coalesce out of GROUP BY Django adds non-aggregate Func annotations to the GROUP BY wholesale, so the Coalesce(custom cover, comic cover) mtime subqueries were evaluated once per pre-aggregation joined row. In folder mode the root folder's ancestors M2M fans out to every descendant comic, and each evaluation re-scanned that same set: an 18k-comic folder took minutes to browse. Delegate get_group_by_cols to the source subqueries (their correlation column is already in the GROUP BY), restoring bare-Subquery behavior: 0.7s. Co-Authored-By: Claude Fable 5 <[email protected]> commit f19552423ab4e009ba6bbb812cddeec570b6d1ab Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:31:16 2026 -0700 age rating As-tagged tab: right-justify the standardized column Move the parenthetical standardized equivalent out of the row title into a right-aligned append-slot column, so the As-tagged list reads as two columns: raw tag left, "(Standardized)" right. Co-Authored-By: Claude Fable 5 <[email protected]> commit 73088c6ff2ecd4145d9970ff2d8a2f02d79bac43 Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:19:34 2026 -0700 age rating As-tagged tab: show all tagged values with standardized name The As-tagged tab previously hid every tagged value that had a Metron mapping, leaving it nearly empty (often just Rating Pending). Now it lists every raw tagged age rating, annotating each with its standardized equivalent in parentheses when they differ, e.g. "MA15+ (Teen Plus)". The tab is hidden only when the library has no tagged values at all. The old right-aligned metronName column, its None-row header hack, and dead taggedColumnHeader styles are removed in favor of the inline parenthetical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 2de448a635b5918843a785f008bacc6828868275 Author: AJ Slater <[email protected]> Date: Tue Jun 9 22:01:03 2026 -0700 remove out of date help commit 09484cc5970fb7116846befe384ebe2e5f525ce0 Author: AJ Slater <[email protected]> Date: Tue Jun 9 21:46:41 2026 -0700 age rating filter sub-menu: replace expansion panels with tabs The Standardized / As-tagged split now renders as compact tabs instead of accordion panels. When no unstandardized tagged values exist the tab bar is hidden entirely and only the Standardized list shows. A primary checkbox icon on a tab marks filters with active selections, and the shared search switches tabs when matches are exclusive to the other tab. Co-Authored-By: Claude Fable 5 <[email protected]> commit d44b81697389ebed8330c7591ac08607a14b06d8 Author: AJ Slater <[email protected]> Date: Tue Jun 9 21:12:34 2026 -0700 fix importer tests leaking an open class-level transaction BaseTestImporter.tearDownClass never called super(), so TestCase's class atomics were never rolled back. The import run's open transaction (and its SQLite shared-cache table locks) leaked into every subsequent test, making fts_rebuild's PRAGMA wal_checkpoint fail with 'database table is locked' in full-suite runs. Also run FtsRebuildTests as TransactionTestCase: fts_rebuild checkpoints the WAL, which is incompatible with any wrapping transaction that has already executed statements. Production calls it from the janitor in autocommit, so the unwrapped test matches real semantics and is robust against future fixtures touching the DB first. Co-Authored-By: Claude Fable 5 <[email protected]> commit 835b6d020116c03ae409413726d42abd6d9d52fa Author: AJ Slater <[email protected]> Date: Tue Jun 9 20:34:23 2026 -0700 remove duplicate status commit 82c1128f3ce8583ebcd632377acab2696d8f82e1 Author: AJ Slater <[email protected]> Date: Tue Jun 9 20:32:40 2026 -0700 fix basedpyright deprecation warnings: Iterator -> Generator for contextmanagers Co-Authored-By: Claude Fable 5 <[email protected]> commit 66ab9f913fe2bc0fc08fa3d785b2b6baea6bb5fd Author: AJ Slater <[email protected]> Date: Tue Jun 9 11:54:21 2026 -0700 Squashed commit of the following: commit 3a339a4a016f0f16055cf086ee085b6e02a23ac2 Author: AJ Slater <[email protected]> Date: Tue Jun 9 11:54:06 2026 -0700 update deps and version to 2.0.0a9 commit 2923da8cfec40ca9c56bd1a28ed6bcb6b1cac756 Author: AJ Slater <[email protected]> Date: Tue Jun 9 01:37:03 2026 -0700 refactor(librarian): log FTS rebuild from the shared fn for every caller fts_rebuild() was the only integrity helper that logged nothing itself, relying on JanitorDBFTSRebuildStatus.log_success to surface a generic status line. Its three siblings (integrity_check, fts_integrity_check, fix_foreign_keys) each log a specific success inside the shared function so any caller benefits. Match them: log the rebuild success in the shared fn and drop the status's log_success, so the wrapper emits one generic INFO line plus the specific SUCCESS line like the other integrity tasks. No live caller bypassed the wrapper -- this is a symmetry/defensive change. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 099d4ba1eafccf1a399782d3c431da3ba868ced4 Author: AJ Slater <[email protected]> Date: Tue Jun 9 01:31:45 2026 -0700 fix(librarian): log snapshot/restore and failed-import re-queue for every caller Completion logs for these librarian operations lived in a single caller's wrapper, so other entry points ran silently: - User-data snapshot logged only from the nightly janitor wrapper, so the admin "Snapshot now" button produced a backup with no log. Move the log into the shared snapshot_sidecar(); restore() now logs a completion line (covering the admin view and the restore_user_data CLI command). Drop the now-redundant janitor log. - force_update_all_failed_imports (admin "Failed Imports" button) had no status and no log, running silently -- especially on the zero-failed no-op. Log the queued total once, mirroring force_update's sibling. Add regression tests asserting both shared paths emit a summary line. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit dd3d57bdc241bb8b3c1e2630bf95792c34dd9fe7 Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:41:55 2026 -0700 Squashed commit of the following: commit c009b02c3147ebf30bd0c0f86b1f46d6dbab4f5a Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:41:35 2026 -0700 format md commit bd4bf37372f32d6026260771636b94764681ea61 Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:41:29 2026 -0700 update deps commit f40076874f7fa1886fbad1f0046c73079df1eaaa Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:30:40 2026 -0700 update version to 2.0.0a8 commit d394064c062bc562640bcdda7c5201c9c911ec91 Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:30:18 2026 -0700 docs: require a scribe priority for new librarian jobs New ScribeTask/JanitorTask classes must be registered in _SCRIBE_TASK_PRIORITY (and janitor jobs in _JANITOR_METHOD_MAP + _NIGHTLY_TASK_CLASSES). Notes the ValueError symptom when omitted and the test_scribe_priority.py guard. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c7429fc42ad1667d2aeeca823496cc20509244b5 Author: AJ Slater <[email protected]> Date: Tue Jun 9 00:30:11 2026 -0700 fix(librarian): stop nightly janitor crashes from unranked task and stale show-flags JanitorFolderRelationsCheckTask was dispatched via _JANITOR_METHOD_MAP / _NIGHTLY_TASK_CLASSES but absent from _SCRIBE_TASK_PRIORITY, so get_task_priority's tuple.index raised ValueError when it was queued and the folder-relations repair never ran. The user_data sidecar dump/restore still read SettingsBrowserShow.{p,i,s,v}, renamed to {publishers,imprints,series,volumes} in migration 0044, so every SettingsBrowser row failed to serialize and was silently dropped from the nightly snapshot (and would fail on restore). The sidecar SQL columns stay show_{p,i,s,v} (storage keys, no schema migration). Add regression tests for both, each proven to fail pre-fix: test_scribe_priority.py enforces _JANITOR_METHOD_MAP keys are a subset of _SCRIBE_TASK_PRIORITY; test_user_data_dump/restore now create a real SettingsBrowser+Show and round-trip the show flags. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 0356e309401d8ec78b5e7c108c7d60ec2c9e7d0f Author: AJ Slater <[email protected]> Date: Mon Jun 8 20:40:20 2026 -0700 fix(types): clear basedpyright and ty errors in folder-relations code make typecheck (basedpyright) reported 10 errors and make ty 1; both now pass cleanly. - foreign_keys.py: django-stubs types Field.remote_field as Field, which hides the M2M .through attribute. Cast remote_field to the real ManyToManyRel so both checkers resolve it, dropping the old ty:ignore. - test_janitor_folder_relations.py: ignore reportUninitializedInstanceVariable on setUp fixtures (matching test_metadata_path.py); None-narrow the optional parent_folder FK before .path; use parent_folder.pk instead of the plugin-only parent_folder_id companion attr basedpyright can't see. - test_metadata_path.py: annotate _get_path -> str and coalesce the optional .get("path") so .startswith() no longer trips Optional access. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c3a7ac6c857baa8c94dca5e81c1ad81229b06d69 Author: AJ Slater <[email protected]> Date: Mon Jun 8 20:24:01 2026 -0700 refactor(librarian): drop folder repair from startup; fix job descs Startup queued JanitorAdoptOrphanFoldersTask alongside the search-index sync, but structural folder repair doesn't belong at boot: it's an idempotent no-op when the DB is consistent, only drifts from botched imports (not offline filesystem changes), and logically follows a re-poll rather than preceding it. Adopt-folders also already queues its own SearchIndexSyncTask, which startup runs anyway. Both folder repairs (adopt + folder-relations) still run nightly and on demand. Startup now reconciles only the search index against the DB. Also corrects two Jobs-menu descriptions that overstated startup: - cleanup_tagging_state never ran the full task at startup (only the online-tag thread clears its scan marker); now "Runs nightly." - adopt_folders updated to match the behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 2378794c7526c36c77b2cec481a32938ad94dd7d Author: AJ Slater <[email protected]> Date: Mon Jun 8 19:57:35 2026 -0700 format commit 3c2e0b22d8e24d6f4d374b2ee7eeb8b4ff1a699e Author: AJ Slater <[email protected]> Date: Mon Jun 8 19:56:26 2026 -0700 bump version 2.0.0a7 commit b8226af7eadc386ece5b04c1fa66156b5ae21d87 Author: AJ Slater <[email protected]> Date: Mon Jun 8 19:55:12 2026 -0700 more spacing for button commit 08833e9c2cff4b1e64917ce8b9ca04ccf83ee8ae Author: AJ Slater <[email protected]> Date: Mon Jun 8 19:51:08 2026 -0700 refactor(admin): move Database jobs section above Search Index Reorder the ADMIN_JOBS menu groups so Database precedes Search Index. Pure reorder of the two group dicts; no behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8e4eae3702531996be49a1fefa9e2169420e78b0 Author: AJ Slater <[email protected]> Date: Mon Jun 8 19:49:04 2026 -0700 feat(admin): add Remove Orphan Settings cleanup job JanitorCleanupSettingsTask (status JAS) ran only via nightly maintenance — unlike every sibling cleanup, it had no standalone admin Jobs button. Wire it into the Jobs menu: - Map cleanup_settings -> JanitorCleanupSettingsTask in the dispatch map - Add the "Remove Orphan Settings" button to the Cleanup group - Add JAS to the nightly-job status list so Run Nightly Maintenance surfaces its progress The serializer's valid-choice set and the frontend menu both derive from ADMIN_JOBS, so the button alone makes the task POST-able. Also trims the db_folder_relations_check job description. Co-Author…
The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <[email protected]>
_get_move_create_folders_one_layer returned bare path strings, but
bulk_folders_create expects key tuples (MODEL_REL_MAP[Folder] == (("path",),
...)) and iterates each entry to extract the path. Bare strings were iterated
character by character, producing single-char paths like PosixPath('K') whose
stat() raised FileNotFoundError and crashed the ScribeThread during import.
Wrap the path in a tuple to match the contract, and add a regression test
covering a folder move whose destination parent isn't yet a tracked Folder.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Admin Tagging tab now shows a live status table for the online tagging batch: per-comic state, per-source rate-limit countdowns, batch ETA, and match-review status. Backed by a daemon-published session snapshot in the tagging cache, exposed via GET /api/v4/admin/tag-sessions/snapshot and refreshed over the existing task.progress websocket. - Snapshot: session_snapshot.py builds a capped, JSON-safe per-comic snapshot from OnlineTagOutcomeStats + pending prompts; published throttled from the runner, frozen inactive on finish, cleared at startup / janitor. - Session lifecycle: pause / resume / abort / dismiss controls in the table, wired to the online-tag store and admin endpoints. - Match-review fixes: - direct-id fetch for explicit picks (no rate-limit-prone re-search drift) - record user_matched / user_skipped outcomes, overlaid on the snapshot - client guard against resurrecting just-answered prompts in the dialog - optimistic local overlay so resolutions show during rate-limit stalls Tests: backend snapshot / resume / session-manager / janitor suites; frontend store + status-table specs. ruff/ty/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
The Folder model annotates parent_folder as ForeignKey explicitly, so django-stubs doesn't synthesize the implicit parent_folder_id attribute. Compare the FK object instead (Django compares models by pk). Pre-existing since ce1fd81; behavior unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]>
commit 5e6f4dfc55ee4ad3f8fab48b7b33ffe02f17d8a4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:21 2026 -0700 update version 2.0.0a13 commit 59d607d6c886cccb093fbc07dea5a2ef97ab1dc0 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:25:51 2026 -0700 test(onlinetag): exercise replay path in unmatched-write test The candidate carried issue_id 123, routing the resolve through the explicit-id fetch (real fetch_tags_by_explicit_id) instead of the replay session the test sets up and claims to verify. That crashed on the empty touched .cbz before any assertion ran. Drop the issue_id so the candidate falls back to the replay path guarded by _first_tags. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit dfa2349e8b821fbaa93b53fb4639bd2d66db8522 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:19:14 2026 -0700 fix(mail): ASCII hyphen in noqa comments to dodge complexipy panic complexipy's extract_comment_marker slices noqa comment text by a fixed byte offset (16); a multi-byte em-dash straddling that offset lands mid-char and panics the Rust core, aborting the whole run. Swap the two em-dashes in the noqa comments for ASCII hyphens. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 1799ce937559d62a667a0bf66f819f1593c93c5e Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:16:34 2026 -0700 decomplexify test for radon commit 6213c0afe641942e9e1d28f090970c6da524ceb4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:14:19 2026 -0700 remove unused ty ignore commit dc59f1dc9560e177f9e0c36ea4552208014746d6 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:13:38 2026 -0700 remove redundant definition commit 5d7a204d5069783c8638bc8f76f84bf15840239a Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:10:36 2026 -0700 Squashed commit of the following: commit 1274bad663bf382d76453e65a6fc935b8b3d4559 Merge: a8aca7b33 72fad6861 Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:57:33 2026 -0700 Merge branch 'main' into develop commit 72fad6861b0af14731847b5444f097fdec699fda Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:06:27 2026 -0700 v1.12.8 (#781) commit a8aca7b3341e7afdee8e99829be88d77e4d988c5 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:50:21 2026 -0700 fix(test): call super().tearDownClass() in BaseTestImporter Skipping super() left the class-level atomic from setUpTestData uncommitted-but-unrolled-back, leaking the codex_init() "admin" superuser into later test classes and causing UNIQUE constraint failures (e.g. test_bookmark_filter_isolation) in full-suite runs. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit d7570ecf99e1a15449dec2f7a0550122b9d405df Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:16:59 2026 -0700 remove gitignore commit be211c5724d5d7cc25ca367813fa34244097f372 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:12:03 2026 -0700 fix(types): narrow OPDS redirect settings to Mapping for ty Replace a stale `ty: ignore[no-matching-overload]` with an isinstance guard so `settings` is a Mapping before `_copy_params_into`, clearing the `invalid-argument-type` diagnostic from `make ty`. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5dd3a0dd9feb18aa4b13c15b000c048befdeceef Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:09:05 2026 -0700 update devenv commit e5a2a41881d35e2d8716bd6dfefdd64fa3792b5c Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:07:34 2026 -0700 shfmt commit 1f867c1497e4edc14315d63c7eeb8c3e2b0f07bd Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:05:38 2026 -0700 shfmt commit c5366787b56212e12a69c5aaab1776474962a31b Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:02:56 2026 -0700 fix(bookmark): scope read/unread filters per user (v1.12.8) The UNREAD filter used Q(bookmark=None), a multi-valued relationship test for "comic has no bookmark from anyone" rather than "no bookmark for me". Once any user finished a comic it dropped out of every other user's (and anonymous visitors') unread view. UNREAD now negates the per-user finished predicate (~(my_filter & finished=True)), compiling to a per-user NOT-EXISTS subquery. Also harden get_my_bookmark_filter: an anonymous visitor with no established session_key resolved to `session_id IS NULL`, which matches every authenticated user's bookmarks. Return a match-nothing filter instead; a session key is minted only when a bookmark is written. Adds tests/test_bookmark_filter_isolation.py covering cross-user and anonymous isolation for READ/UNREAD/IN_PROGRESS. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 541907b6708dfbbde197432960c6adc6dcf44b02 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:44:58 2026 -0700 fix(frontend): convert v-dialog em widths to px so dialogs aren't slivers Vuetify 4's v-dialog defaults to location 'center center', which activates the connected overlay location strategy. That strategy caps width with parseFloat(props.maxWidth), so "20em" parses to 20 and the em is dropped, collapsing dialogs to a ~20px sliver. Convert every em-based max-width/ min-width on a v-dialog (or forwarded via $attrs) to unitless px at 16px/em. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5918d0b621cd635f3de32f142d4dff7ec9c45242 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:41:25 2026 -0700 move VPullToRefersh out of labs commit 57d5cf3888c2dc0aab55e46447b41a459d37003a Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:32:39 2026 -0700 update devenv and deps commit cd944f0e4cf7044ba8e72cb6f90539ca892adb72 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:25:16 2026 -0700 Squashed commit of the following: commit b5874b67774af29f43603f01690a0ca97b43c46a Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:24:54 2026 -0700 update deps commit e0a16dff8e66c357b2d2aa5e4ff3ea7decce4572 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:23:51 2026 -0700 remove pytest-gitignore commit 35b84324300352499f3c455aa43201c3f475cd21 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:20:46 2026 -0700 update version to 2.0.0a12 commit e3e1167851960a42fef9a5ceba3b3b5470f7a56c Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:19:54 2026 -0700 Fix basedpyright error on Folder FK access in moved-parents test The Folder model annotates parent_folder as ForeignKey explicitly, so django-stubs doesn't synthesize the implicit parent_folder_id attribute. Compare the FK object instead (Django compares models by pk). Pre-existing since ce1fd81ad; behavior unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 0ee9c966f9d1dcfd84d36e6077778fd9354d9574 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:45 2026 -0700 update deps commit 38f527b57ce30be0f162d7aae2bf51d938e3141b Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:39 2026 -0700 ignore benchmark dir commit 7007c109accb1654c5b61580336d71cedfb5ce3f Author: AJ Slater <[email protected]> Date: Sun Jun 14 17:56:39 2026 -0700 Add online tagging status table with live progress and session controls Admin Tagging tab now shows a live status table for the online tagging batch: per-comic state, per-source rate-limit countdowns, batch ETA, and match-review status. Backed by a daemon-published session snapshot in the tagging cache, exposed via GET /api/v4/admin/tag-sessions/snapshot and refreshed over the existing task.progress websocket. - Snapshot: session_snapshot.py builds a capped, JSON-safe per-comic snapshot from OnlineTagOutcomeStats + pending prompts; published throttled from the runner, frozen inactive on finish, cleared at startup / janitor. - Session lifecycle: pause / resume / abort / dismiss controls in the table, wired to the online-tag store and admin endpoints. - Match-review fixes: - direct-id fetch for explicit picks (no rate-limit-prone re-search drift) - record user_matched / user_skipped outcomes, overlaid on the snapshot - client guard against resurrecting just-answered prompts in the dialog - optimistic local overlay so resolutions show during rate-limit stalls Tests: backend snapshot / resume / session-manager / janitor suites; frontend store + status-table specs. ruff/ty/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit ce1fd81adb4823ac454950ab66905a38e3f6d9e1 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:56:39 2026 -0700 Fix ScribeThread crash on folder move under untracked parent _get_move_create_folders_one_layer returned bare path strings, but bulk_folders_create expects key tuples (MODEL_REL_MAP[Folder] == (("path",), ...)) and iterates each entry to extract the path. Bare strings were iterated character by character, producing single-char paths like PosixPath('K') whose stat() raised FileNotFoundError and crashed the ScribeThread during import. Wrap the path in a tuple to match the contract, and add a regression test covering a folder move whose destination parent isn't yet a tracked Folder. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 265988a4b44ba7ebe9c810bedc11a42796853d90 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:30:44 2026 -0700 Delete duplicate migration 0044 that broke test-DB setup The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e6), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c31c61e6175d24d9e1dde1835c7235c367c251c9 Author: AJ Slater <[email protected]> Date: Sat Jun 13 02:00:11 2026 -0700 fix some kind of merge issue commit 92e526940e96b6ea9679363159395270233e75f2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:55 2026 -0700 Squashed commit of the following: commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:29 2026 -0700 refactor(complexity): clear radon cc rank-C findings bin/lint-complexity.sh reported three rank-C functions; extract helpers to drop each below the gate (radon cc --min C now empty). - AdminTagByIdView.post: pull primary-identifier parsing and merge extra-id resolution into _resolve_primary / _resolve_extra_ids. - OnlineTagSessionManager._apply_resolution: extract _handle_unresolved and _enqueue_resolved_write. - test_update_protagonist: extract repeated protagonist assertions into a helper (radon counts each assert as a branch). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 55bb659fcc8c02e73c20a34823652a4f81be1d56 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:46:04 2026 -0700 refactor(migrations): squash 0044 into 0043 Fold the merge_all_sources field into the ComicboxTaggingDefaults CreateModel in 0043 and drop the standalone 0044. makemigrations --check reports no drift. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 7a988737a63286cc85396346579e124e5fff8e5b Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:44:59 2026 -0700 feat(onlinetag): merge metadata from all sources Add an opt-in "merge all sources" mode that queries every enabled online source per comic and merges the results (comicbox first_wins=False) for maximum tag completeness, instead of stopping at the first match. - Admin tagging defaults: merge_all_sources boolean (default off), settable as the default and overridable per scan. - Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate + rate-limit warning multiply calls by source count (kept in sync with estimate.py) so the doubled API cost is visible. - By-ID: chips input accepting multiple URLs/ids; merging fetches two explicit ids (one per source) and merges them. Source select shows only when a token can't be auto-identified (bare number). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:34:56 2026 -0700 update deps commit 473221d7a09403b1848f202d55902b71e31319c2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:22:58 2026 -0700 fix(lint): clear ruff/pyright findings; fix benchmark import path - tests: add docstrings, name magic estimate values, use pytest.raises and out-of-class exception message - onlinetag: collapse implicit f-string concatenation in log call - benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*) which was a real runtime ImportError in library generation Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 45a9fde6076a5b0b15be889769e9a55af4b0678a Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:13:49 2026 -0700 refactor(migrations): squash 0044 + 0045 into 0043 Fold the comic-leading composite Bookmark indexes (former 0044) and the LibrarianStatus eta/retry_at fields + JFR status-type choice (former 0045) into the consolidated 0043 migration; delete both files. makemigrations --check reports no changes against the model state. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 22da6b9cd85615f3d7b8e7043818babac313d24e Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:35 2026 -0700 update deps commit 6f27454623e43cccd95253d5dfe66471820c9349 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:15 2026 -0700 fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns Two online-tagging bugs plus a status-UX overhaul. Prompt answers given while a scan was running were lost on refresh: the response task sat in the queue behind the busy OnlineTagThread, so the cache kept the prompt. Now the drain loop removes answered prompts from the cache inline (race-free, single-threaded) and marks the fingerprint so the running scan stops re-persisting it; the network re-fetch + write is deferred until the scan releases the thread. A rate-limited lookup looked hung: collect_results could raise mid-pass (budget exhausted, network error) without ever calling finish(), leaving the status row frozen forever. finish() now runs in a finally. The rate-limit status was a static "rate limited ~57s" — a second subtitle writer clobbered the parseable one, and nothing refreshed it during comicbox's blocking sleep. Replaced with two absolute-timestamp countdowns the admin UI ticks down to live: - retry_at: time until the next online request, re-anchored per attempt. - eta: total time remaining, carrying forward the launcher dialog's estimate (ported verbatim to estimate.py), re-estimated on each comic completion and pushed out by rate-limit waits. Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in the pending status_type choices drift), plumbed through Status and StatusController. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit eccf41bc603b118bdbd63f4e4918a30da87140ae Author: AJ Slater <[email protected]> Date: Fri Jun 12 22:39:15 2026 -0700 feat(tagging): admin-orderable online source priority defaultSources order is now run priority end to end. comicbox runs sources in the given order under first-wins, so the admin can choose which source is primary and which is fallback: - Tagging tab renders the source rows in defaultSources order with per-row up/down arrows; enabling a source appends it at lowest priority. A hint explains that the first matching source wins. - The launcher dialog normalizes its checkbox selection to the admin's priority order before starting a session (checkbox v-model records click order, not priority). - Serializers validate defaultSources / sources: known names only, deduped, order preserved (unknown source → 400). - explicit_id / session settings pass ordered tuples to comicbox's retyped OnlineLookupSettings.sources. Co-Authored-By: Claude Fable 5 <[email protected]> commit 02869ddf4d027c325637dd41081199c59f1141ad Author: AJ Slater <[email protected]> Date: Fri Jun 12 17:06:25 2026 -0700 add METRON_INFO to default formats to write commit a214c9c93cf679194beeb5c42eb66deea933a3da Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:36 2026 -0700 update deps commit 65a294362171bded71547430dd51b322ea83dcef Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:08 2026 -0700 don't make abort button red commit de49b5dd9529446ab34a7468bda694fae39c7a00 Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:52:14 2026 -0700 fix(frontend): remove duplicated Tag Write Errors sidebar item admin-menu.vue contained the same tag-write-errors CodexListItem twice (re-added in a later squash while the original stayed), so any error rendered two identical sidebar entries. One item per error kind; the counts live in the admin tables. Co-Authored-By: Claude Fable 5 <[email protected]> commit e0bd853d9ed1991859849e11d85d487409dc1194 Author: AJ Slater <[email protected]> Date: Fri Jun 12 13:43:19 2026 -0700 fix(onlinetag): stop spurious rewrites and prompt loss in online tagging Three interacting bugs broke a single-comic tagging run: - Pass-1 and prompt-resolution writes now require result.matched (new comicbox flag): unmatched results carry the comic's merged existing metadata, so gating on truthy tags re-wrote files with no new information — which also triggered a re-import mid-session. - Pending prompts and tag-write errors move to a dedicated 'tagging' cache (codex.cache.tagging_cache). The session_cache docstring claimed key namespacing protected them from cache.clear(), but Django's file-based clear deletes everything: every import that changed anything (importer finish), Library/Group CRUD, and startup wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'. - Prompt resolution builds its replay session with defer_prompts=True so the preloaded resolution is actually consulted (comicbox only installs its cache-reading selector with defer or a handler; without it an ambiguous re-search fell through to comicbox's interactive CLI prompt inside the daemon). A fingerprint miss — the re-search returned different candidates — now re-queues the fresh deferred prompt instead of dying silently. Co-Authored-By: Claude Fable 5 <[email protected]> commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:45:22 2026 -0700 feat(admin): auto-enable a tagging source when its credentials are first saved Saving credentials for a source that previously had none now adds it to default_sources via the auto-saving draft, so a freshly configured source works without a second checkbox click. Re-saving credentials for an already-configured source leaves the enable checkbox untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:38:11 2026 -0700 Squashed commit of the following: commit 9543008f24441b6b2992f0c818d019f42135f7df Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:21:24 2026 -0700 dockerignore benchmark dir commit 00134a40fe1c37d96e64066735cb42cc03f463c8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:20:24 2026 -0700 mv fixutres to benchmark dir commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:54 2026 -0700 change import path for mock comics commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:05 2026 -0700 ignore fixtures dir commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:11:25 2026 -0700 common subdir for mock & benchmark commit cd768965a544069fb623f9798a2ef7f472016586 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:01:01 2026 -0700 move benchmark import out of ci commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:52:32 2026 -0700 v2.0.0a10 commit 43b6042bca9c546d49268a52002efa4db69651c4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:51:47 2026 -0700 update deps commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:39:57 2026 -0700 fix typecheck and ty warnings Merge implicit f-string concatenations, add missing @override on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:28:58 2026 -0700 remove bad news line commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:18:54 2026 -0700 adapt to comicbox 4.0.0a3 API Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]> commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251 Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:55:25 2026 -0700 update deps commit 919fad63cf5790b82275e5f20207e38f65934a6c Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:48:25 2026 -0700 update deps commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e Author: AJ Slater <[email protected]> Date: Thu Jun 11 14:00:48 2026 -0700 poller: apply discarded .only() projection in manual poll path _handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]> commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 07:52:01 2026 -0700 admin: fix stale Last Poll column after websocket table refetch DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <[email protected]> commit 188dd22775d6beb284b6ddafb3629dcb0affbccb Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:18:16 2026 -0700 edit news for brevity commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:17:05 2026 -0700 NEWS: browser performance entry for 2.0.0 Co-Authored-By: Claude Fable 5 <[email protected]> commit 994d6a586c249376827c1d72494071b4281528fa Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:15:27 2026 -0700 news: v2.0.0 performance section for the import speedups Co-Authored-By: Claude Fable 5 <[email protected]> commit cbb07c980bd5bb317e7a506bec81ba8f66415163 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 cards: Max bookmark timestamp aggregate instead of JSON array + parse loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <[email protected]> commit 01a9394d8ecaaa22e72885b4257545a82220b3d4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 breadcrumbs: batch the folder ancestor chain in one query The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <[email protected]> commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <[email protected]> commit 578807ab0294391095e2f6b6fe3d1b1540f643a4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:55:07 2026 -0700 search: materialize the FTS match set once per browse request The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <[email protected]> commit e56b7eb2dd54cab402041de418832f865269540c Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 lint: format follow-ups for the order/cover perf commits Co-Authored-By: Claude Fable 5 <[email protected]> commit ca67645e4bf757568021e19781d9179ae599bda2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 browser: move the last-route write off the read path; settings row fetched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <[email protected]> commit 94ef67e41554869cfe4c19494cea7e422ff5d30b Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:10:00 2026 -0700 covers: batch cover mtimes post-pagination instead of a second subquery cover_mtime was a .values('updated_at') Subquery over the identical correlated cover queryset that already computes cover_pk — two full executions of the most expensive expression per card (measured 40-49% of the card query across publisher/folder/search pages). Drop the annotation; resolve the representative comics' updated_at after pagination with one indexed pk__in batch over the page's cover_pks and attach it to the cached instances (the serializer iterates the same objects). Custom-cover mtimes keep their cheap rowid subquery and still win the coalesce. _CoverMtimeCoalesce is gone with the annotation that needed it. Responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 browser: zero-pad from page rows; skip dead card serialization in table mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <[email protected]> commit d1789eda2ac5a4351c3039ee42d4677db1c669c6 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 table mode: evaluate the collection page query once, not twice compute_collection_intersections read the page pks via values_list('pk'), a clone that re-executed the entire annotated/ grouped/sorted collection query; the serializer then ran the original queryset again. Iterating the queryset itself primes its result cache, which the serializer reuses — the heaviest table-mode query now runs once per request. Co-Authored-By: Claude Fable 5 <[email protected]> commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 comic listings: stop aggregating order aliases; GROUP BY pk only Two GROUP BY-width fixes for Comic-model querysets: - _alias_filename aggregated Min/Max over the comic's own single path, which turned the alias into an aggregate and dragged every selected column into GROUP BY inside the listing and both cover subqueries. A comic is its own file; use the bare expression (as annotate_comic_extra_specials already did). A filename-ordered folder browse measured 135s before, 0.6s after, identical rows. - The ids JsonGroupArray made Django compute GROUP BY over all ~50 selected Comic columns including TEXT blobs. Force GROUP BY id — every other column is functionally dependent on the pk, so groups are identical and SQLite sorts a one-column key (25-34% off the books query). Not applied to the cover path, where the forced literal column doesn't survive nested-subquery aliasing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 browser: UNREAD filter must not hide comics other users finished The UNREAD arm was Q(bookmark=None) | (mine & unfinished): bookmark=None matches only comics with no bookmark from ANY user or session, so a comic finished by someone else (or a stale anonymous session) matched neither arm and silently vanished from my unread listing (reproduced: 50 foreign finished bookmarks shrank my unread count by 50). UNREAD now probes 'I have no finished bookmark on this comic' with a correlated NOT EXISTS scoped to the requesting user/session. It correlates on the queryset's comic pk, so on collection querysets it binds to the same joined comic row as the other filters in the combined .filter() call — group rows still require ONE comic that satisfies every condition (pinned by test). New comic-leading composite indexes (comic,user)/(comic,session) keep the probe off the user index: under stale sqlite_stat1 the planner flipped a user-scoped probe onto it — a measured 17s vs 27ms plan. EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user (comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged; single-user responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit cad52d79e8242e3b69e4cbd604757f2c7faab728 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 metadata: stop GET writing m2m through tables and corrupting tags _copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True) for every real m2m field on the Comic path — obj is a live saved instance (the 'values dicts' comment was stale), so a GET /metadata DELETE+INSERTed up to 12 through tables, and a multi-comic selection permanently rewrote the first comic's tags to the selection's intersection (reproduced: 339 characters silently became 43). Serve the intersection queryset through the instance's prefetch cache instead: RelatedManager.get_queryset consults it keyed by field name, so the serializer reads the already-optimized queryset query-free. Response verified byte-identical; the 38-query identifier N+1 (the post-write re-read discarded select_related) disappears with it. Co-Authored-By: Claude Fable 5 <[email protected]> commit 71b52749a3b24a8493c6803f0c3cda04466280d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 browse: bind favorite collection code as str so cachalot can cache Binding the Collection StrEnum member itself made cachalot's parameter-type check raise UncachableQuery for every query the favorite subquery landed in — silently uncaching all browse queries. Repeat table-mode browses re-paid ~10s of SQL every time; with .value they serve from cache (16-22ms, 2-3 queries). The bound SQL value is identical either way. Co-Authored-By: Claude Fable 5 <[email protected]> commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 metadata: pk subquery instead of inlined IN lists; count the filtered relation Two fixes to the intersection queries behind GET /metadata: 1. _get_comic_pks returned a materialized frozenset, inlining one bound SQL variable per comic per union arm. With 12 m2m arms, any collection over ~2,730 comics exceeded SQLite's 32,766-variable limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and 17.8k comics). Passing the distinct .values(pk) queryset keeps the statement constant-size (36 params); the same union measured 101ms at 8,576 comics. num_comics is computed once via COUNT and threaded to both consumers. 2. _get_fk_intersection_query counted the hardcoded 'comic' reverse m2m even when the filter traversed the main_*_in_comics FK back-relations — a semantically wrong HAVING over the unfiltered relation that also forced a separate unindexed join (main_team measured 2,069ms -> 12ms counting the filtered relation). Co-Authored-By: Claude Fable 5 <[email protected]> commit 5beb474835c7375f4b726eca63aa05f2ef8b9966 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 arc browse: demote story-arc chain joins to INNER for StoryArc queries With only codex_comic demoted, SQLite planned arc search browses from the arc side and built a cartesian product against the FTS match set: arcs root with a common search term measured 156s (main query 150.8s). Demoting codex_storyarcnumber and the M2M through table lets the plan start from the FTS/comic side (36ms). Scoped to StoryArc-model querysets: there the chain IS the collection relation, so NULL-extended rows are exactly the empty collections force_inner_joins exists to drop. On other models these tables appear as table-view annotation LEFT JOINs, where demotion would drop rows whose comics have no arcs. Co-Authored-By: Claude Fable 5 <[email protected]> commit 1434a5a771fc7445472cb52726f4e86c645df999 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 cover subqueries: join FTS rank only when ordering by search_score The direct fts_q join exists solely to populate codex_comicfts.rank for the cover subquery's search_score ORDER BY, but it was applied on every search regardless of order key — and the MATCH re-executed once per correlated cover evaluation. The pre-materialized fts_sq membership test already filters correctly. A name-sorted search browse measured 135s before, 6.5s after (byte-identical rows); rank-ordered searches keep the join and pick the same top-ranked cover as before. Co-Authored-By: Claude Fable 5 <[email protected]> commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:40 2026 -0700 table mode: keep intersection-sort subqueries out of GROUP BY RawSQL.get_group_by_cols() returns [self], so the correlated intersection-sort subqueries landed in the GROUP BY of aggregated collection queries. Folder is the one collection model without a forced-GROUP-BY entry, so SQLite evaluated the subquery once per pre-aggregation joined row — at folders root the ancestor M2M fans out to every descendant comic (366s request; each of the two queries measured ~256s alone, 85ms after). Same mechanism as the _CoverMtimeCoalesce fix, recurring through a different expression type. _IntersectionSortRawSQL overrides get_group_by_cols() to return []: the subquery's only outer reference is the collection pk, which is always grouped, so grouping is unchanged (verified row-identical). Co-Authored-By: Claude Fable 5 <[email protected]> commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:14:18 2026 -0700 fix librarian status feed order: cursor pagination discarded the order_by The sidebar progress feed showed import phases scrambled — read next to the search statii, query below create. The viewset ordered active rows by (preactive, active, pk), but DRF CursorPagination re-orders the queryset with its own ordering, so the v4 admin refactor's plain pk default (545fc2aa9) silently replaced the registration-stamp order with row-creation order. LibrarianStatusCursorPagination orders the active poll by (preactive, active, pk) — matching start_many's padded registration stamps, with NULL preactive sorting first for rows started without pre-registration (the cover thread case). The Jobs tab history view keeps the pk default. The active feed is ~20 rows against a 200-row page, so the cursor filter never engages the nullable fields. Regression test creates status rows with pk order deliberately opposite the registration stamps and asserts both routes. Co-Authored-By: Claude Fable 5 <[email protected]> commit 396312c6fe9964b36f895dd6a2c8bb84757191c3 Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:55:39 2026 -0700 import timing: don't double-count sub-steps in the phase share total _run_phases reuses the timed_step helper instead of an inline perf_counter copy, and _log_phase_times sums only top-level phases — dotted "phase.step" entries already accrue inside their parents, so including them inflated the total and shrank every phase's share. Co-Authored-By: Claude Fable 5 <[email protected]> commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:47:39 2026 -0700 metadata edit: full-size Protagonist label, live menu, stale-pick clear The Protagonist row inherited the old Main sub-row styling (smaller label, dimmed row) — it is a first-class row now, styled like its tag siblings. The choice menu already tracked the entered character/team names live (it is name-based, no pks involved); what was missing is that removing the picked name left the selection dangling — a watcher now clears it. A protagonist seeded from the DB but absent from the entered lists still survives load untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:28 2026 -0700 metadata: unify main_character/main_team as a single Protagonist field Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is replaced by one "Protagonist" row above the tag table showing the mainCharacter and/or mainTeam chips — both only in the should-never-happen case where both are set. Each chip carries its own browser filter key (characters vs teams) via a per-item filter override in MetadataTags. En route this fixes mapTag's sticky filter: the loop reassigned the filter param, so every row after the first reused the first row's filter key. Edit: the two Main Character / Main Team sub-selects collapse into one Protagonist select offering the entered characters and teams; the pick is sent as the comicbox "protagonist" patch field on write, and the importer resolves it back to main_character or main_team (nulling the other) on re-import. Co-Authored-By: Claude Fable 5 <[email protected]> commit 632a238590f188ee249c4a8164a6936bcdfc779c Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:16 2026 -0700 importer: stop clearing an unchanged protagonist on update imports The fk link phase set main_character/main_team to None whenever a comic's remaining LINK_FKS dict lacked a "protagonist" key. The query prune pops that key when the protagonist already matches the DB, so re-importing a comic with any *other* FK change silently NULLed its protagonist. Updates now only touch the protagonist fields when the metadata explicitly carried one (set-or-clear), matching the absent-means-untouched semantics of every other simple FK tag; creates keep both fields defaulted. Also: a name that is both a Character and a Team now links only main_character — both fields filled is invalid. Regression test drives query → prune → link with only scan_info changed and asserts the protagonist survives (it NULLed before). Co-Authored-By: Claude Fable 5 <[email protected]> commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead Author: AJ Slater <[email protected]> Date: Wed Jun 10 16:16:30 2026 -0700 import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s) TimestampUpdater filtered every collection model through a Count(DISTINCT comic) alias to exclude empty collections — a per-row GROUP BY join aggregate costing ~1.5s per import even with zero deletions, in every import scenario. The count was redundant for two of the three OR branches: a comic__updated_at join match implies the collection has children, and force-updated pks were exempted from the child filter anyway. Only the custom-cover branch can match an empty collection, so it now carries its own join-presence test (comic__isnull=False) instead. Also replaces the fetch-instances + CASE-WHEN bulk_update round trip with a batched .update(updated_at=Now()) over the distinct pks. Also corrects the stale IMPORTER_FILTER_BATCH_SIZE comment: the binding constraint on OR chains is SQLITE_LIMIT_EXPR_DEPTH (1000), not the old 999-variable cap (modern SQLite allows 32766), so 900 remains correct for the remaining chain users. Benchmark, 1000 mock comics: delete phase 1.46s -> 0.10s on fresh and below 0.1s on reimport (5.9s total). Re-stamp semantics covered by tests/importer/test_move_timestamps.py. Co-Authored-By: Claude Fable 5 <[email protected]> commit bac8ee8af974bec78781f5c764e543f8429540a1 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:51:29 2026 -0700 import perf: bulk_update only changed comic fields (update step 8x) update_comics passed all ~40 BULK_UPDATE_COMIC_FIELDS to bulk_update regardless of what changed — CASE-WHEN compilation and parse scale with rows x fields, so a force-reimport whose comics needed nothing but a stat refresh paid the full-width rewrite: 4.4s of the 7s remaining reimport time, almost all inside the two bulk_update manager calls. Collect the union of fields actually applied per batch (proposed md diffs + FK link fields) and update that union plus the presave-derived/timestamp set (stat, size, date, decade, age_rating_metron_index, updated_at). Fields outside BULK_UPDATE_COMIC_FIELDS are dropped as before. Benchmark, 1000 mock comics: forced reimport 10.0s -> 7.0s, update_comics sub-step 4.4s -> 0.55s. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55401e6d372da3a9bf1c8b1cedeecbfd6fa06a27 Author: AJ Slater <[email protected]> Date: Wed Jun 10 15:46:01 2026 -0700 import perf: narrow existing-row queries with a selector IN, not Q-OR chains Multi-key existing-row lookups (query phase query_existing_mds, link and create pk-map builders) built per-key AND/OR Q chains — one Django filter resolution per leaf and a giant OR for SQLite to parse, per 500-900-key batch. Profiling a fresh import showed ~40% of its time in this filter machinery. Both consumers already match exact key tuples in Python from the fetched values, so the chain only ever bounded the fetch. Replace it with a single indexed IN on the model's most selective non-null key rel (MODEL_SELECTOR_REL_MAP: imprint/series name, volume series name, credit person, identifier key, story-arc name, folder path). Superset rows — the same selector value under a different parent — are harmless extra map entries. Keys with a None selector value fall back to the old chain (selector columns are non-null, so normally none). Also: read phase sub-instrumented (extract vs aggregate) and the benchmark gained --profile (cProfile per scenario). Benchmark, 1000 mock comics: fresh 12.1s -> 9.4s (query phase 2.6s -> 1.0s, link 2.5s -> 1.7s); reimport 11.8s -> 10.0s (query 3.4s -> 1.6s). Tag/link counts byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 610c51b67580006567b168e0e260c6c6097433d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:11:28 2026 -0700 import perf: skip archive opens when the fs stat snapshot is unchanged The fs prefilter compared disk mtime against the stored *embedded* metadata mtime, which is older than the file's mtime for any archive copied into the library after tagging — so the common at-rest comic never skipped and every modified-event re-poll opened its archive (an unrar spawn per CBR) just for comicbox to no-op. Reuse the stat call to also compare disk (mtime, size) against the stored Comic.stat snapshot — the poller's own modified test — and skip the worker entirely on a match. Paths with no stored embedded mtime still always survive, so a library imported with the import-metadata flag off picks up tags after the flag turns on. Benchmark, 1000 mock comics: noop poll 1.0s -> 0.2s (7.6s at baseline); zero archive opens. Reimport unaffected (force path never populates the prefilter's mtime map). Co-Authored-By: Claude Fable 5 <[email protected]> commit 06bec468cd68b423efb8f2b9da01be0f250c9d34 Author: AJ Slater <[email protected]> Date: Wed Jun 10 14:06:59 2026 -0700 import perf: skip true no-op stat-only updates (noop poll 7.6s -> 1s) _envelope_deltas included metadata_mtime unconditionally, so every file whose tags comicbox skipped still routed through the stat-only update: the comic row rewritten, its covers purged and re-queued for generation, and the import counted as changed -> cache.clear() + LIBRARY_CHANGED broadcast. A modified-event burst over an unchanged library rewrote every row and regenerated every cover. Compare metadata_mtime against the stored value, and skip the stat-only routing entirely when the envelope is unchanged AND the on-disk (mtime, size) still matches Comic.stat — the same equality the poller's SnapshotDiff uses, so a skipped file is exactly one the poller would not re-emit (no stale-stat re-poll loop). Touched files (stat moved, content same) keep today's full stat-only path, which deliberately refreshes Comic.stat via presave. Benchmark, 1000 mock comics: noop poll 7.6s -> 1.0s with "No updates necessary" (the remaining ~0.9s is archive opens for the comicbox mtime check). Fresh and reimport unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 488af595b62d643f130098f38eb2d0d8760230cd Author: AJ Slater <[email protected]> Date: Wed Jun 10 13:40:39 2026 -0700 import perf: batch-resolve comic fk links (fresh import 3x) Comic creation resolved every FK link with objects.get() per (comic, field) — folder, two protagonist lookups and ~one per FK field, ~14 single-row SELECTs per created comic. create_comics was 59% of a fresh import. prepare_fk_link_instance_maps now runs once per chunk after the FK create/update steps: one batched query per referenced field over the distinct key tuples (reusing the link phase's pk-map builders), plus in_bulk for instances. get_comic_fk_links stitches from the maps. Instances, not pks, so Comic(**md), the update path's setattr/default handling and presave's age_rating dereference are unchanged. Missing rows still raise DoesNotExist and skip the comic. The parent-folder map is also library-scoped now; the old Folder.objects.get(path=...) could MultipleObjectsReturned when two libraries shared a directory path. Also fixes a latent crash exposed by the new path: _build_pk_map_multi_key sorted key tuples that mix None with values (volume year/number_to, story-arc numbers) — TypeError when two keys first differ at a None position. Sort key now orders None last. Benchmark, 1000 mock comics: fresh 35.1s -> 12.1s; create_comics sub-step 21.7s -> 0.25s (map building 0.5s). Cumulative since baseline: fresh 2.9x, forced reimport 131.6s -> 11.8s (11x). Sub-step timing for create_and_update phases included (timed_step helper; dotted names excluded from share totals). Co-Authored-By: Claude Fable 5 <[email protected]> commit 94dd32a4ab9a74e2fd250aea22340c41ff35b5e5 Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:59:44 2026 -0700 import perf: compare fk prune links as values tuples (query 12s -> 3.7s) The fk prune select_related'd only the directly referenced FK fields, then walked each key rel via getattr chains — every hop past depth one (volume -> series -> imprint -> publisher) lazy-loaded a single-row SELECT per comic, and the protagonist check lazy-loaded main_character and main_team likewise. Query every referenced field's key rels in one batched values_list and compare row slices against the proposed key tuples. Also makes the protagonist prunable when it's the only referenced field (it was skipped before because it isn't a concrete FK for select_related). Benchmark, 1000 mock comics: reimport 23.9s -> 13.2s, query phase 12.2s -> 3.7s. With the m2m prune rewrite, forced reimport is 10x faster than the 131.6s baseline. Fresh and noop unchanged. Co-Authored-By: Claude Fable 5 <[email protected]> commit 55ceae11efd87c421f59b616ec65bf64882d4a8e Author: AJ Slater <[email protected]> Date: Wed Jun 10 08:53:01 2026 -0700 import perf: scan m2m prune off the through tables (10x) The m2m prune walked prefetched related objects per comic and read key attrs through instance getattr chains — for credits, identifiers and story-arc-numbers each FK in the key tuple lazy-loaded with a single-row SELECT per through row. A forced reimport of 1000 comics spent 121s (92%) in this phase. Scan the through tables directly instead: one values_list query per (field, batch) with the key attrs joined in SQL, pruned via set ops against the proposed links. No comic or related-model instances, no lazy FK loads. get_through_model moves from the link phase to const so the query phase can share it. Same pruning semantics (kept/deleted/FTS-existing/moved…
commit 15a6b8c58bd31e2d3ce558db9a35c3bf3d4be623 Author: AJ Slater <[email protected]> Date: Mon Jun 15 21:15:28 2026 -0700 version 2.0.0a14 commit 3ed935b45d978fe81118996285111a57fc340067 Author: AJ Slater <[email protected]> Date: Mon Jun 15 21:14:52 2026 -0700 update deps commit 4201d0747a3f43ee7aa5298ce229c07ab5b02eac Author: AJ Slater <[email protected]> Date: Mon Jun 15 20:03:44 2026 -0700 fix(lazy-import): stop the tag-icon hover loop on no-metadata comics Hovering a browser card's tag icon triggers a lazy metadata import gated on whether the comic "has metadata". Both the frontend hover gate and the lazy-import worker keyed off Comic.metadata_mtime — but that field is comicbox's *embedded* metadata mtime, which stays NULL forever for an archive with no metadata file. So a no-metadata comic always reported hasMetadata=false; after each import the browser refreshed, the card remounted, the cursor was still over the icon, and the import re-fired endlessly. Add a dedicated Comic.metadata_imported_at marker, stamped by a single path-scoped bulk UPDATE at import finish for every comic a forced/lazy pass touched (covers the SKIPPED no-metadata comics that never reach the per-comic write path), gated on force_import_metadata so ordinary watcher imports don't rewrite it. The worker gate and the has_metadata annotations (browser + reader) now read the marker, with metadata_mtime kept as a fallback. The field + backfill (COALESCE(metadata_mtime, updated_at), so existing comics don't regress to un-imported) is folded into the consolidated v2 migration 0043. metadata_mtime is left untouched — it is fed back to comicbox as the skip-re-extraction baseline, so overloading it would corrupt that protocol. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit a665bd183640a9abe33c6b083a784c3dcdfa4533 Author: AJ Slater <[email protected]> Date: Mon Jun 15 19:45:54 2026 -0700 rename batch to session commit 00f29ab33681432f6d3bb25e4bea46a5328dbd03 Author: AJ Slater <[email protected]> Date: Mon Jun 15 19:44:28 2026 -0700 feat(onlinetag): refine match review dialog for pause/resume workflow - Expand the first match panel by default so the admin can act at once. - Replace the dead "Abort Session" button (abortSession never existed) with a "Pause" button wired to pauseSession, matching the Tagging tab. - Show "Cancel" instead of "Dismiss" while the session is unfinished (active or resumable), derived from the store snapshot. - Add a tooltip to the sidebar "Matches to Review" item. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c1beab7ecbef3a5ce6fa765551d06141e6b03d39 Author: AJ Slater <[email protected]> Date: Mon Jun 15 02:06:47 2026 -0700 feat(onlinetag): default delete_original to True Flip the ComicboxTaggingDefaults.delete_original field default and the 0043 seed so newly tagged comics replace their originals by default. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 910fa88989256d8ef6f01b2a501c9e4f81e25cfa Author: AJ Slater <[email protected]> Date: Mon Jun 15 02:00:01 2026 -0700 fix(tagging): scroll long comic names instead of truncating The Online Tagging Status table truncated long comic names with an ellipsis, putting them out of reach. Replace the path cell's overflow/text-overflow ellipsis with overflow-x: auto so a long name scrolls horizontally within its existing width. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 05d44ba3b0f808cdb8a780da05f147333a486625 Author: AJ Slater <[email protected]> Date: Mon Jun 15 01:59:23 2026 -0700 fix(onlinetag): enable tag-by-id from reader metadata screen The reader builds its metadata book from books.current, which carries a pk but no ids array. The online-tag dialog's idTaggable gate requires book.ids.length === 1, so the tag-by-id tab never appeared for a single comic opened from the reader, while browser card items (which include an ids array) worked. Derive ids from the pk to match the card shape. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8afde179d0dbc5a9e35d3523afb88e1c4daf9ad0 Author: AJ Slater <[email protected]> Date: Mon Jun 15 01:45:28 2026 -0700 Squashed commit of the following: commit a6c315e224b2b6170d437099736afcbbf97e5479 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:31:51 2026 -0700 update deps commit 98f55c7bf96be3d4c56c3fb12b10e95270a39cbc Merge: 5ad35f1cd 1274bad66 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:31:20 2026 -0700 Merge branch 'develop' into pre-release commit 5ad35f1cd4d7b40935e4ca19c018f312bdef887b Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:35 2026 -0700 Squashed commit of the following: commit 5e6f4dfc55ee4ad3f8fab48b7b33ffe02f17d8a4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:26:21 2026 -0700 update version 2.0.0a13 commit 59d607d6c886cccb093fbc07dea5a2ef97ab1dc0 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:25:51 2026 -0700 test(onlinetag): exercise replay path in unmatched-write test The candidate carried issue_id 123, routing the resolve through the explicit-id fetch (real fetch_tags_by_explicit_id) instead of the replay session the test sets up and claims to verify. That crashed on the empty touched .cbz before any assertion ran. Drop the issue_id so the candidate falls back to the replay path guarded by _first_tags. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit dfa2349e8b821fbaa93b53fb4639bd2d66db8522 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:19:14 2026 -0700 fix(mail): ASCII hyphen in noqa comments to dodge complexipy panic complexipy's extract_comment_marker slices noqa comment text by a fixed byte offset (16); a multi-byte em-dash straddling that offset lands mid-char and panics the Rust core, aborting the whole run. Swap the two em-dashes in the noqa comments for ASCII hyphens. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 1799ce937559d62a667a0bf66f819f1593c93c5e Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:16:34 2026 -0700 decomplexify test for radon commit 6213c0afe641942e9e1d28f090970c6da524ceb4 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:14:19 2026 -0700 remove unused ty ignore commit dc59f1dc9560e177f9e0c36ea4552208014746d6 Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:13:38 2026 -0700 remove redundant definition commit 5d7a204d5069783c8638bc8f76f84bf15840239a Author: AJ Slater <[email protected]> Date: Sun Jun 14 23:10:36 2026 -0700 Squashed commit of the following: commit 1274bad663bf382d76453e65a6fc935b8b3d4559 Merge: a8aca7b33 72fad6861 Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:57:33 2026 -0700 Merge branch 'main' into develop commit 72fad6861b0af14731847b5444f097fdec699fda Author: AJ Slater <[email protected]> Date: Sun Jun 14 22:06:27 2026 -0700 v1.12.8 (#781) commit a8aca7b3341e7afdee8e99829be88d77e4d988c5 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:50:21 2026 -0700 fix(test): call super().tearDownClass() in BaseTestImporter Skipping super() left the class-level atomic from setUpTestData uncommitted-but-unrolled-back, leaking the codex_init() "admin" superuser into later test classes and causing UNIQUE constraint failures (e.g. test_bookmark_filter_isolation) in full-suite runs. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit d7570ecf99e1a15449dec2f7a0550122b9d405df Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:16:59 2026 -0700 remove gitignore commit be211c5724d5d7cc25ca367813fa34244097f372 Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:12:03 2026 -0700 fix(types): narrow OPDS redirect settings to Mapping for ty Replace a stale `ty: ignore[no-matching-overload]` with an isinstance guard so `settings` is a Mapping before `_copy_params_into`, clearing the `invalid-argument-type` diagnostic from `make ty`. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5dd3a0dd9feb18aa4b13c15b000c048befdeceef Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:09:05 2026 -0700 update devenv commit e5a2a41881d35e2d8716bd6dfefdd64fa3792b5c Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:07:34 2026 -0700 shfmt commit 1f867c1497e4edc14315d63c7eeb8c3e2b0f07bd Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:05:38 2026 -0700 shfmt commit c5366787b56212e12a69c5aaab1776474962a31b Author: AJ Slater <[email protected]> Date: Sun Jun 14 21:02:56 2026 -0700 fix(bookmark): scope read/unread filters per user (v1.12.8) The UNREAD filter used Q(bookmark=None), a multi-valued relationship test for "comic has no bookmark from anyone" rather than "no bookmark for me". Once any user finished a comic it dropped out of every other user's (and anonymous visitors') unread view. UNREAD now negates the per-user finished predicate (~(my_filter & finished=True)), compiling to a per-user NOT-EXISTS subquery. Also harden get_my_bookmark_filter: an anonymous visitor with no established session_key resolved to `session_id IS NULL`, which matches every authenticated user's bookmarks. Return a match-nothing filter instead; a session key is minted only when a bookmark is written. Adds tests/test_bookmark_filter_isolation.py covering cross-user and anonymous isolation for READ/UNREAD/IN_PROGRESS. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 541907b6708dfbbde197432960c6adc6dcf44b02 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:44:58 2026 -0700 fix(frontend): convert v-dialog em widths to px so dialogs aren't slivers Vuetify 4's v-dialog defaults to location 'center center', which activates the connected overlay location strategy. That strategy caps width with parseFloat(props.maxWidth), so "20em" parses to 20 and the em is dropped, collapsing dialogs to a ~20px sliver. Convert every em-based max-width/ min-width on a v-dialog (or forwarded via $attrs) to unitless px at 16px/em. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 5918d0b621cd635f3de32f142d4dff7ec9c45242 Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:41:25 2026 -0700 move VPullToRefersh out of labs commit 57d5cf3888c2dc0aab55e46447b41a459d37003a Author: AJ Slater <[email protected]> Date: Sun Jun 14 19:32:39 2026 -0700 update devenv and deps commit cd944f0e4cf7044ba8e72cb6f90539ca892adb72 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:25:16 2026 -0700 Squashed commit of the following: commit b5874b67774af29f43603f01690a0ca97b43c46a Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:24:54 2026 -0700 update deps commit e0a16dff8e66c357b2d2aa5e4ff3ea7decce4572 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:23:51 2026 -0700 remove pytest-gitignore commit 35b84324300352499f3c455aa43201c3f475cd21 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:20:46 2026 -0700 update version to 2.0.0a12 commit e3e1167851960a42fef9a5ceba3b3b5470f7a56c Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:19:54 2026 -0700 Fix basedpyright error on Folder FK access in moved-parents test The Folder model annotates parent_folder as ForeignKey explicitly, so django-stubs doesn't synthesize the implicit parent_folder_id attribute. Compare the FK object instead (Django compares models by pk). Pre-existing since ce1fd81ad; behavior unchanged. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 0ee9c966f9d1dcfd84d36e6077778fd9354d9574 Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:45 2026 -0700 update deps commit 38f527b57ce30be0f162d7aae2bf51d938e3141b Author: AJ Slater <[email protected]> Date: Sun Jun 14 18:13:39 2026 -0700 ignore benchmark dir commit 7007c109accb1654c5b61580336d71cedfb5ce3f Author: AJ Slater <[email protected]> Date: Sun Jun 14 17:56:39 2026 -0700 Add online tagging status table with live progress and session controls Admin Tagging tab now shows a live status table for the online tagging batch: per-comic state, per-source rate-limit countdowns, batch ETA, and match-review status. Backed by a daemon-published session snapshot in the tagging cache, exposed via GET /api/v4/admin/tag-sessions/snapshot and refreshed over the existing task.progress websocket. - Snapshot: session_snapshot.py builds a capped, JSON-safe per-comic snapshot from OnlineTagOutcomeStats + pending prompts; published throttled from the runner, frozen inactive on finish, cleared at startup / janitor. - Session lifecycle: pause / resume / abort / dismiss controls in the table, wired to the online-tag store and admin endpoints. - Match-review fixes: - direct-id fetch for explicit picks (no rate-limit-prone re-search drift) - record user_matched / user_skipped outcomes, overlaid on the snapshot - client guard against resurrecting just-answered prompts in the dialog - optimistic local overlay so resolutions show during rate-limit stalls Tests: backend snapshot / resume / session-manager / janitor suites; frontend store + status-table specs. ruff/ty/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit ce1fd81adb4823ac454950ab66905a38e3f6d9e1 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:56:39 2026 -0700 Fix ScribeThread crash on folder move under untracked parent _get_move_create_folders_one_layer returned bare path strings, but bulk_folders_create expects key tuples (MODEL_REL_MAP[Folder] == (("path",), ...)) and iterates each entry to extract the path. Bare strings were iterated character by character, producing single-char paths like PosixPath('K') whose stat() raised FileNotFoundError and crashed the ScribeThread during import. Wrap the path in a tuple to match the contract, and add a regression test covering a folder move whose destination parent isn't yet a tracked Folder. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 265988a4b44ba7ebe9c810bedc11a42796853d90 Author: AJ Slater <[email protected]> Date: Sat Jun 13 11:30:44 2026 -0700 Delete duplicate migration 0044 that broke test-DB setup The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e6), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit c31c61e6175d24d9e1dde1835c7235c367c251c9 Author: AJ Slater <[email protected]> Date: Sat Jun 13 02:00:11 2026 -0700 fix some kind of merge issue commit 92e526940e96b6ea9679363159395270233e75f2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:55 2026 -0700 Squashed commit of the following: commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:54:29 2026 -0700 refactor(complexity): clear radon cc rank-C findings bin/lint-complexity.sh reported three rank-C functions; extract helpers to drop each below the gate (radon cc --min C now empty). - AdminTagByIdView.post: pull primary-identifier parsing and merge extra-id resolution into _resolve_primary / _resolve_extra_ids. - OnlineTagSessionManager._apply_resolution: extract _handle_unresolved and _enqueue_resolved_write. - test_update_protagonist: extract repeated protagonist assertions into a helper (radon counts each assert as a branch). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 55bb659fcc8c02e73c20a34823652a4f81be1d56 Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:46:04 2026 -0700 refactor(migrations): squash 0044 into 0043 Fold the merge_all_sources field into the ComicboxTaggingDefaults CreateModel in 0043 and drop the standalone 0044. makemigrations --check reports no drift. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 7a988737a63286cc85396346579e124e5fff8e5b Author: AJ Slater <[email protected]> Date: Sat Jun 13 01:44:59 2026 -0700 feat(onlinetag): merge metadata from all sources Add an opt-in "merge all sources" mode that queries every enabled online source per comic and merges the results (comicbox first_wins=False) for maximum tag completeness, instead of stopping at the first match. - Admin tagging defaults: merge_all_sources boolean (default off), settable as the default and overridable per scan. - Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate + rate-limit warning multiply calls by source count (kept in sync with estimate.py) so the doubled API cost is visible. - By-ID: chips input accepting multiple URLs/ids; merging fetches two explicit ids (one per source) and merges them. Source select shows only when a token can't be auto-identified (bare number). Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:34:56 2026 -0700 update deps commit 473221d7a09403b1848f202d55902b71e31319c2 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:22:58 2026 -0700 fix(lint): clear ruff/pyright findings; fix benchmark import path - tests: add docstrings, name magic estimate values, use pytest.raises and out-of-class exception message - onlinetag: collapse implicit f-string concatenation in log call - benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*) which was a real runtime ImportError in library generation Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 45a9fde6076a5b0b15be889769e9a55af4b0678a Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:13:49 2026 -0700 refactor(migrations): squash 0044 + 0045 into 0043 Fold the comic-leading composite Bookmark indexes (former 0044) and the LibrarianStatus eta/retry_at fields + JFR status-type choice (former 0045) into the consolidated 0043 migration; delete both files. makemigrations --check reports no changes against the model state. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit 22da6b9cd85615f3d7b8e7043818babac313d24e Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:35 2026 -0700 update deps commit 6f27454623e43cccd95253d5dfe66471820c9349 Author: AJ Slater <[email protected]> Date: Sat Jun 13 00:04:15 2026 -0700 fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns Two online-tagging bugs plus a status-UX overhaul. Prompt answers given while a scan was running were lost on refresh: the response task sat in the queue behind the busy OnlineTagThread, so the cache kept the prompt. Now the drain loop removes answered prompts from the cache inline (race-free, single-threaded) and marks the fingerprint so the running scan stops re-persisting it; the network re-fetch + write is deferred until the scan releases the thread. A rate-limited lookup looked hung: collect_results could raise mid-pass (budget exhausted, network error) without ever calling finish(), leaving the status row frozen forever. finish() now runs in a finally. The rate-limit status was a static "rate limited ~57s" — a second subtitle writer clobbered the parseable one, and nothing refreshed it during comicbox's blocking sleep. Replaced with two absolute-timestamp countdowns the admin UI ticks down to live: - retry_at: time until the next online request, re-anchored per attempt. - eta: total time remaining, carrying forward the launcher dialog's estimate (ported verbatim to estimate.py), re-estimated on each comic completion and pushed out by rate-limit waits. Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in the pending status_type choices drift), plumbed through Status and StatusController. Co-Authored-By: Claude Opus 4.8 <[email protected]> commit eccf41bc603b118bdbd63f4e4918a30da87140ae Author: AJ Slater <[email protected]> Date: Fri Jun 12 22:39:15 2026 -0700 feat(tagging): admin-orderable online source priority defaultSources order is now run priority end to end. comicbox runs sources in the given order under first-wins, so the admin can choose which source is primary and which is fallback: - Tagging tab renders the source rows in defaultSources order with per-row up/down arrows; enabling a source appends it at lowest priority. A hint explains that the first matching source wins. - The launcher dialog normalizes its checkbox selection to the admin's priority order before starting a session (checkbox v-model records click order, not priority). - Serializers validate defaultSources / sources: known names only, deduped, order preserved (unknown source → 400). - explicit_id / session settings pass ordered tuples to comicbox's retyped OnlineLookupSettings.sources. Co-Authored-By: Claude Fable 5 <[email protected]> commit 02869ddf4d027c325637dd41081199c59f1141ad Author: AJ Slater <[email protected]> Date: Fri Jun 12 17:06:25 2026 -0700 add METRON_INFO to default formats to write commit a214c9c93cf679194beeb5c42eb66deea933a3da Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:36 2026 -0700 update deps commit 65a294362171bded71547430dd51b322ea83dcef Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:58:08 2026 -0700 don't make abort button red commit de49b5dd9529446ab34a7468bda694fae39c7a00 Author: AJ Slater <[email protected]> Date: Fri Jun 12 14:52:14 2026 -0700 fix(frontend): remove duplicated Tag Write Errors sidebar item admin-menu.vue contained the same tag-write-errors CodexListItem twice (re-added in a later squash while the original stayed), so any error rendered two identical sidebar entries. One item per error kind; the counts live in the admin tables. Co-Authored-By: Claude Fable 5 <[email protected]> commit e0bd853d9ed1991859849e11d85d487409dc1194 Author: AJ Slater <[email protected]> Date: Fri Jun 12 13:43:19 2026 -0700 fix(onlinetag): stop spurious rewrites and prompt loss in online tagging Three interacting bugs broke a single-comic tagging run: - Pass-1 and prompt-resolution writes now require result.matched (new comicbox flag): unmatched results carry the comic's merged existing metadata, so gating on truthy tags re-wrote files with no new information — which also triggered a re-import mid-session. - Pending prompts and tag-write errors move to a dedicated 'tagging' cache (codex.cache.tagging_cache). The session_cache docstring claimed key namespacing protected them from cache.clear(), but Django's file-based clear deletes everything: every import that changed anything (importer finish), Library/Group CRUD, and startup wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'. - Prompt resolution builds its replay session with defer_prompts=True so the preloaded resolution is actually consulted (comicbox only installs its cache-reading selector with defer or a handler; without it an ambiguous re-search fell through to comicbox's interactive CLI prompt inside the daemon). A fingerprint miss — the re-search returned different candidates — now re-queues the fresh deferred prompt instead of dying silently. Co-Authored-By: Claude Fable 5 <[email protected]> commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:45:22 2026 -0700 feat(admin): auto-enable a tagging source when its credentials are first saved Saving credentials for a source that previously had none now adds it to default_sources via the auto-saving draft, so a freshly configured source works without a second checkbox click. Re-saving credentials for an already-configured source leaves the enable checkbox untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853 Author: AJ Slater <[email protected]> Date: Fri Jun 12 11:38:11 2026 -0700 Squashed commit of the following: commit 9543008f24441b6b2992f0c818d019f42135f7df Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:21:24 2026 -0700 dockerignore benchmark dir commit 00134a40fe1c37d96e64066735cb42cc03f463c8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:20:24 2026 -0700 mv fixutres to benchmark dir commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:54 2026 -0700 change import path for mock comics commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:13:05 2026 -0700 ignore fixtures dir commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:11:25 2026 -0700 common subdir for mock & benchmark commit cd768965a544069fb623f9798a2ef7f472016586 Author: AJ Slater <[email protected]> Date: Fri Jun 12 10:01:01 2026 -0700 move benchmark import out of ci commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:52:32 2026 -0700 v2.0.0a10 commit 43b6042bca9c546d49268a52002efa4db69651c4 Author: AJ Slater <[email protected]> Date: Fri Jun 12 09:51:47 2026 -0700 update deps commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:39:57 2026 -0700 fix typecheck and ty warnings Merge implicit f-string concatenations, add missing @override on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555 Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:28:58 2026 -0700 remove bad news line commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac Author: AJ Slater <[email protected]> Date: Fri Jun 12 02:18:54 2026 -0700 adapt to comicbox 4.0.0a3 API Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <[email protected]> commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251 Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:55:25 2026 -0700 update deps commit 919fad63cf5790b82275e5f20207e38f65934a6c Author: AJ Slater <[email protected]> Date: Thu Jun 11 21:48:25 2026 -0700 update deps commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e Author: AJ Slater <[email protected]> Date: Thu Jun 11 14:00:48 2026 -0700 poller: apply discarded .only() projection in manual poll path _handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <[email protected]> commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 07:52:01 2026 -0700 admin: fix stale Last Poll column after websocket table refetch DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <[email protected]> commit 188dd22775d6beb284b6ddafb3629dcb0affbccb Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:18:16 2026 -0700 edit news for brevity commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:17:05 2026 -0700 NEWS: browser performance entry for 2.0.0 Co-Authored-By: Claude Fable 5 <[email protected]> commit 994d6a586c249376827c1d72494071b4281528fa Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:15:27 2026 -0700 news: v2.0.0 performance section for the import speedups Co-Authored-By: Claude Fable 5 <[email protected]> commit cbb07c980bd5bb317e7a506bec81ba8f66415163 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 cards: Max bookmark timestamp aggregate instead of JSON array + parse loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <[email protected]> commit 01a9394d8ecaaa22e72885b4257545a82220b3d4 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 breadcrumbs: batch the folder ancestor chain in one query The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <[email protected]> commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733 Author: AJ Slater <[email protected]> Date: Thu Jun 11 00:06:03 2026 -0700 mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <[email protected]> commit 578807ab0294391095e2f6b6fe3d1b1540f643a4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:55:07 2026 -0700 search: materialize the FTS match set once per browse request The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <[email protected]> commit e56b7eb2dd54cab402041de418832f865269540c Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 lint: format follow-ups for the order/cover perf commits Co-Authored-By: Claude Fable 5 <[email protected]> commit ca67645e4bf757568021e19781d9179ae599bda2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 23:46:46 2026 -0700 browser: move the last-route write off the read path; settings row fetched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <[email protected]> commit 94ef67e41554869cfe4c19494cea7e422ff5d30b Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:10:00 2026 -0700 covers: batch cover mtimes post-pagination instead of a second subquery cover_mtime was a .values('updated_at') Subquery over the identical correlated cover queryset that already computes cover_pk — two full executions of the most expensive expression per card (measured 40-49% of the card query across publisher/folder/search pages). Drop the annotation; resolve the representative comics' updated_at after pagination with one indexed pk__in batch over the page's cover_pks and attach it to the cached instances (the serializer iterates the same objects). Custom-cover mtimes keep their cheap rowid subquery and still win the coalesce. _CoverMtimeCoalesce is gone with the annotation that needed it. Responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 browser: zero-pad from page rows; skip dead card serialization in table mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <[email protected]> commit d1789eda2ac5a4351c3039ee42d4677db1c669c6 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 table mode: evaluate the collection page query once, not twice compute_collection_intersections read the page pks via values_list('pk'), a clone that re-executed the entire annotated/ grouped/sorted collection query; the serializer then ran the original queryset again. Iterating the queryset itself primes its result cache, which the serializer reuses — the heaviest table-mode query now runs once per request. Co-Authored-By: Claude Fable 5 <[email protected]> commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28 Author: AJ Slater <[email protected]> Date: Wed Jun 10 21:06:50 2026 -0700 comic listings: stop aggregating order aliases; GROUP BY pk only Two GROUP BY-width fixes for Comic-model querysets: - _alias_filename aggregated Min/Max over the comic's own single path, which turned the alias into an aggregate and dragged every selected column into GROUP BY inside the listing and both cover subqueries. A comic is its own file; use the bare expression (as annotate_comic_extra_specials already did). A filename-ordered folder browse measured 135s before, 0.6s after, identical rows. - The ids JsonGroupArray made Django compute GROUP BY over all ~50 selected Comic columns including TEXT blobs. Force GROUP BY id — every other column is functionally dependent on the pk, so groups are identical and SQLite sorts a one-column key (25-34% off the books query). Not applied to the cover path, where the forced literal column doesn't survive nested-subquery aliasing. Co-Authored-By: Claude Fable 5 <[email protected]> commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 browser: UNREAD filter must not hide comics other users finished The UNREAD arm was Q(bookmark=None) | (mine & unfinished): bookmark=None matches only comics with no bookmark from ANY user or session, so a comic finished by someone else (or a stale anonymous session) matched neither arm and silently vanished from my unread listing (reproduced: 50 foreign finished bookmarks shrank my unread count by 50). UNREAD now probes 'I have no finished bookmark on this comic' with a correlated NOT EXISTS scoped to the requesting user/session. It correlates on the queryset's comic pk, so on collection querysets it binds to the same joined comic row as the other filters in the combined .filter() call — group rows still require ONE comic that satisfies every condition (pinned by test). New comic-leading composite indexes (comic,user)/(comic,session) keep the probe off the user index: under stale sqlite_stat1 the planner flipped a user-scoped probe onto it — a measured 17s vs 27ms plan. EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user (comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged; single-user responses verified byte-identical. Co-Authored-By: Claude Fable 5 <[email protected]> commit cad52d79e8242e3b69e4cbd604757f2c7faab728 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:56:40 2026 -0700 metadata: stop GET writing m2m through tables and corrupting tags _copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True) for every real m2m field on the Comic path — obj is a live saved instance (the 'values dicts' comment was stale), so a GET /metadata DELETE+INSERTed up to 12 through tables, and a multi-comic selection permanently rewrote the first comic's tags to the selection's intersection (reproduced: 339 characters silently became 43). Serve the intersection queryset through the instance's prefetch cache instead: RelatedManager.get_queryset consults it keyed by field name, so the serializer reads the already-optimized queryset query-free. Response verified byte-identical; the 38-query identifier N+1 (the post-write re-read discarded select_related) disappears with it. Co-Authored-By: Claude Fable 5 <[email protected]> commit 71b52749a3b24a8493c6803f0c3cda04466280d4 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 browse: bind favorite collection code as str so cachalot can cache Binding the Collection StrEnum member itself made cachalot's parameter-type check raise UncachableQuery for every query the favorite subquery landed in — silently uncaching all browse queries. Repeat table-mode browses re-paid ~10s of SQL every time; with .value they serve from cache (16-22ms, 2-3 queries). The bound SQL value is identical either way. Co-Authored-By: Claude Fable 5 <[email protected]> commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:19:11 2026 -0700 metadata: pk subquery instead of inlined IN lists; count the filtered relation Two fixes to the intersection queries behind GET /metadata: 1. _get_comic_pks returned a materialized frozenset, inlining one bound SQL variable per comic per union arm. With 12 m2m arms, any collection over ~2,730 comics exceeded SQLite's 32,766-variable limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and 17.8k comics). Passing the distinct .values(pk) queryset keeps the statement constant-size (36 params); the same union measured 101ms at 8,576 comics. num_comics is computed once via COUNT and threaded to both consumers. 2. _get_fk_intersection_query counted the hardcoded 'comic' reverse m2m even when the filter traversed the main_*_in_comics FK back-relations — a semantically wrong HAVING over the unfiltered relation that also forced a separate unindexed join (main_team measured 2,069ms -> 12ms counting the filtered relation). Co-Authored-By: Claude Fable 5 <[email protected]> commit 5beb474835c7375f4b726eca63aa05f2ef8b9966 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 arc browse: demote story-arc chain joins to INNER for StoryArc queries With only codex_comic demoted, SQLite planned arc search browses from the arc side and built a cartesian product against the FTS match set: arcs root with a common search term measured 156s (main query 150.8s). Demoting codex_storyarcnumber and the M2M through table lets the plan start from the FTS/comic side (36ms). Scoped to StoryArc-model querysets: there the chain IS the collection relation, so NULL-extended rows are exactly the empty collections force_inner_joins exists to drop. On other models these tables appear as table-view annotation LEFT JOINs, where demotion would drop rows whose comics have no arcs. Co-Authored-By: Claude Fable 5 <[email protected]> commit 1434a5a771fc7445472cb52726f4e86c645df999 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:54 2026 -0700 cover subqueries: join FTS rank only when ordering by search_score The direct fts_q join exists solely to populate codex_comicfts.rank for the cover subquery's search_score ORDER BY, but it was applied on every search regardless of order key — and the MATCH re-executed once per correlated cover evaluation. The pre-materialized fts_sq membership test already filters correctly. A name-sorted search browse measured 135s before, 6.5s after (byte-identical rows); rank-ordered searches keep the join and pick the same top-ranked cover as before. Co-Authored-By: Claude Fable 5 <[email protected]> commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192 Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:18:40 2026 -0700 table mode: keep intersection-sort subqueries out of GROUP BY RawSQL.get_group_by_cols() returns [self], so the correlated intersection-sort subqueries landed in the GROUP BY of aggregated collection queries. Folder is the one collection model without a forced-GROUP-BY entry, so SQLite evaluated the subquery once per pre-aggregation joined row — at folders root the ancestor M2M fans out to every descendant comic (366s request; each of the two queries measured ~256s alone, 85ms after). Same mechanism as the _CoverMtimeCoalesce fix, recurring through a different expression type. _IntersectionSortRawSQL overrides get_group_by_cols() to return []: the subquery's only outer reference is the collection pk, which is always grouped, so grouping is unchanged (verified row-identical). Co-Authored-By: Claude Fable 5 <[email protected]> commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c Author: AJ Slater <[email protected]> Date: Wed Jun 10 20:14:18 2026 -0700 fix librarian status feed order: cursor pagination discarded the order_by The sidebar progress feed showed import phases scrambled — read next to the search statii, query below create. The viewset ordered active rows by (preactive, active, pk), but DRF CursorPagination re-orders the queryset with its own ordering, so the v4 admin refactor's plain pk default (545fc2aa9) silently replaced the registration-stamp order with row-creation order. LibrarianStatusCursorPagination orders the active poll by (preactive, active, pk) — matching start_many's padded registration stamps, with NULL preactive sorting first for rows started without pre-registration (the cover thread case). The Jobs tab history view keeps the pk default. The active feed is ~20 rows against a 200-row page, so the cursor filter never engages the nullable fields. Regression test creates status rows with pk order deliberately opposite the registration stamps and asserts both routes. Co-Authored-By: Claude Fable 5 <[email protected]> commit 396312c6fe9964b36f895dd6a2c8bb84757191c3 Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:55:39 2026 -0700 import timing: don't double-count sub-steps in the phase share total _run_phases reuses the timed_step helper instead of an inline perf_counter copy, and _log_phase_times sums only top-level phases — dotted "phase.step" entries already accrue inside their parents, so including them inflated the total and shrank every phase's share. Co-Authored-By: Claude Fable 5 <[email protected]> commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:47:39 2026 -0700 metadata edit: full-size Protagonist label, live menu, stale-pick clear The Protagonist row inherited the old Main sub-row styling (smaller label, dimmed row) — it is a first-class row now, styled like its tag siblings. The choice menu already tracked the entered character/team names live (it is name-based, no pks involved); what was missing is that removing the picked name left the selection dangling — a watcher now clears it. A protagonist seeded from the DB but absent from the entered lists still survives load untouched. Co-Authored-By: Claude Fable 5 <[email protected]> commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:28 2026 -0700 metadata: unify main_character/main_team as a single Protagonist field Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is replaced by one "Protagonist" row above the tag table showing the mainCharacter and/or mainTeam chips — both only in the should-never-happen case where both are set. Each chip carries its own browser filter key (characters vs teams) via a per-item filter override in MetadataTags. En route this fixes mapTag's sticky filter: the loop reassigned the filter param, so every row after the first reused the first row's filter key. Edit: the two Main Character / Main Team sub-selects collapse into one Protagonist select offering the entered characters and teams; the pick is sent as the comicbox "protagonist" patch field on write, and the importer resolves it back to main_character or main_team (nulling the other) on re-import. Co-Authored-By: Claude Fable 5 <[email protected]> commit 632a238590f188ee249c4a8164a6936bcdfc779c Author: AJ Slater <[email protected]> Date: Wed Jun 10 19:29:16 2026 -0700 importer: stop clearing an unchanged protagonist on update imports The fk link phase set main_character/main_team to None whenever a comic's remaining LINK_FKS dict lacked a "protagonist" key. The query prune pops that key when the protagonist already matches the DB, so re-importing a comic with any *other* FK change silently NULLed its protagonist. Updates now only touch the protagonist fields when the metadata explicitly carried one (set-or-clear), matching the absent-means-untouched semantics of every other simple FK tag; creates keep both fields defaulted. Also: a name that is both a Character and a Team now links only main_character — both fields filled is invalid. Regression test drives query → prune → link with only scan_info changed and asserts the protagonist survives (it NULLed before). Co-Authored-By: Claude Fable 5 <[email protected]> commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead Author: AJ Slater <[email protected]> Date: Wed Jun 10 16:16:30 2026 -0700 import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s) TimestampUpdater filtered every collection model through a Count(DISTINCT comic) alias to exclude empty collections — a per-row GROUP BY join aggregate costing ~1.5s per import even with zero deletions, in every import scenario. The count was redundant for two of the three OR branches: a comic__u…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.