fix(transcribe): open a fresh DB connection per job transition#189
Merged
Conversation
rt911-db sets idle_session_timeout=10min on the video_grabber database (leak protection added 2026-07-08), but whisper keeps transcribe-item busy for 15-20 minutes. The flow held one connection across the whole run, so it was dead by the final stage='done' UPDATE — and the failure path reused the same poisoned connection, raising PendingRollbackError and leaving the job stuck in 'transcribing' for the supervisor to reclaim and re-transcribe forever (186 failed runs since 2026-07-09). transition_transcribe_job now opens its own short-lived connection, so no connection ever idles across transcription. Also alias the CCTV3_ identifier prefix to the cctv4 slug — the test expecting it landed in 3be9c05 without the mapping, leaving CI on main red and blocking image builds. Co-Authored-By: Claude Fable 5 <[email protected]>
|
✅ Playwright E2E — 1 passed · 0 failed · 0 flaky · 0 skipped |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every
transcribe-itemrun since 2026-07-09 fails withPendingRollbackError(186 failed runs). Root cause chain:rt911-dbsetsidle_session_timeout=10minon thevideo_grabberdatabase (added during the 2026-07-08 connection-leak firefight).transcribe_item_flowopened one DB connection and held it idle for the 15–20 minutes whisper takes, so Postgres killed it before the finalstage='done'UPDATE (server closed the connection unexpectedly).exceptpath then reused the same invalidated connection →PendingRollbackError, so the job was never markeddoneorfailed— it stayed intranscribing, the supervisor reclaimed it, and the GPU re-transcribed the same programs in a loop (the SRT/VTT uploads actually succeed; only the bookkeeping write dies).Fix
transition_transcribe_jobnow opens its own fresh, short-lived connection per transition instead of taking a held one, so no connection ever idles across transcription. Theidle_session_timeoutleak backstop stays in place.Regression tests simulate the production failure: every connection opened before
transcribe_wavis killed mid-transcription, and the flow must still recorddone(andfailedon error) on fresh connections.Also: unblock red CI on main
test_identifier_prefix_fallback_for_international_feedhas been failing onmainsince 3be9c05 (CCTV3→CCTV4 rename) — the test landed but theCCTV3_identifier-prefix alias didn't, which also blocked image builds. Added"cctv3": "cctv4"toKNOWN_CHANNELS.Verification
pytest tests/— 303 passed, 8 skipped (only the documented live-Postgrestest_migrationserrors locally)ruff check video_grabber/ tests/— cleanNote:
process-itemand the usenet flows still hold a connection across long stages (a multi-GB IA download can idle >10min) — same latent bug class, follow-up candidate.🤖 Generated with Claude Code