Follow-up to #50 / PR #51, which moves the pyannote pipeline to CUDA.
_get_pipeline calls pipeline.to(torch.device("cuda")) with no guard. Combined with the project's intentionally-resident model caches, a CUDA run now pins two models in VRAM at once:
transcriber._MODEL_CACHE — the WhisperModel, resident between recordings by designdiarizer._PIPELINE_CACHE — now the pyannote pipeline, also resident
On a small card this can exhaust VRAM. large-v3 on CUDA is documented at ~10 GB in the README; adding pyannote's segmentation + embedding models on top will OOM on 8 GB and below.
Current behaviour on OOM: torch.cuda.OutOfMemoryError propagates out of DiarizationWorker.run, is caught by the generic handler, and surfaces as a diarization error. Per .claude/rules/transcription-pipeline.md the transcript is still rendered and persisted, so nothing is lost — but the user gets a raw CUDA OOM message and no speakers, with no path to recovery short of switching the whole app to CPU.
Proposed fix: catch OOM (and any exception) from the .to(cuda) move and from the pipeline call, fall back to a CPU pipeline for that run, and emit a progress note ("GPU out of memory — running diarization on CPU"). Do not cache a half-moved pipeline under the CUDA key.
Consider also whether the Whisper model should be evicted from VRAM before diarization starts, since the two stages never run concurrently (the job queue is serial). That would be a larger change and is worth measuring first — the reload cost is seconds, the OOM cost is a failed diarization.
Not urgent on the dev machine (RTX 4070 Ti, 12 GB, default model base/small). Matters for users on smaller cards running the large models.
Follow-up to #50 / PR #51, which moves the pyannote pipeline to CUDA.
_get_pipelinecallspipeline.to(torch.device("cuda"))with no guard. Combined with the project's intentionally-resident model caches, a CUDA run now pins two models in VRAM at once:transcriber._MODEL_CACHE— the WhisperModel, resident between recordings by designdiarizer._PIPELINE_CACHE— now the pyannote pipeline, also residentOn a small card this can exhaust VRAM.
large-v3on CUDA is documented at ~10 GB in the README; adding pyannote's segmentation + embedding models on top will OOM on 8 GB and below.Current behaviour on OOM:
torch.cuda.OutOfMemoryErrorpropagates out ofDiarizationWorker.run, is caught by the generic handler, and surfaces as a diarization error. Per.claude/rules/transcription-pipeline.mdthe transcript is still rendered and persisted, so nothing is lost — but the user gets a raw CUDA OOM message and no speakers, with no path to recovery short of switching the whole app to CPU.Proposed fix: catch OOM (and any exception) from the
.to(cuda)move and from the pipeline call, fall back to a CPU pipeline for that run, and emit a progress note ("GPU out of memory — running diarization on CPU"). Do not cache a half-moved pipeline under the CUDA key.Consider also whether the Whisper model should be evicted from VRAM before diarization starts, since the two stages never run concurrently (the job queue is serial). That would be a larger change and is worth measuring first — the reload cost is seconds, the OOM cost is a failed diarization.
Not urgent on the dev machine (RTX 4070 Ti, 12 GB, default model
base/small). Matters for users on smaller cards running the large models.