Skip to content

feat(transcription): GPU execution provider with CPU fallback (Track-B step 6)#14

Open
farce1 wants to merge 1 commit into
feat/asr-model-pickerfrom
feat/asr-gpu-provider
Open

feat(transcription): GPU execution provider with CPU fallback (Track-B step 6)#14
farce1 wants to merge 1 commit into
feat/asr-model-pickerfrom
feat/asr-gpu-provider

Conversation

@farce1

@farce1 farce1 commented Jun 14, 2026

Copy link
Copy Markdown
Owner

Track-B step 6 — GPU execution provider with CPU fallback

WhisperEngine/ParakeetEngine previously hardcoded the ONNX Runtime provider
to "cpu". This selects a platform-appropriate execution provider instead —
CoreML on macOS, CPU elsewhere — and retries on CPU if a non-CPU provider
fails to load
, so transcription can never regress relative to the CPU baseline.

Stacked on feat/asr-model-picker (#13): #9 ← #10 ← #11 ← #12 ← #13 ← this.

Design notes

  • sherpa-rs is built with only download-binaries (no cuda feature), so the prebuilt binaries are CPU-only off macOS; CoreML is the only GPU EP this build can use. CUDA selection would be dead code, so it's intentionally omitted (YAGNI) until a CUDA-enabled build is introduced.
  • select_provider(os) (pure) and with_cpu_fallback(provider, build) (the safety policy) are both unit-tested; load_with_provider carries the real recognizer config.
  • Runtime GPU speedup is not verified in this environment (needs the models + audio); the tested CPU-fallback guarantees no functional regression regardless.

Gate

BE: clippy clean, cargo build ✓, cargo test --lib 12 passed (the only 3 failures are the inherited maintainer-owned REPLACE_WITH_ SHA256 release gate). FE: unchanged on this branch.

Merges once main's vendor + REPLACE_WITH_ blockers are cleared and the chain rebases.

Pick the ONNX Runtime execution provider per platform (CoreML on macOS,
CPU elsewhere) instead of hardcoding CPU, and retry on CPU if a non-CPU
provider fails to load so transcription never regresses. Both the
selection and the fallback policy are unit-tested.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant