Skip to content

feat(transcription): ParakeetEngine via sherpa-rs transducer (Track-B step 2)#10

Open
farce1 wants to merge 1 commit into
feat/asr-engine-traitfrom
feat/parakeet-engine
Open

feat(transcription): ParakeetEngine via sherpa-rs transducer (Track-B step 2)#10
farce1 wants to merge 1 commit into
feat/asr-engine-traitfrom
feat/parakeet-engine

Conversation

@farce1

@farce1 farce1 commented Jun 14, 2026

Copy link
Copy Markdown
Owner

What — Track-B step 2 (stacked on #9)

Implements NVIDIA Parakeet-TDT-v3 behind the AsrEngine trait from #9 — and, after verifying the sherpa-rs API, does it with zero new dependencies.

Why sherpa-rs, not transcribe-rs

You asked for transcribe-rs; researching it surfaced a real cost: transcribe-rs pulls ort (ONNX Runtime 1.22), which collides with our sherpa-rs-sys's bundled ONNX Runtime (~1.17) — and we must keep sherpa (VAD + diarization). That meant a second ONNX Runtime + ort load-dynamic + dylib bundling. You opted to verify the sherpa path first — and it works: sherpa-rs 0.6.8 ships transducer::TransducerRecognizer (wraps SherpaOnnxOfflineRecognizer), which loads NeMo Parakeet-TDT directly. So Parakeet rides our existing sherpa-rs / single ONNX Runtime. No transcribe-rs, no dual-ORT, no packaging changes.

Changes

  • engine.rs: ParakeetEngine wrapping TransducerRecognizer (model_type="nemo_transducer", decoding_method="greedy_search", 16 kHz / feature_dim 80, CPU), implementing AsrEngineAsrOutput { text, language: "" }.
  • engine.rs: load_engine(engine: &str, model_dir) -> Box<dyn AsrEngine> — string dispatch mirroring the existing meetings.asr_engine value; default "whisper".
  • worker.rs/mod.rs: worker builds the engine via the factory from WorkerConfig.asr_engine (set to "whisper" for now → behavior unchanged).

Verification

  • cargo check + clippy -D warnings pass — confirms Parakeet compiles against sherpa-rs (the key de-risking) with no dead-code. Full cargo test: only the pre-existing model_archive_consts_tests (REPLACE_WITH_ placeholders) fail.
  • ⚠️ Runtime not yet validated — needs the sherpa-onnx Parakeet-v3 int8 model (encoder/decoder/joiner.int8.onnx + tokens.txt from csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8) + a real recording. Same untestable-in-CI nature as the existing Whisper FFI path.

Model (next step)

Use the sherpa-onnx export sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8 (NOT Handy's transcribe-rs tarball — different layout). Next steps: per-engine model-dir resolution in model.rs, the download (parallel to the Whisper download), the engine-selection setting (default Parakeet-v3, Whisper-turbo fallback) + picker UI.

Merge notes

Stacked on #9 (base = feat/asr-engine-trait); retarget to main once #9 merges. Overlaps worker.rs/mod.rs with #4/#6/#8.

CI red on the usual maintainer-only blockers (vendor binary + REPLACE_WITH_), not this PR. Attribution: Parakeet-TDT-0.6B-v3 is CC-BY-4.0 (credit NVIDIA in the app's licenses screen).

… AsrEngine

Track-B step 2. Implements NVIDIA Parakeet-TDT-v3 via sherpa-rs'
TransducerRecognizer (nemo_transducer, greedy_search, 16kHz/80-dim) behind the
AsrEngine trait — reusing the existing sherpa-rs / ONNX Runtime. No transcribe-rs,
no second ONNX Runtime, no new dependency (the dual-ORT risk is avoided).

A string-dispatched load_engine(engine, model_dir) factory selects the engine via
WorkerConfig.asr_engine (default 'whisper' — behavior unchanged). Parakeet model-dir
resolution, engine-selection setting/UI, and model download follow in later steps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant