ASR low‑resource language training

[libby-97]

I’d like to inquire whether current ASR models support fine-tuning to improve recognition accuracy for an existing low-resource language, specifically Dutch.
If such fine-tuning is feasible, could you kindly provide a general list of the datasets that have already been used for training? This would help us avoid potential data duplication during subsequent training for Dutch and mitigate issues such as overfitting or catastrophic forgetting.
Thanks！

[pengzhiliang]

Hi, yes — VibeVoice ASR supports fine-tuning via LoRA. Please check the finetuning-asr/ directory, which includes:

• LoRA fine-tuning script (lora_finetune.py)
• Inference with fine-tuned model (inference_lora.py)
• Data format documentation and a toy dataset for reference

You can prepare your Dutch audio data following the format described in the README, then run LoRA fine-tuning to improve recognition accuracy for Dutch.

[Mohcinimohamed]

I'm doing exactly this for Darija (Moroccan Arabic), another low-resource language. LoRA finetuning with 81 hours of community-collected data following the format in finetuning-asr/.

A few practical notes from my experience:

• QLoRA (4-bit) works on a single 16GB GPU (Quadro RTX 5000)
• customized_context helps with domain-specific terms
• Community data collection via a mobile app scaled to 27k samples

See PR #334 for a Darija toy dataset example. The same approach should work for Dutch.