feat: Normalisation added for speech to text- #17 - #35
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
VishnuKrishnathu
left a comment
VishnuKrishnathu
left a comment
There was a problem hiding this comment.
Please edit the base branch and raise to the correct branch. You can get the details from @VISHNUDAS-tunerlabs
| transcript = " ".join(content for _, content in transcripts) | ||
| return {'status': 200, 'content': transcript} | ||
| raw_transcript = " ".join(content for _, content in transcripts) | ||
| print("final transcript: ", raw_transcript) |
There was a problem hiding this comment.
Please remove the print statement. You can use logger instead.
2 participants
When users speak to the chatbot in Hindi, the Automatic Speech Recognition (ASR) / Speech-to-Text (STT) pipeline transcribes the audio literally. For instance, numbers, dates, or currencies are written out as spoken words (e.g., transcribing spoken numbers like "कक्षा दो" or "एक सौ पचास" into words).
The Inverse Text Normalization (ITN) feature automatically parses the spoken Hindi text and converts it into its standard written/numerical formats (e.g., converting "कक्षा दो" to "कक्षा 2" and "एक सौ पचास" to "150"), making the transcribed text cleaner and much more readable for both the user and the downstream LLM processing.
Package Used: indic-itn==0.1.0 (specifically the HindiITN module)
Summary by CodeRabbit
New Features
Added Hindi, Tamil, Telugu, Kannada transcription text normalization to improve number and formatting consistency.
Added support for creating.
Improvements
Raised the minimum supported Python version to 3.11 and refreshed related dependencies to include text normalization.
Improved multi-part transcription output handling with normalization and safer fallback behavior.