normalisation added - #28
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
bhavana-venugopal
left a comment
There was a problem hiding this comment.
@ramkumar-ns-dev Please change the base branch to the latest release-2.3.0 branch, and reshare.
@bhavana-venugopal here's the updated PR : #35 |
|
once the PR merged, i'll close this PR. |
When users speak to the chatbot in Hindi, the Automatic Speech Recognition (ASR) / Speech-to-Text (STT) pipeline transcribes the audio literally. For instance, numbers, dates, or currencies are written out as spoken words (e.g., transcribing spoken numbers like "कक्षा दो" or "एक सौ पचास" into words).
The Inverse Text Normalization (ITN) feature automatically parses the spoken Hindi text and converts it into its standard written/numerical formats (e.g., converting "कक्षा दो" to "कक्षा 2" and "एक सौ पचास" to "150"), making the transcribed text cleaner and much more readable for both the user and the downstream LLM processing.
Package Used: indic-itn==0.1.0 (specifically the HindiITN module)
Summary by CodeRabbit
New Features
Added Hindi, Tamil, Telugu, Kannada transcription text normalization to improve number and formatting consistency.
Added support for creating.
Improvements
Raised the minimum supported Python version to 3.11 and refreshed related dependencies to include text normalization.
Improved multi-part transcription output handling with normalization and safer fallback behavior.