Japanese Pitch Accent Analyzer is a specialized research and study tool designed to help learners master Japanese prosody. It allows users to compare their speech against reference audio (e.g., anime clips) using AI-driven forced alignment and pitch normalization.
The project focuses on Mora-synchronous comparison, moving beyond raw waveform matching to provide linguistically meaningful feedback.
- Vocal Extraction (UVR): Uses
audio-separator(Ultimate Vocal Remover) to isolate voices from background music/noise in anime clips. - Voice Activity Detection (VAD): Employs Silero VAD to trim silences, ensuring the analyzer focuses only on spoken segments.
- Forced Alignment: Utilizes
Wav2Vec2to map Hiragana/Kanji transcripts to exact timestamps in the audio. - Theoretical Prediction: Integrates
pyopenjtalkto generate the "dictionary correct" pitch accent pattern (High/Low) for the input text. It uses theMarinepredictor.
- Acoustic Scaling: Converts raw Hertz (Hz) to Semitones relative to the speaker's mean. This allows a deep-voiced male to be compared directly with a high-pitched female character.
- Mora-Anchored Plotting: Instead of simple time-warping, pitch is aggregated per Mora (syllable), providing an "OJAD-style" visual graph.
To install this project use the following commands.
# Clone the repository
git clone <your-repo-url>
cd pitch_accent
# Create environment and install dependencies
pip install -e .The recommended models are:
They should be included in the .env file with the path to directory of the checkpoints like:
ALIGNER_MODEL_PATH="checkpoints/wav2vec/japanese-wav2vec2-large-rs35kh"
VOCALS_REMOVER_PATH="checkpoints/vocals"The standalone app is designed for rapid iteration of the pitch accent analysis logic.
python src/pitch-accent-analyzer/app.py- Upload Reference: Provide the anime audio segment.
- Input Transcript: Enter the Hiragana/Kanji (e.g., わたしはげんきです).
- Record: Speak the same sentence into the microphone.
- Analyze: View the synchronized pitch contours.
The pitch is aligned to Moras instead of using the time as X-axis.
The pitch is normalized using semitones with the formula12 * log2(f / mean_f).
Normalizing to semitones relative to the mean removes the "Gender/Age Gap"
while preserving the "Intonation Shape."
Fundamental Frequency (F0) estimation (Parselmouth/Praat) fails in the presence of harmonic noise (music). Without UVR, the anime pitch curve would be jagged and incorrect.
Handles unified ingestion. It converts non-WAV formats to WAV on-the-fly, as Parselmouth is strict about headers.
The "brain" of the temporal alignment.
align(audio, text): Generates character-level timestamps.group_into_moras(spans): Merges small kana (e.g., ょ) into their parent mora for correct Japanese prosody analysis.
Uses matplotlib-fontja to ensure Japanese characters render correctly
across all Operating Systems without manual font configuration.
This tool is strictly for personal educational use. It does not facilitate the distribution of copyrighted content. All audio processing is performed locally; no data is uploaded to external servers.
- Japanese Aligner: reazon-research/japanese-wav2vec2-large-rs35kh
- Parselmouth: https://github.com/YannickJadoul/Parselmouth
- OJAD: https://www.gavo.t.u-tokyo.ac.jp/ojad/phrasing/index
- Ultimate Vocal Remover: python-audio-separator
aipracticecafe aipracticecafe-codeberg
This project is licensed under the MIT. Details are in the LICENSE file.