Add background-noise injection for voice simulations - #127
Open
dalmia wants to merge 9 commits into
Open
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
dalmia
force-pushed
the
claude/festive-rosalind-92d0d6
branch
2 times, most recently
from
July 10, 2026 07:36
4524010 to
bccfc80
Compare
Simulate a caller in a noisy place (street, cafe, crowd, home) so the tested agent's STT is stressed under realistic background. Noise is a per-persona setting, mixed continuously under the simulated caller's audio. - calibrate_agent/agent/noise/: schema + resolver (off/fixed/random/mixture, seeded per run), assets (env + Vaani speaker pool, prepare_assets), the SimulationNoiseGenerator (steady loop / event scatter / crowd overlay + backgroundify via scipy filters + synthetic-IR reverb / merge / RMS-normalize / seamless loop), and clean+noisy save (continuous reconstruction). - run_simulation.py: read persona.noise, build the track, attach SoundfileMixer to the sim-user transport, write clean_ + noisy conversation audio. - utils.combine_audio_files: optional prefix kwarg (default ""). - pyproject: add soundfile + scipy (audiomentations dropped — soxr conflicts with pipecat 1.0); register agent/assets/noise package-data. - Examples (per-persona noise variants + README), docs page, and full design spec + TODO under design/. Default-off: existing runs unchanged. Generated ESC-50-derived assets are gitignored (CC BY-NC; CC0 re-source before shipping — Phase 10). 57 unit tests; verified end-to-end across all noise modes. Co-Authored-By: Claude Opus 4.8 <[email protected]>
- run_simulation.py: when max_turns is reached, force-cancel the pipeline after a short grace if the graceful EndFrame hasn't drained. A continuous background-noise mixer emits audio non-stop, so the EndFrame could stall behind an endless output stream and the call hung until the agent's idle-timeout (~5 min). Verified: the hang condition now ends in ~2s; non-noise sims still flush gracefully first. - Spec/docs: merge Terminology + glossary and de-jargon it, per-persona distribution wording (noise resolves per persona × scenario), drop "babble" for "background speakers". - Add example configs: easy/hard (env/people/mixed at low vs high difficulty), matrix (type × difficulty grid), progressive (monotonic ramp), showcase (all modes). Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add scripts/fetch_noise_assets.py: pulls ESC-50 environmental clips and Vaani speaker clips from the Hugging Face Hub, places them under data/, and runs prepare_assets(). Runbook step B.3 is now a single command. Co-Authored-By: Claude Opus 4.8 <[email protected]>
dalmia
force-pushed
the
claude/festive-rosalind-92d0d6
branch
from
July 11, 2026 09:50
bccfc80 to
18528ba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
calibrate_agent/agent/noise/package — config schema + per-run resolver (off/fixed/random/mixture, seeded), asset prep, the generator (steady loop / event scatter / crowd overlay + backgroundify filters + synthetic-IR reverb / merge / RMS-normalize / seamless loop), and clean+noisy save — wired intorun_simulation.pyviaSoundfileMixer. Adds aprefixkwarg tocombine_audio_files.design/.Notes
design/noise_injection_todo.md); regenerate locally viaprepare_assets.audiomentationscouldn't be used (itssoxr<1.0pin conflicts with pipecat 1.0) → implemented withscipy+soundfile+soxr.docs/noise-injection.mdxisn't indocs/docs.jsonnav yet.