feat: true streaming acoustic decode + vv_capi_tts_stream (realtime-0.5B)#8
Merged
Conversation
Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Removes gitignored-fixture dependency so the bit-exact gate actually runs in CI. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Thread StreamingCache through the whole acoustic decoder (decoder_forward_streaming) and add run_decoder_chunk_streaming, a chunk driver mirroring run_encoder_chunk_streaming. Move the frame-major -> ggml-order latent packing into decode_latent_sequence (shared pack_latents_ggml_order helper) so single-shot and streaming consume the same frame-major layout and feed the decoder byte-identically. Gated parity test proves chunked == single-shot at rmse/rms=0.0000. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Interleave the realtime-0.5b LM window loop with per-window acoustic decode so audio is produced as it is generated, delivered via an on_chunk callback. The LM loop (dpm_solver_sample, EOS logic, latents) is byte-identical to the batch path; the only change is decoding each completed speech window through a shared decoder StreamingCache instead of buffering all latents and decoding once at the end. Concatenating the emitted chunks is bit-exact vs the former single-shot decode (proven by the Task 2 decoder-parity gate). vibevoice_tts_generate's realtime path is now a thin wrapper that calls vibevoice_tts_generate_streaming with an on_chunk that appends to the output buffer (DRY - one loop, two entry points). on_chunk returning false aborts the run early and cleanly. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Jul 9, 2026
localai-org/vibevoice.cpp#8 merged to master as 000e372; move the pin off the PR branch commit onto the merged master commit. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code]
mudler
added a commit
to mudler/LocalAI
that referenced
this pull request
Jul 9, 2026
…ream) (#10764) * feat(vibevoice-cpp): true streaming TTSStream via vv_capi_tts_stream Replaces the synth-to-tempfile TTSStream hack with a real streaming path: binds the new vv_capi_tts_stream callback ABI via a single reusable purego callback (CGO_ENABLED=0-safe, no runtime/cgo), copies each int16 PCM window into the gRPC results channel after the streaming WAV header. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * test(vibevoice-cpp): real-model streaming integration test with TTFA measurement Gated behind VIBEVOICE_IT=1, this Ginkgo spec dlopens the engine .so and drives the exact Go->purego->C TTSStream/TTS path against the real vibevoice-realtime-0.5B model. It measures time-to-first-audio for the streaming path versus the batch path and asserts the streaming win: 44-byte WAV header first, >=2 PCM windows, non-silent audio, and TTFA < total_stream. Without the env var the spec skips so CI and normal go test are unaffected. Measured: TTFA 2.38s vs batch deliver-time 39.96s (first audio in 5.9% of the batch time, ~17x faster), 18 stream chunks, non-silent 24kHz PCM. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(vibevoice-cpp): pin streaming-decoder engine build Bumps VIBEVOICE_CPP_VERSION to the streaming-decoder engine commit that adds vv_capi_tts_stream (localai-org/vibevoice.cpp#8). Re-pin to the merged master commit once that PR lands. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(vibevoice-cpp): re-pin to merged streaming-decoder commit localai-org/vibevoice.cpp#8 merged to master as 000e372; move the pin off the PR branch commit onto the merged master commit. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * test(vibevoice-cpp): check writer errors in TTFA report (errcheck) golangci-lint errcheck flagged the unchecked fmt.Fprintf calls that print the streaming TTFA headline. Build the report once with fmt.Sprintf and write it per destination with an explicitly discarded error, matching the GinkgoWriter reporting idiom used by the other backend tests. Signed-off-by: Ettore Di Giacinto <[email protected]> Assisted-by: Claude:claude-fable-5 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <[email protected]> Co-authored-by: Ettore Di Giacinto <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
True streaming acoustic decode for realtime-0.5B TTS
VibeVoice-Realtime is a streaming-first model, and the LM already generates speech latents incrementally. The only batched-to-the-end stage was the acoustic decoder (
decode_latent_sequenceran once over the whole latent sequence). This PR makes the decoder stream, interleaves it with the LM window loop, and exposes a callback ABI so callers get audio as it is produced.What's here
sconv_transpose1d_causal_streaming(src/conv1d.cpp): the one new primitive. KeepsL = ceil((K-1)/stride)input frames of left context per conv and emits exactly[L*stride : L*stride + T_chunk*stride)of the transpose-conv output. Bit-exact vs a single-shot pass, by construction (proof in the design doc).decoder_forward_streaming+run_decoder_chunk_streaming: thread aStreamingCachethrough the whole decoder, mirroring the existing encoder streaming path. Reusesblock1d_forward_streamingandsconv1d_causal_streaming. (Every decoder forward conv is stride-1, sois_finalis a no-op and chunked decode is bit-exact for any window sizes.)vibevoice_tts_generate_streaming: interleaves the (byte-identical) LM window loop with per-window decode, emitting through astd::function<bool(const float*, int)>callback. The batchvibevoice_tts_generaterealtime path is now a thin wrapper over it (DRY).vv_capi_tts_stream: flat purego-friendly callback ABI. int16 conversion reusessave_wav_pcm16exactly.Scope: realtime-0.5B only. The 1.5B path is untouched (streaming returns a distinct error for 1.5B).
Validation
test_sconvt_streaming(always-run, self-contained): chunked transpose-conv == single-shot,max_abs=0.0across K=2·stride ratios.test_decoder_chunked_parity(gated): chunked decode ==decode_latent_sequence, rmse/rms = 0.0000 (bit-exact).test_generate_stream_parity(gated): >=2 windows, length/first/last match, clean early-abort.test_capi_stream(gated): streamed int16 byte-identical to file PCM.Refactor note
The frame-major -> ggml-order latent packing moved from the two generate call sites INTO
decode_latent_sequence(so batch and streaming feed the decoder identically). Verified element-for-element identical;test_e2e_tts(batch path) stays green.Assisted-by: Claude:claude-opus-4-8 [Claude Code]
🤖 Generated with Claude Code