fix(providers): buffer partial UTF-8 across SSE chunk boundaries#238
Merged
Conversation
Streaming SSE decoding called String::from_utf8_lossy on each raw bytes_stream() chunk independently before handing the text to the line/block framers. When a multibyte UTF-8 character (e.g. a 3-byte CJK character) was split across two network chunks, each half was lossy-decoded separately and replaced with U+FFFD, dropping the character from streamed output. Add a byte-level Utf8StreamDecoder carry buffer that only emits complete UTF-8 sequences per chunk and retains the incomplete trailing bytes until the next chunk. Apply it to the OpenAI, OpenAI-Responses and Anthropic SSE paths, flushing any remainder at the true end of the stream. The Bedrock path already buffers raw bytes and is unaffected. Adds unit tests covering a CJK string split mid-character across chunk boundaries (no U+FFFD) and a legitimately-truncated tail. Addresses Sentry ELECTRON-3NH / ELECTRON-3P5.
kaizhou-lab
added a commit
to iOfficeAI/AionCore
that referenced
this pull request
Jul 23, 2026
## What Bump the `aion-*` git dependencies from **v0.2.6 → v0.2.7** and refresh `Cargo.lock` (`3cb928d4` → `445a18e1`). ## Why v0.2.7 picks up the streaming SSE UTF-8 fix ([iOfficeAI/aionrs#238](iOfficeAI/aionrs#238)): partial multibyte UTF-8 characters split across network chunk boundaries were previously `from_utf8_lossy`-decoded per chunk and dropped as `�` (U+FFFD), garbling streamed **CJK** model output. v0.2.7 buffers the incomplete trailing bytes across chunks so complete characters are emitted. Reported via Sentry **ELECTRON-3NH** / **ELECTRON-3P5** (OpenCode Zen `deepseek-v4-flash-free`); the defect affected all streaming providers with multibyte output. ## Changes - `Cargo.toml`: all six `aion-*` tag pins `v0.2.6` → `v0.2.7`. - `Cargo.lock`: aionrs source updated to `tag=v0.2.7#445a18e1`; 169 other dependencies unchanged. ## Verification - `cargo build` (full workspace) → `Finished` (exit 0), compiles clean against v0.2.7. Co-authored-by: zk <>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Streaming CJK (Chinese/Japanese/Korean) model output intermittently dropped characters, replacing them with
�(U+FFFD). Reported via Sentry ELECTRON-3NH / ELECTRON-3P5 (OpenCode Zendeepseek-v4-flash-free), but the defect affects all streaming providers on the OpenAI-compatible / Anthropic paths.Root cause
crates/aion-providers/src/stream_process.rsdecoded each rawbytes_stream()chunk independently withString::from_utf8_lossy(&chunk)before handing the text to the&str-accumulating framers. When a multibyte UTF-8 character (a 3-byte CJK char) is split across two network chunks, each half is lossy-decoded separately and each incomplete fragment becomesU+FFFD— so the character is lost. The signature (isolated�exactly at CJK character boundaries, surrounded by byte-perfect text) is a textbook split-multibyte-across-chunk-boundary bug.Fix
Utf8StreamDecodercarry buffer inframing.rsthat only emits complete UTF-8 sequences per chunk and retains the incomplete trailing bytes until the next chunk.Tests
cargo test -p aion-providersgreen (222 + 4 + 9 + 11 passed).framing_testcases: a CJK string (权限管理服务器) split mid-character across chunks reassembles with zero U+FFFD; a legitimately-truncated final byte is handled byflush()without panic; ASCII pass-through unchanged.