Skip to content

fix(providers): buffer partial UTF-8 across SSE chunk boundaries#238

Merged
kaizhou-lab merged 1 commit into
mainfrom
fix/sse-utf8-chunk-boundary
Jul 23, 2026
Merged

fix(providers): buffer partial UTF-8 across SSE chunk boundaries#238
kaizhou-lab merged 1 commit into
mainfrom
fix/sse-utf8-chunk-boundary

Conversation

@kaizhou-lab

Copy link
Copy Markdown
Contributor

Problem

Streaming CJK (Chinese/Japanese/Korean) model output intermittently dropped characters, replacing them with (U+FFFD). Reported via Sentry ELECTRON-3NH / ELECTRON-3P5 (OpenCode Zen deepseek-v4-flash-free), but the defect affects all streaming providers on the OpenAI-compatible / Anthropic paths.

Root cause

crates/aion-providers/src/stream_process.rs decoded each raw bytes_stream() chunk independently with String::from_utf8_lossy(&chunk) before handing the text to the &str-accumulating framers. When a multibyte UTF-8 character (a 3-byte CJK char) is split across two network chunks, each half is lossy-decoded separately and each incomplete fragment becomes U+FFFD — so the character is lost. The signature (isolated exactly at CJK character boundaries, surrounded by byte-perfect text) is a textbook split-multibyte-across-chunk-boundary bug.

Fix

  • Add a byte-level Utf8StreamDecoder carry buffer in framing.rs that only emits complete UTF-8 sequences per chunk and retains the incomplete trailing bytes until the next chunk.
  • Use it in the OpenAI, OpenAI-Responses and Anthropic SSE paths; flush any remainder at the true end of the stream.
  • The Bedrock path already buffers raw bytes and is untouched.

Tests

  • cargo test -p aion-providers green (222 + 4 + 9 + 11 passed).
  • New framing_test cases: a CJK string (权限管理服务器) split mid-character across chunks reassembles with zero U+FFFD; a legitimately-truncated final byte is handled by flush() without panic; ASCII pass-through unchanged.

Streaming SSE decoding called String::from_utf8_lossy on each raw
bytes_stream() chunk independently before handing the text to the
line/block framers. When a multibyte UTF-8 character (e.g. a 3-byte
CJK character) was split across two network chunks, each half was
lossy-decoded separately and replaced with U+FFFD, dropping the
character from streamed output.

Add a byte-level Utf8StreamDecoder carry buffer that only emits
complete UTF-8 sequences per chunk and retains the incomplete
trailing bytes until the next chunk. Apply it to the OpenAI,
OpenAI-Responses and Anthropic SSE paths, flushing any remainder at
the true end of the stream. The Bedrock path already buffers raw
bytes and is unaffected.

Adds unit tests covering a CJK string split mid-character across
chunk boundaries (no U+FFFD) and a legitimately-truncated tail.

Addresses Sentry ELECTRON-3NH / ELECTRON-3P5.
@kaizhou-lab
kaizhou-lab merged commit a23390d into main Jul 23, 2026
13 checks passed
@kaizhou-lab
kaizhou-lab deleted the fix/sse-utf8-chunk-boundary branch July 23, 2026 08:32
kaizhou-lab added a commit to iOfficeAI/AionCore that referenced this pull request Jul 23, 2026
## What

Bump the `aion-*` git dependencies from **v0.2.6 → v0.2.7** and refresh
`Cargo.lock` (`3cb928d4` → `445a18e1`).

## Why

v0.2.7 picks up the streaming SSE UTF-8 fix
([iOfficeAI/aionrs#238](iOfficeAI/aionrs#238)):
partial multibyte UTF-8 characters split across network chunk boundaries
were previously `from_utf8_lossy`-decoded per chunk and dropped as `�`
(U+FFFD), garbling streamed **CJK** model output. v0.2.7 buffers the
incomplete trailing bytes across chunks so complete characters are
emitted.

Reported via Sentry **ELECTRON-3NH** / **ELECTRON-3P5** (OpenCode Zen
`deepseek-v4-flash-free`); the defect affected all streaming providers
with multibyte output.

## Changes

- `Cargo.toml`: all six `aion-*` tag pins `v0.2.6` → `v0.2.7`.
- `Cargo.lock`: aionrs source updated to `tag=v0.2.7#445a18e1`; 169
other dependencies unchanged.

## Verification

- `cargo build` (full workspace) → `Finished` (exit 0), compiles clean
against v0.2.7.

Co-authored-by: zk <>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant