Skip to content

fix: recover reasoning-only compact summaries instead of failing with EmptyResponse#220

Open
minhtq1234 wants to merge 1 commit into
iOfficeAI:mainfrom
minhtq1234:fix/compact-reasoning-empty-response
Open

fix: recover reasoning-only compact summaries instead of failing with EmptyResponse#220
minhtq1234 wants to merge 1 commit into
iOfficeAI:mainfrom
minhtq1234:fix/compact-reasoning-empty-response

Conversation

@minhtq1234

Copy link
Copy Markdown

Problem

Running /compact (and autocompact) fails with Compact failed: Empty response from LLM when the conversation model is a reasoning model served via an OpenAI‑compatible endpoint. Observed in AionUi with MiniMax M2.5 (custom provider): the model generates for ~17s, then the turn ends with event_type="Error" text_len=0.

Root cause

  • The compact request sets thinking: ThinkingConfig::Disabled (crates/aion-agent/src/compact/auto.rs), but the OpenAI projector only acts on ThinkingConfig::Enabled (crates/aion-providers/src/projector.rs), so "disabled" never reaches the wire — reasoning models reason anyway.
  • The OpenAI-compatible stream parser maps reasoning_content deltas to LlmEvent::ThinkingDelta (crates/aion-providers/src/openai.rs).
  • collect_stream_text in compact/auto.rs accumulated only TextDelta and discarded thinking deltas. A model that emits its whole summary as reasoning yields an empty string → CompactError::EmptyResponse, even though a complete summary was produced (output budget is not the issue — COMPACT_MAX_OUTPUT_TOKENS is 20k).

Fix

collect_stream_text now accumulates thinking deltas alongside text and falls back to the reasoning content when the model produced no plain text. format_compact_summary already degrades gracefully when <summary> tags are absent, so the recovered content flows through the existing parsing unchanged.

Behavior is otherwise preserved:

  • Text output, when present, always wins — reasoning is only used as a fallback.
  • A truly empty response (no text, no reasoning) still fails with EmptyResponse.
  • A channel that closes without Done can now salvage reasoning-only output; if nothing was received it still fails with EmptyResponse as before.

Testing

  • New regression test reasoning_only_response_is_recovered (ThinkingDelta-only stream → compaction succeeds, summary contains the recovered content, failure counter untouched).
  • Existing empty_response_fails still passes (truly empty stream → EmptyResponse).
  • cargo test -p aion-agent — all tests pass.
  • cargo fmt --all -- --check and cargo clippy -p aion-agent -- -D warnings — clean.

Repro context

AionUi desktop → aionrs conversation → MiniMax M2.5 (custom OpenAI-compatible provider) → /compactCompact failed: Empty response from LLM. With this change the reasoning-only summary is recovered and compaction completes.

… EmptyResponse

Reasoning models (e.g. MiniMax M2.5) served via OpenAI-compatible
endpoints may emit the entire compact summary as reasoning_content,
which the provider maps to LlmEvent::ThinkingDelta. The compact stream
collector discarded thinking deltas, so the collected text was empty and
/compact and autocompact failed with "Empty response from LLM" even
though the model produced a full summary. ThinkingConfig::Disabled does
not prevent this: not every provider honors it.

Collect thinking deltas alongside text and fall back to the reasoning
content when the model produced no plain text. format_compact_summary
already degrades gracefully when no <summary> tag is present. Truly
empty responses still fail with EmptyResponse.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant