fix(tools): don't mask empty streaming tool-selection as %!w(<nil>)#65
Merged
Conversation
When the streaming tool-selection decision returned no tool calls and empty
content on every attempt, the retry loop exhausted with lastErr still nil, so
the final fmt.Errorf("...: %w", maxRetries, lastErr) rendered the literal
"%!w(<nil>)" — hiding the real cause.
This reliably bites image turns with reasoning models (e.g. gemma-4-e2b via
LocalAI): the model streams reasoning but its reasoning exhausts the output
budget before any visible content, finishing with finish_reason=length and no
content. The empty-content path then retried three identical truncations and
reported %!w(<nil>).
Capture finish_reason from StreamEventDone and, in the empty-content branch:
- finish_reason=length: fail fast with an actionable error (retrying
truncates identically) instead of looping into a nil wrap.
- otherwise: keep retrying, but always record a real lastErr so the final
error is never a nil wrap.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
mudler
added a commit
to mudler/nib
that referenced
this pull request
Jul 7, 2026
Pulls github.com/mudler/cogito#65 (v0.11.1-0.20260707230916-588db3bcbae5): streaming tool-selection no longer masks an empty/truncated decision as "%!w(<nil>)". Fixes image turns where a reasoning model's reasoning exhausts the output budget (finish_reason=length) before producing content — nib now surfaces an actionable error instead of the meaningless nil wrap. Co-authored-by: Ettore Di Giacinto <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Image turns with a reasoning model (e.g.
gemma-4-e2bvia LocalAI) fail with the meaningless error:%!w(<nil>)is Go telling usfmt.Errorf("...: %w", …, lastErr)was called with a nillastErr.Root cause (reproduced against LocalAI v4.6.2 + gemma-4-e2b)
decisionWithStreaming's empty-content retry path (content == "" && len(toolCalls) == 0)continued without settinglastErr. When every attempt hit it, the loop exhausted withlastErr == niland rendered%!w(<nil>).This reliably happens on image turns: the vision model streams reasoning whose length exhausts the output-token budget (vision tokens crowd the context), finishing with
finish_reason=lengthand no visible content and no tool call. The loop then retried three identical truncations and reported the nil wrap. Verified deterministically: with a constrained budget the model emitsreasoningonly,content_len=0,tools=0,finish=length.Fix
Capture
finish_reasonfromStreamEventDone, and in the empty-content branch:finish_reason=length→ fail fast with an actionable error (retrying truncates identically), never a nil wrap.lastErrso the final error is a real cause.The non-streaming
decision()path already accepted empty content without this retry, so only the streaming path was affected.Tests
decision_empty_content_test.go:TestDecisionWithStreamingTruncatedEmptyContent— reproduces the exact%!w(<nil>); asserts a meaningful,length-naming error and fail-fast (1 attempt).TestDecisionWithStreamingEmptyContentNoFinishReason— genuinely-empty responses still error out, never nil-wrapped.Both fail on
main(RED) and pass with the fix. Existing streaming tests unchanged.🤖 Generated with Claude Code