fix(tokens): gpt-5.5 and gpt-5.4 to 128000 — now actually measured#113
Merged
Conversation
added 2 commits
July 21, 2026 00:03
Both were 32768. The catalog said 128000 and I twice failed to establish whether that was true. #111 raised them to 100000 citing an upstream maximum. That number came from blockrun_llm/validation.py:233 — a client-side guard — so the probe never reached a provider. #112 reverted them to 32768 and recorded the catalog's 128000 as unrefuted but unverified. Re-probed 2026-07-21 with the SDK guard bypassed in-process: both accept 128000, as did all 19 models advertising a ceiling above 100000, including zai/glm-5.2 at 262144. Zero rejections. The catalog was right the whole time; the measurement was broken. Saying so explicitly because the previous conclusion pointed the other way, and this table is exactly where a wrong belief gets frozen. Root cause fixed upstream in blockrun-llm#27 and blockrun-llm-ts#15. Effect: normal turns are unchanged (CAPPED_MAX_TOKENS 16384). On a max_tokens stop the escalation ceiling goes from min(65536, 32768)=32768 to min(65536, 128000)=65536 — recovery doubles instead of clamping back below where it started. Local suite 636 pass.
Review of this PR turned up a wrong dependency claim I had made out loud: that the SDK fixes needed to merge before Franklin could raise these values, or users' local SDK would reject the larger number. Franklin never runs that guard. Both request paths — src/agent/llm.ts and src/proxy/server.ts — use raw fetch and import only payment helpers from @blockrun/llm, never chatCompletion, so validateMaxTokens is not on either path. These values are independent of whether blockrun-llm#27 and blockrun-llm-ts#15 land, and the merge ordering I recommended was unnecessary. Also drops the hardcoded '19 models' count, which would rot as the catalog changes, and marks the referenced SDK PRs as open rather than implying the root cause is already fixed everywhere.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
openai/gpt-5.5openai/gpt-5.4Why these moved twice
#111 raised them to 100,000 citing "upstream's stated maximum". I closed it: that number came from
blockrun_llm/validation.py:233, a client-side guard, so the probe never reached a provider. #112 put them back at 32,768 and recorded the catalog's 128,000 as unrefuted but unverified.Re-probed 2026-07-21 with the SDK guard bypassed in-process. Both accept 128,000 — as did all 19 models advertising a ceiling above 100,000, including
zai/glm-5.2at 262,144. Zero rejections.The catalog was right the whole time. The measurement was broken. Worth saying plainly, because the previous conclusion pointed the opposite way and this table is exactly where a wrong belief gets frozen.
Root cause fixed upstream: BlockRunAI/blockrun-llm#27 and BlockRunAI/blockrun-llm-ts#15 raise that guard from 100,000 to a 1,000,000 typo bound and reword the message so it no longer reads like a provider response.
Effect
CAPPED_MAX_TOKENSis 16,384 andESCALATED_MAX_TOKENSis 65,536, so a normal turn is unchanged. On amax_tokensstop the escalation ceiling goes frommin(65536, 32768) = 32768tomin(65536, 128000) = 65536— the recovery path doubles instead of being clamped back below where it started.Local suite 636 pass, 0 fail.