fix(bin): thread codex max reasoning effort instead of dropping it - #1203
Open
talktejas wants to merge 3 commits into
Open
fix(bin): thread codex max reasoning effort instead of dropping it#1203talktejas wants to merge 3 commits into
talktejas wants to merge 3 commits into
Conversation
codex-cli 0.145.0 accepts the whole shared effort vocabulary through max, so fm-spawn no longer omits the flag for max. Omitting it left the worker at the ambient default effort with no error, a silent downgrade of what the caller asked for. The codex case now covers every level fm-spawn accepts, so no requested level can be dropped; an unsupported value would fail visibly at the API. ultra stays out of the vocabulary because its parallel-agent fan-out does not belong inside firstmate supervision.
The portable serial lane tests the pull-request merge ref, and the merge base carried a teardown conformance case that pinned tmux's old permissive kill-window selector. That case fails against main's exact-selector adapter, so the lane failed on material this branch never touched. Main already replaced the pinned assertion with a selector-form-agnostic one; merging it makes the branch's merge state green and matches what CI runs.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix a silent thinking-level downgrade when firstmate spawns a Codex worker. bin/fm-spawn.sh capped codex effort at xhigh and silently omitted the flag for max, so a caller asking for max got the worker's ambient default effort instead - a two-step downgrade with no error. Empirically verified against the installed codex-cli 0.145.0: the model catalog advertises max on the GPT-5.6 family, a live codex exec run at max reported 'reasoning effort: max', and an invalid value is rejected by the API with a visible 400 rather than downgraded. Chose to raise the ceiling rather than refuse loudly, because max is genuinely supported and, since the codex case now covers the entire vocabulary fm-spawn already validates (low|medium|high|xhigh|max), no requested level can fall through unmatched - the drop path is structurally gone rather than guarded. ultra is deliberately excluded even though the catalog advertises it: its parallel-agent fan-out changes execution shape and must never nest inside firstmate supervision. Scope was deliberately limited to codex; grok, opencode, kimi, claude and pi behaviour is intentionally unchanged, including grok's genuine low|medium|high ceiling. The old test test_codex_omits_invalid_max_effort was replaced by test_codex_threads_every_accepted_effort_level, which loops all five levels and asserts each reaches the launch command; it was confirmed to fail against the old code. The single-level codex test was folded into that loop as now-redundant. The harness-adapters skill's codex verification stamp was updated from 0.142.1 to 0.145.0. This is firstmate's own tracked material, so repo style applies: one sentence per line in Markdown, plain dash, no agent co-author.
What Changed
bin/fm-spawn.shnow includesmaxin the codex branch ofeffort_flag_for_harness, so--effort maxemits-c 'model_reasoning_effort="max"'instead of silently omitting the flag and letting the worker fall back to its ambient default. The codex case now covers the full vocabularyfm-spawnalready validates (low|medium|high|xhigh|max), so no accepted level can fall through unmatched;ultrastays excluded because its parallel-agent fan-out must not nest inside firstmate supervision.bin/fm-bootstrap.shcrew_dispatch_validateacceptscodex/maxas well, so a config-driven dispatch profile agrees with the direct spawn path. grok, opencode, kimi, claude and pi effort vocabularies are unchanged, including grok's genuinelow|medium|highceiling.test_codex_omits_invalid_max_effortwas replaced bytest_codex_threads_every_accepted_effort_level, which loops all five levels and asserts each reaches the launch command (the single-level codex test folded into it as redundant); the two bootstrap crew-dispatch rows now assert codexmaxis accepted. The harness-adapters skill's codex verification stamp moved from codex-cli 0.142.1 to 0.145.0 with the max evidence recorded.Risk Assessment
✅ Low: The follow-up commit is a two-line vocabulary alignment plus the two test rows that asserted the contradicted behavior, leaving every other harness and the excluded ultra level untouched, and it closes the only reachable divergence found in round 1.
Testing
I ran the two suites the change touches — fm-spawn-dispatch-profile and fm-bootstrap — and both pass, then went past pass/fail to capture the actual command firstmate would run: driving real fm-spawn through the suite's fake tmux at each effort level shows
codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="max"' ..., while the same driver against the base commit's fm-spawn.sh produces the same command with no effort flag at all, reproducing the silent downgrade the change fixes and confirming xhigh was unaffected. A live fm-bootstrap run confirms the config-facing half: a crew-dispatch rule with codex/max is now accepted silently, and grok/max still fails withinvalid effort: grok:max, so the deliberate scope limit to codex holds. No UI surface is involved, so the reviewer-visible evidence is CLI transcripts rather than screenshots; the worktree is clean and all evidence lives in the temp evidence directory.Evidence: codex launch command, base vs fixed, per effort level
=== BEFORE (base fa0d85d): max silently loses the effort flag === --effort xhigh -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="xhigh"' --dangerously-bypass-approvals-and-sandbox ... --effort max -> codex --model 'gpt-5.6-codex' --dangerously-bypass-approvals-and-sandbox ... === AFTER (c266166): every accepted level reaches codex === --effort low -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="low"' --dangerously-bypass-approvals-and-sandbox --effort medium -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="medium"' --dangerously-bypass-approvals-and-sandbox --effort high -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="high"' --dangerously-bypass-approvals-and-sandbox --effort xhigh -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="xhigh"' --dangerously-bypass-approvals-and-sandbox --effort max -> codex --model 'gpt-5.6-codex' -c 'model_reasoning_effort="max"' --dangerously-bypass-approvals-and-sandboxEvidence: fm-bootstrap crew-dispatch validation: codex:max accepted, grok:max still rejected
--- codex effort=max --- (no CREW_DISPATCH complaint - config accepted) --- grok effort=max --- CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: grok:maxEvidence: Full per-level launch commands and recorded state meta (fixed code)
Evidence: Evidence driver script
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 2 issues found → auto-fixed ✅
bin/fm-bootstrap.sh:722- The crew-dispatch config validator still caps codex effort at xhigh, so the raised ceiling only applies to the direct spawn path. Concrete path: config/crew-dispatch.json containing {"rules":[{"when":"big feature","use":{"harness":"codex","model":"gpt-5","effort":"max"}}]} makes crew_dispatch_validate print 'CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: codex:max', while the identical profile passed asfm-spawn --harness codex --effort maxnow launches with -c 'model_reasoning_effort="max"'. tests/fm-bootstrap.test.sh:780 and :797 assert that rejection, so the two owners of codex's effort vocabulary (effort_flag_for_harness and effort_ok) now disagree and the disagreement is test-locked. The intent's claim that no requested level can fall through unmatched holds for the launch command but not for the config-driven dispatch profile that feeds it. Earliest supported fix: add "max" to the codex list in effort_ok and update the two bootstrap test expectations, keeping grok's genuine low|medium|high ceiling untouched.bin/fm-spawn.sh:602- The codexcasenow matches the entire vocabulary that fm-spawn.sh:197 validates, so the branch can never fall through — it is functionally an unconditional printf. That is the intended structural property (the drop path is gone rather than guarded), and keeping the explicit list documents the verified vocabulary; noting only so the redundancy is not mistaken for a live guard.🔧 Fix: accept codex max effort in crew-dispatch validator
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
bash tests/fm-spawn-dispatch-profile.test.sh— all pass, including the newtest_codex_threads_every_accepted_effort_levellooping low|medium|high|xhigh|maxbash tests/fm-bootstrap.test.sh— all pass, including the flippedcodex max effort is acceptedandarray profile codex max effort is acceptedcrew-dispatch rowsManual before/after driver/tmp/no-mistakes-evidence/01KYMW35QNBBD19PSX9KPJATPF/codex-effort-evidence.shrun against bothgit checkout fa0d85d -- bin/fm-spawn.shand the targetc266166(file restored afterwards; worktree clean) to capture the literal codex launch command per effort levelLivebin/fm-bootstrap.shrun with a realconfig/crew-dispatch.jsoncontainingcodex/effort:maxandgrok/effort:maxto compare validator output✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.