Skip to content

feat(eve): accept per-selection reasoning in dynamic model selection (#629)#769

Open
ZhYGuoL wants to merge 1 commit into
vercel:mainfrom
ZhYGuoL:feat/dynamic-model-selection-reasoning
Open

feat(eve): accept per-selection reasoning in dynamic model selection (#629)#769
ZhYGuoL wants to merge 1 commit into
vercel:mainfrom
ZhYGuoL:feat/dynamic-model-selection-reasoning

Conversation

@ZhYGuoL

@ZhYGuoL ZhYGuoL commented Jul 14, 2026

Copy link
Copy Markdown

A defineDynamic selection accepted { model, modelContextWindowTokens, modelOptions } but not reasoning, so a downshifted session kept the agent-level reasoning (e.g. xhigh), defeating the point of dropping to a cheap classifier.

What changed

  • Add reasoning?: AgentReasoningDefinition to PublicAgentModelSelectionDefinition and to the runtime model reference InternalAgentModelDefinition (packages/eve/src/shared/agent-definition.ts).
  • Add reasoning to DYNAMIC_MODEL_SELECTION_KEYS and validate it in validateDynamicModelSelection against the same enum the agent-level field accepts, reusing the now-exported normalizeAgentReasoningDefinition (packages/eve/src/runtime/agent/resolve-model.ts, packages/eve/src/internal/authored-definition/core.ts). Both selection-shape error strings (resolve-model.ts:192 and :236) now list reasoning?.
  • Carry the value onto the reference in normalizeDynamicRuntimeModelResult, and apply it at the model-call precedence point: session.agent.modelReference.reasoning ?? session.agent.reasoning (packages/eve/src/harness/tool-loop.ts:852). Per-selection overrides the agent default; omitted falls back to it. Same precedence as modelContextWindowTokens/modelOptions.
  • Document the field on the selection object (docs/agent-config.md).

How I verified

  • Before: a resolver returning { model, reasoning } throws unknown key(s): reasoning at resolve-model.ts:178. Reverting only the tool-loop.ts precedence line, the override test sees the model call get reasoning: "high" (agent default) instead of "low" (per-selection).
  • After: per-selection reasoning lands on the reference and reaches the model call; omitting it keeps the agent default. Invalid values are rejected with a reasoning-specific error.
  • Tests (packages/eve/src/runtime/agent/resolve-model.test.ts, packages/eve/src/harness/tool-loop.test.ts): "carries a per-selection reasoning onto the reference", "leaves reasoning unset when the selection omits it", "rejects a selection with an invalid reasoning value", "lets a per-selection reasoning override the agent reasoning effort". Each fails before, passes after.
  • Full local green: pnpm lint, pnpm typecheck, pnpm test:unit (4687 passed, 1 skipped), pnpm docs:check.

Problem 2 from the issue (small-model outputSchema reliability) is out of scope for this PR.

Closes #629.

A defineDynamic selection accepted { model, modelContextWindowTokens,
modelOptions } but not reasoning, so a downshifted session inherited the
agent-level reasoning (e.g. xhigh), defeating the point of dropping to a
cheap classifier.

Add reasoning to PublicAgentModelSelectionDefinition and to the runtime
model reference, validate it against the same enum the agent-level field
accepts, and apply it at the model-call precedence point so a per-selection
reasoning overrides the agent default and falls back to it when omitted.
Same precedence semantics as modelContextWindowTokens and modelOptions.

Closes vercel#629 (Problem 1). Problem 2 (small-model outputSchema reliability)
is out of scope.

Signed-off-by: ZhYGuoL <[email protected]>
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@ZhYGuoL is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dynamic model selection: per-selection reasoning control + small-model reliability for task-session outputSchema

1 participant