feat(eve): accept per-selection reasoning in dynamic model selection (#629)#769
Open
ZhYGuoL wants to merge 1 commit into
Open
feat(eve): accept per-selection reasoning in dynamic model selection (#629)#769ZhYGuoL wants to merge 1 commit into
ZhYGuoL wants to merge 1 commit into
Conversation
A defineDynamic selection accepted { model, modelContextWindowTokens,
modelOptions } but not reasoning, so a downshifted session inherited the
agent-level reasoning (e.g. xhigh), defeating the point of dropping to a
cheap classifier.
Add reasoning to PublicAgentModelSelectionDefinition and to the runtime
model reference, validate it against the same enum the agent-level field
accepts, and apply it at the model-call precedence point so a per-selection
reasoning overrides the agent default and falls back to it when omitted.
Same precedence semantics as modelContextWindowTokens and modelOptions.
Closes vercel#629 (Problem 1). Problem 2 (small-model outputSchema reliability)
is out of scope.
Signed-off-by: ZhYGuoL <[email protected]>
Contributor
|
@ZhYGuoL is attempting to deploy a commit to the Vercel Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
defineDynamicselection accepted{ model, modelContextWindowTokens, modelOptions }but notreasoning, so a downshifted session kept the agent-levelreasoning(e.g.xhigh), defeating the point of dropping to a cheap classifier.What changed
reasoning?: AgentReasoningDefinitiontoPublicAgentModelSelectionDefinitionand to the runtime model referenceInternalAgentModelDefinition(packages/eve/src/shared/agent-definition.ts).reasoningtoDYNAMIC_MODEL_SELECTION_KEYSand validate it invalidateDynamicModelSelectionagainst the same enum the agent-level field accepts, reusing the now-exportednormalizeAgentReasoningDefinition(packages/eve/src/runtime/agent/resolve-model.ts,packages/eve/src/internal/authored-definition/core.ts). Both selection-shape error strings (resolve-model.ts:192and:236) now listreasoning?.normalizeDynamicRuntimeModelResult, and apply it at the model-call precedence point:session.agent.modelReference.reasoning ?? session.agent.reasoning(packages/eve/src/harness/tool-loop.ts:852). Per-selection overrides the agent default; omitted falls back to it. Same precedence asmodelContextWindowTokens/modelOptions.docs/agent-config.md).How I verified
{ model, reasoning }throwsunknown key(s): reasoningatresolve-model.ts:178. Reverting only thetool-loop.tsprecedence line, the override test sees the model call getreasoning: "high"(agent default) instead of"low"(per-selection).reasoninglands on the reference and reaches the model call; omitting it keeps the agent default. Invalid values are rejected with a reasoning-specific error.packages/eve/src/runtime/agent/resolve-model.test.ts,packages/eve/src/harness/tool-loop.test.ts): "carries a per-selection reasoning onto the reference", "leaves reasoning unset when the selection omits it", "rejects a selection with an invalid reasoning value", "lets a per-selection reasoning override the agent reasoning effort". Each fails before, passes after.pnpm lint,pnpm typecheck,pnpm test:unit(4687 passed, 1 skipped),pnpm docs:check.Problem 2 from the issue (small-model
outputSchemareliability) is out of scope for this PR.Closes #629.