You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The problem
Agents using the tools sometimes hallucinate extra parameters. This causes the tool call to fail. In some cases, this leads to several rounds of trial-and-error until the agent finally finds the correct combination or pivots. This causes unnecessary token burn.
Across 72 evals, 3 trajectories contained extra "description" parameter to navigate_page MCP tool that could be safely ignored. The error message was "params must NOT have additional properties" which is ambiguous. In total this accounted for 5 failed tool calls. This seems to be aggravated by the evals asking for the agent to be verbose and describe each tool usage, but that is not an unreasonable usage scenario in itself.
Similar to the above, 1 run using CLI version had 2 attempts to use click tool with extra --filePath ... parameter, likely hallucinated from take_snapshot tool (both calls used --includeSnapshot true). In this case however error message clearly stated "Unknown argument: filePath." and model fixed the call on the next try.
Describe the solution you'd like
Allow extra named flags and ignore them for tools that do not use positional arguments. This seems to be in line with some other MCPs and tools that tend to allow agents to be extra chatty with tool invocations
Improve parsing error messages to list unexpected parameter names explicitly in all cases.
Is your feature request related to a problem? Please describe.
The problem
Agents using the tools sometimes hallucinate extra parameters. This causes the tool call to fail. In some cases, this leads to several rounds of trial-and-error until the agent finally finds the correct combination or pivots. This causes unnecessary token burn.
Examples
--expressionflag instead of positional parameters. There were 6 failed attempts, 2 of them can be directly attributed to the hallucinated flag and unclear error message. Two other could be a consequence of failed fixes to this issue.navigate_pageMCP tool that could be safely ignored. The error message was "params must NOT have additional properties" which is ambiguous. In total this accounted for 5 failed tool calls. This seems to be aggravated by the evals asking for the agent to be verbose and describe each tool usage, but that is not an unreasonable usage scenario in itself.clicktool with extra--filePath ...parameter, likely hallucinated fromtake_snapshottool (both calls used--includeSnapshot true). In this case however error message clearly stated "Unknown argument: filePath." and model fixed the call on the next try.Describe the solution you'd like
Describe alternatives you've considered
NA
Additional context