Skip to content

Tool argument array constraints are stricter than docs imply #4

Description

@MaestroError

Summary

assertToolUsed($tool, [...]) currently uses strict array equality for argument matching, which makes tool assertions brittle in realistic runs.

Why this matters

Agent tool arguments are often associative arrays where key order may vary, or where the actual invocation includes additional harmless fields. Strict equality makes the simple array constraint form fail more often than users are likely to expect.

Evidence

  • src/Concerns/HasToolAssertions.php:135-137 uses $invocation->arguments === $constraint.
  • The README presents the array form as a straightforward argument constraint:
    • README.md:494-497

Current risk

These assertions can fail even when the invocation contains the expected arguments, for example because:

  • keys are in a different order, or
  • the invocation includes extra metadata alongside the expected subset.

That makes the array shorthand much less ergonomic than it appears in docs.

Suggested fix

This is only a suggestion; the actual fix should be planned carefully.

Possible directions:

  • Treat array constraints as subset matches rather than full strict equality.
  • Or introduce/document two explicit modes, for example exact-match vs subset-match.
  • Keep the closure-based form as the escape hatch for advanced constraints.
  • Add tests covering key-order differences and additional invocation arguments.

Notes

This is mostly a developer-experience issue, but it can also cause confusing false negatives in test suites that rely on tool assertions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions