Skip to content

Proposal: make Move drag atomic with an explicit start #326

Description

@zengfanfan

Problem

Move(drag=True) starts from the cursor's position at execution time. An agent that knows both endpoints must first move to the source and then issue a second tool call, leaving a race window in which the cursor can change. The current drag helper also does not expose a bounded duration for intermediate movement.

Proposed additive API

Extend the existing Move tool rather than add another tool:

Move(
    loc=[end_x, end_y],
    drag=True,
    from_loc=[start_x, start_y],  # optional; current cursor remains the default
    duration=0.25,               # optional; seconds, bounded to 0..10
)

Behavior and compatibility:

  • Omitting both new parameters preserves the existing current-cursor drag.
  • from_loc and duration are rejected unless drag=True.
  • Both endpoints are validated before any mouse input.
  • The mouse button is released in a finally path if pressing or movement raises.
  • A non-zero duration emits intermediate positions and always finishes at the requested endpoint.
  • Because Move can drag, its MCP annotations should describe it as destructive and non-idempotent.

Duplicate check

I searched open and closed issues/PRs for explicit-start, deterministic, duration-based, and mouse-drag proposals. I found no implementation other than my withdrawn earlier proposal (#312/#313). PR #93 proposes a multi-point MousePath, but it does not perform button-held drag-and-drop or make the existing Move drag atomic.

Validation prepared

  • Unit coverage for legacy compatibility, JSON-list coercion, endpoint validation, duration bounds, intermediate movement, and release-on-error.
  • Ruff lint/format checks on the changed files.
  • Manual Windows 10 drag smoke tests.

Design choices

  • Use additive from_loc and duration parameters on the existing Move tool.
  • Retain the current cursor as the default start for backward compatibility.
  • Correct the Move annotations in the same focused change because drag-and-drop is destructive and non-idempotent.

The complete implementation is available in #328. Feedback is welcome through PR review, but no reply to this issue is required before review or merge.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions