Skip to content

Add stateful pointer input#340

Open
zengfanfan wants to merge 1 commit into
CursorTouch:mainfrom
zengfanfan:codex/stateful-pointer-input
Open

Add stateful pointer input#340
zengfanfan wants to merge 1 commit into
CursorTouch:mainfrom
zengfanfan:codex/stateful-pointer-input

Conversation

@zengfanfan

Copy link
Copy Markdown
Contributor

Description

Add one consolidated Pointer tool for mouse-button gestures that need to span multiple MCP calls:

Pointer(action="down", loc=[100, 100], button="left")
Pointer(action="move", loc=[140, 120], duration=0.2)
Pointer(action="move", loc=[180, 160])
Pointer(action="up")

The tool supports down, move, up, and cancel actions. It tracks at most one held button, serializes state changes, releases input on movement/setup failures, and automatically releases held input after 30 seconds by default (maximum 120 seconds). cancel releases left, right, and middle buttons for recovery, and normal server shutdown also clears tracked input.

The existing Click and Move interfaces remain unchanged. This PR adds one public tool rather than separate mouse-down, mouse-move, mouse-up, and recovery tools.

Motivation

Move(drag=True) is the right interface for an atomic drag that starts and finishes in one call. Some desktop workflows instead require multiple independently chosen path segments while a button remains held, such as free-form drawing or non-linear selection. Those workflows need explicit cross-call pointer state without overloading the existing stateless Click and Move contracts.

This complements the atomic explicit-start drag support merged in #328; it does not replace it.

Testing

  • ruff check passed for all changed Python files.
  • ruff format --check passed for all changed Python files.
  • Focused Pointer and drag regression suite: 71 passed.
  • Full suite: 422 passed, 3 failed.
    • The three failures are existing import failures in tests/test_iserror_compliance.py for module-level powershell_tool, clipboard_tool, and registry_tool symbols.
    • All three were reproduced on an unmodified upstream/main worktree at c46b8e4.
  • Real Windows smoke test passed against a temporary Tk window: the window received down, move, and up events in order, and the controller ended with no held button.

Screenshots

Not applicable; this changes input behavior without adding UI.

Related Issues

None.

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant