feat(agent-service): add native computer use tools

[hiqiancheng]

First external contributors may need to complete the CLA Assistant check before merge.
For code changes, this PR must link the related issue or RFC in the section below.

Summary

Adds a TouchAI-native computer-use gateway for desktop automation without bundling Cua, Python, sandbox runtimes, OmniParser, or model weights.

The PR introduces disabled-by-default built-in tools for computer_session, computer_observe, and computer_act, plus a Windows-first Rust runtime that reports platform capabilities, observes native/tree-first desktop state, executes one bounded action with structured receipts, and leaves adapter seams for future Cua-like or vision grounding providers.

It also updates AgentService tool-round scheduling so computer_act runs sequentially within a model tool round and only one successful computer action is counted per round, while later rounds can act again after another model step.

Related issue or RFC

Link the issue or RFC:

• Related to [RFC]: support computer use for grounded desktop interaction #111

For code changes, link the tracking issue. Only documentation wording, link fixes, or comment-only cleanups may skip the issue-first flow.

AI assistance disclosure

If AI tools materially assisted this contribution, disclose that here or point to the relevant commit trailer.

• Tool(s) used: OpenAI Codex with delegated review agents
• Scope of assistance: product research synthesis, implementation, tests, review loops, PR drafting
• Human review or rewrite performed: every changed file was reviewed in the local diff; blocking review findings were converted into regression tests and fixes before submission
• Architecture or boundary impact: adds a new built-in desktop tool boundary across AgentService, BuiltInToolService, NativeService, Rust native commands, and seed data

Testing evidence

List the commands you ran and the results you observed.

• Every code PR: pnpm test:pr (includes type check, lint, format, all tests, and frontend coverage)
  • Passed locally. Observed desktop unit tests 123 passed / 692 tests, Rust tests passed, coverage completed, and site build completed. Existing lint warnings remain in unrelated Vue/test files, with 0 lint errors.
• Rust behavior / test changes: pnpm test:coverage:rust (requires cargo-tarpaulin, or rely on CI)
  • Not run locally; this draft relies on CI coverage evidence before being marked ready. Rust behavior was verified with pnpm test:pr, targeted cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --lib core::built_in_tools::computer -- --nocapture, and cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --test computer_commands -- --nocapture.
• Desktop startup / window / search / popup / settings / E2E harness / workflow changes: pnpm test:e2e
  • Not run; this change adds built-in computer-use tooling and native command/runtime paths, not the desktop startup/window/search/popup/settings E2E surfaces.
• Additional targeted checks:
  • git diff --check passed.
  • pnpm exec prettier --check ... passed for touched TS/docs tests.
  • pnpm --filter @touchai/desktop type:check passed.
  • pnpm --filter @touchai/desktop test:typecheck passed.
  • pnpm --filter @touchai/desktop lint:check passed with 20 existing warnings and 0 errors.
  • pnpm exec vitest run --configLoader runner tests/services/BuiltInToolService/tools/computer/index.test.ts tests/services/native-service.test.ts tests/services/AgentService/execution/tool-round-scheduling.test.ts --passWithNoTests passed: 3 files, 64 tests.
  • cargo fmt --manifest-path apps/desktop/src-tauri/Cargo.toml --all -- --check passed.
  • cargo check --manifest-path apps/desktop/src-tauri/Cargo.toml --lib passed with existing try_lock_mutex warnings.

Did you follow TDD (test-first) for feature and fix work? Strongly recommended. See docs/testing/testing.md.

pnpm test:pr
pnpm test:coverage:rust
pnpm test:e2e

Regression tests were added before fixing the late review findings for tree-first defaults, background scope-only rejection, observed native target gating, routeHint background-policy bypass, and post-action observation contract.

Risk notes

• AgentService, runtime, MCP, or schema impact: AgentService tool-round scheduling changes built-in tool execution order when builtin__computer_act is present; NativeService and Rust command contracts gain computer-use request/response types; Rust runtime owns desktop action policy and receipts.
• database baseline or migration impact: runtime seed adds the three computer-use tools disabled by default with high risk level and conservative config.
• release or packaging impact: no bundled Python, Cua runtime, sandbox, OCR/VLM model weights, or additional packaged binaries are introduced.

Screenshots or recordings

No UI surface changed.

Checklist

• The PR title follows Conventional Commits and is valid for squash merge.
• This PR is either ready for review or explicitly marked as a Draft PR.
• I did not use [WIP] or similar title prefixes.
• If AI materially assisted this PR, I disclosed the tools and scope and I personally reviewed every affected change.
• I can explain the why, what, and how of this change without relying on an AI tool.
• If this touches AgentService, runtime, MCP, or schema boundaries, there is an accepted RFC.
• If this changes architecture or adds a new cross-boundary abstraction, there is an accepted RFC.
• I ran pnpm test:pr for this code PR, or this is a docs-only change.
• If I changed Rust behavior or tests, I reviewed pnpm test:coverage:rust or relied on CI coverage evidence.
• If I changed desktop startup/window/search/popup/settings/E2E paths, I ran pnpm test:e2e locally or documented why CI is the first valid proof.
• I added tests or explained why tests are not appropriate.
• I updated docs when behavior changed.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 20169d38-cb85-4c89-9729-5f7b2992e15c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-111-native-computer-use

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}