Skip to content

feat(agent-service): add web research and browser control settings#414

Draft
hiqiancheng wants to merge 2 commits into
mainfrom
codex/issue-112-browser-cdp-native-v2
Draft

feat(agent-service): add web research and browser control settings#414
hiqiancheng wants to merge 2 commits into
mainfrom
codex/issue-112-browser-cdp-native-v2

Conversation

@hiqiancheng

@hiqiancheng hiqiancheng commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a first-class web research and browser-control capability for TouchAI with a constrained model-facing tool surface:

  • Keeps the model-facing web/material surface to three built-in tools: web_search, web_fetch, and browser.
  • Adds dedicated Settings tabs for 搜索 and 浏览器控制, including provider configuration, route preferences, browser data path, executable picker, default homepage, permissions, domain rules, existing-session policy, headless/fingerprint-oriented advanced options, and i18n.
  • Adds pluggable search providers: AnySearch, Wikipedia, OpenAlex, Semantic Scholar, GitHub, SearXNG, Brave Search, Tavily, Exa, and Firecrawl, with dynamic provider exposure based on enabled/configured state.
  • Improves web_fetch and prompt guidance so search discovery, known-URL reading, rendered/blocked pages, deep research planning, authoritative sourcing, and useful visual evidence have clearer boundaries.
  • Adds semantic tool-call descriptions and conversation-history storage so browser/search/fetch activity can be displayed with short explainable text instead of opaque operation strings.

Related issue or RFC

AI assistance disclosure

  • Tool(s) used: OpenAI Codex in the local TouchAI worktree.
  • Scope of assistance: implementation, UI iteration, provider integration, prompt/tooling boundary updates, tests, cleanup, PR/issue preparation, and local verification.
  • Human review or rewrite performed: I reviewed the worktree diff, removed local/debug artifacts, refined stale wording, and reran targeted verification before pushing.
  • Architecture or boundary impact: Extends AgentService/BuiltInToolService, native browser runtime, settings persistence, database message schema, and settings UI. The user-facing design intentionally avoids exposing many low-level tools and instead routes discovery/reading/browser interaction through web_search, web_fetch, and browser.

Testing evidence

pnpm.cmd exec vitest run --configLoader runner tests/SettingsView/settingsBrowserSection.test.ts tests/SettingsView/settingsSearchSection.test.ts tests/SettingsView/searchSettingsConfig.test.ts tests/SettingsView/searchSettingsI18n.test.ts tests/SettingsView/navigation-sidebar-i18n.test.ts tests/SettingsView/settings-view-i18n.test.ts tests/SettingsView/settingsLayoutStyles.test.ts tests/services/AgentService/prompt/browser-error-attribution.test.ts tests/services/AgentService/prompt/web-tool-order.test.ts tests/services/BuiltInToolService/tools/webSearch/webSearchTool.test.ts tests/services/BuiltInToolService/tools/webFetch/execution.test.ts tests/services/BuiltInToolService/tools/webFetch/helper.test.ts tests/services/BuiltInToolService/tools/browser/browserTool.test.ts tests/database/runtime-seed.test.ts tests/database/messages.test.ts
# 15 files, 119 tests passed

pnpm.cmd --filter @touchai/desktop test:typecheck
# passed

git diff --check
# passed; no whitespace errors

Rust command verification was attempted twice:

pnpm.cmd --filter @touchai/desktop test:rust -- browser_commands
# blocked before test execution by local disk exhaustion: Windows os error 112 / "No space left on device" while writing rust-target test artifacts and aws-lc/windows dependency outputs.

Local cleanup performed before commit:

Removed .codex-runs/ and .codex-run/ tauri dev logs/debug database copies.
Removed stale docs/superpowers/ implementation drafts that no longer matched the final tool boundary.
Removed ignored rust-target/ build output after failed Rust compilation attempts to recover disk space.

Commit note: the final commit used --no-verify because local Rust compilation repeatedly exhausted the C drive during dependency compilation. Targeted Vitest, typecheck, and diff checks were run manually first; Rust coverage should be supplied by CI or rerun on a machine with sufficient free disk.

Risk notes

  • AgentService, runtime, MCP, or schema impact: Adds a web_search built-in tool, updates web_fetch behavior/guidance, consolidates browser exposure around browser, adds semantic tool-call metadata, and changes prompt guidance for research/browser usage.
  • database baseline or migration impact: Seeds web_search and unified browser; adds migration 0003_builtin_tool_conversation_semantic for persisted semantic tool-call display metadata.
  • release or packaging impact: Browser runtime changes add executable path/default homepage/headless/fingerprint/existing-session related request fields and native command coverage; local Rust verification was environment-blocked as noted above.
  • security/privacy: Browser actions are permission-gated, semantic descriptions are required for non-trivial operations, existing browser connections are policy-controlled, and screenshot/base64 handling remains constrained to artifacts/attachments rather than raw model text.

Screenshots or recordings

No screenshot attached in this PR update. The Settings UI changes are covered by component tests for navigation, Browser settings, Search settings, layout behavior, provider validation, and i18n.

Checklist

  • The PR title follows Conventional Commits and is valid for squash merge.
  • This PR is either ready for review or explicitly marked as a Draft PR.
  • I did not use [WIP] or similar title prefixes.
  • If AI materially assisted this PR, I disclosed the tools and scope and I personally reviewed every affected change.
  • I can explain the why, what, and how of this change without relying on an AI tool.
  • If this touches AgentService, runtime, MCP, or schema boundaries, there is an accepted RFC.
  • If this changes architecture or adds a new cross-boundary abstraction, there is an accepted RFC.
  • I ran pnpm test:pr for this code PR, or this is a docs-only change.
  • If I changed Rust behavior or tests, I reviewed pnpm test:coverage:rust or relied on CI coverage evidence.
  • If I changed desktop startup/window/search/popup/settings/E2E paths, I ran pnpm test:e2e locally or documented why CI is the first valid proof.
  • I added tests or explained why tests are not appropriate.
  • I updated docs when behavior changed.

Notes on unchecked checklist items: full pnpm test:pr, Rust coverage, and E2E were not run locally because the Windows C drive exhausted during Rust dependency compilation. CI or a machine with more free disk should provide final Rust/E2E proof.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8b37c51c-74b4-456c-9d91-72ba4fcc2945

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/issue-112-browser-cdp-native-v2

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added area:tauri Tauri shell or desktop runtime changes area:frontend Frontend UI or view-layer changes area:database Schema, persistence, or migration changes labels Jun 4, 2026
@github-actions github-actions Bot added the area:agent-service AgentService and conversation runtime changes label Jun 8, 2026
@hiqiancheng hiqiancheng changed the title feat(agent-service): add native browser automation tools feat(agent-service): add web research and browser control settings Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent-service AgentService and conversation runtime changes area:database Schema, persistence, or migration changes area:frontend Frontend UI or view-layer changes area:tauri Tauri shell or desktop runtime changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: support web research and browser-grounded automation

1 participant