Add an Agent Integration Center with guided setup, end-to-end verification, instructions, and activity inspection

[ekalb81]

Outcome

Make Odometer's agent integration an inspectable product experience: humans can install, configure, verify, understand, and remove it, while agents can reliably discover when and how to use Odometer's analytics.

This is a P0 release gate for the first MCP delivery in #47. A configuration file or running process alone must never be presented as proof that the integration works.

Relationships

• Parent roadmap: Roadmap: close competitive analytics gaps on shared local-first foundations #36
• Companion implementation: Expose shared analytics through read-only CLI, local API, MCP, and statusline adapters #47
• Phase: 3 — controlled expansion
• Dependency: consume the shared read-only query/adapter contract from Expose shared analytics through read-only CLI, local API, MCP, and statusline adapters #47; do not create a second analytics implementation

Problem

MCP integrations are commonly added with no trustworthy answer to basic questions:

• Is the server installed?
• Did the correct client configuration load?
• Can the client launch and initialize it?
• Are the expected tools exposed?
• Is Odometer's data current and queryable?
• Has a real agent successfully used it?
• What should the agent be told, and how should it choose the right tool?
• If it failed, what exactly failed and how can the user safely fix it?

Odometer should make each state visible and testable instead of collapsing them into a single "connected" badge.

Integration Center

Add a human-facing integration surface with a card for each supported client (initially Codex, Claude Code, and generic MCP).

Each card must distinguish these states:

1. Not installed — no compatible client executable is discoverable
2. Installed — executable and version are verified
3. Configured — a valid Odometer entry exists at the expected user/project scope
4. Launchable — the client-equivalent server command starts successfully
5. Connected — MCP initialization/handshake succeeds
6. Tools exposed — tools/list returns the expected names and schemas
7. Data ready — a real odometer_status canary returns a current usable snapshot
8. Actively used — a successful call from the real client has been observed

Only the canary-backed states may produce a green Working result.

Human actions

Provide:

• Install or repair
• Choose user or project scope
• Preview the exact configuration change before applying it
• Apply a reversible configuration change while preserving unrelated settings
• Test now
• Copy the equivalent manual command
• Open the relevant configuration location
• Copy tailored agent instructions
• Explain when a client restart or fresh task is required
• Start a fresh-task verification flow
• Remove the integration or restore a timestamped backup
• Show the last successful connection, canary, real use, and bounded error

Automatic setup must not place secrets in process arguments or logs.

Agent front door

Expose a compact, low-cost odometer_status tool as the reliable first call when freshness or capabilities are unknown. It should return:

• server and schema version
• snapshot age and generation time
• scan and durable-ledger readiness
• available harnesses and date coverage
• session count
• pricing, credit, and quota coverage
• missing or fallback-priced models
• supported dimensions and filters
• recommended next calls
• warnings and known limitations

In particular, status/results must distinguish observed token/credit data, estimated API cost, and authoritative subscription/quota data. Never imply that subscription usage is available when no authoritative source exists.

Tool discoverability and guidance

Every MCP tool description should include:

• Use when
• Do not use when
• Interpretation
• Example requests

Results should suggest the next useful call where appropriate. Do not rely only on MCP server-level instructions because client support and instruction handling vary.

Generate copyable instruction blocks for:

• generic MCP clients
• Codex
• Claude Code
• an optional Odometer skill/plugin package

The generated instructions should tell an agent to:

• use Odometer for session tokens, credits, estimated API cost, subagent activity, tool-use patterns, and optimization opportunities
• call odometer_status when freshness or capability is uncertain
• search before fetching details when it does not have a session ID
• treat cohort comparisons and optimization findings as observational
• surface fallback pricing, incomplete coverage, and unavailable subscription quota explicitly

Include example user prompts and their expected tool-call sequence so humans can test a fresh task without guessing.

Privacy-safe activity inspection

Record enough local integration telemetry to prove and troubleshoot real use:

• client name and version
• tool name
• timestamp and duration
• success/failure
• bounded error code
• returned row count and byte count
• snapshot/schema version
• fallback-pricing and coverage flags

Do not record prompts, tool arguments, session IDs, local paths, result contents, credentials, or other sensitive payload data.

The UI should show recent activity and eventually identify configured-but-unused or consistently inefficient call patterns.

Stable diagnostics

Use the same diagnostic codes and remediation text across MCP, CLI, and UI:

• integration_not_configured
• server_launch_failed
• protocol_version_mismatch
• tool_catalog_mismatch
• ledger_not_ready
• scan_in_progress
• session_not_found
• pricing_incomplete
• query_too_broad
• snapshot_expired

Each diagnostic must state:

1. what failed
2. the bounded evidence used to determine it
3. the next safe action

Architecture constraints

• Reuse Expose shared analytics through read-only CLI, local API, MCP, and statusline adapters #47's shared typed read-only query service and adapters for the canary and diagnostics.
• Keep filesystem/config access and sensitive inspection in Rust; expose typed Tauri IPC to the Svelte UI.
• Keep the local API loopback-only and token-protected per Expose shared analytics through read-only CLI, local API, MCP, and statusline adapters #47.
• Make setup changes minimal, previewable, reversible, and scoped.
• Preserve unrelated client configuration.
• Keep MCP resources read-only and avoid returning raw session JSONL or prompt content.
• Do not implement a separate analytics path in the Integration Center.

Acceptance criteria

• A user can connect supported Codex and Claude Code installations without manually editing configuration.
• Installed, configured, launchable, connected, tool-catalog, data-ready, and actively-used states are reported separately.
• Test now performs initialization, tools/list, and a real odometer_status canary.
• Restart/fresh-task requirements are detected, explained, and verifiable.
• An uninstructed compatible agent can select the right tools from their descriptions.
• The copied Codex and Claude instruction blocks produce the documented tool sequence in fresh-task tests.
• Recent activity proves that a real client made a successful call without retaining sensitive payloads.
• Setup/removal is reversible and preserves unrelated configuration.
• Failures are covered for missing binaries, invalid scope, bad working directory, stale client sessions, protocol/catalog version mismatch, unavailable data, and incomplete pricing.
• UI, MCP, and CLI diagnostics use the same stable codes and remediation contract.
• Documentation includes setup, verification, example prompts, expected results, privacy boundaries, limitations, and removal.
• This ships with—not after—the first MCP release from Expose shared analytics through read-only CLI, local API, MCP, and statusline adapters #47.

Out of scope

• Mutating or deleting session data through an agent integration
• Exposing raw prompts, tool arguments, session paths, or result payloads
• Internet-facing or remotely reachable analytics endpoints
• Claiming causal optimization results from observational telemetry

[ekalb81]

Draft PR #86 is adjacent setup hardening for this work: it makes composable receipt-hook installation and observed receipt status reliable across existing Codex/Claude user configuration. It does not implement the subscription-usage semantics requested here and does not close this issue; #57 still depends on the evidence/semantics work tracked in #47.