feat(ollama): route ACP agents to local Ollama models via env injection#626
Draft
ChristianLuciani wants to merge 10 commits into
Draft
feat(ollama): route ACP agents to local Ollama models via env injection#626ChristianLuciani wants to merge 10 commits into
ChristianLuciani wants to merge 10 commits into
Conversation
Adds support for Ollama Launch (v0.15+), allowing AionCore users to run ACP agents with Ollama-hosted models via `ollama launch <agent>`, eliminating the need for API keys or provider configuration. Supported agents: claude, opencode, codex, copilot, pi, hermes, droid, qwen. Implementation: - Add OLLAMA_LAUNCH_MAP constant and helper functions to aionui-common - Create ollama module in aionui-ai-agent for runtime PATH detection - Add ollama_compatible computed field to AgentMetadata for UI filtering - Add use_ollama boolean flag to AcpBuildExtra for per-session control - Modify ACP factory to delegate to `ollama launch <agent>` when enabled - Graceful fallback to native launch preserves existing behaviour Testing: 619 existing + 6 new Ollama tests pass (625 total), cargo clippy -- -D warnings clean, cargo fmt clean, backwards compatible.
Ollama Launch requires an explicit --model when running without a TTY
('model selection requires an interactive terminal'). The previous
implementation ran 'ollama launch <agent>' with no --model, which
would fail in AionCore's headless child-process environment.
Changes:
- Add ollama_model: Option<String> to AcpBuildExtra
- Pass --model <ollama_model> -y when ollama_model is set
- Fall back to native launch with a WARN when use_ollama=true
but ollama_model is missing
- Add 4 deserialization tests for the new field
- Ensure struct-literal tests include the new field
Replace #[serde(default)] on use_ollama with an explicit custom Deserialize implementation. The Raw helper struct uses Option<bool> for use_ollama and maps None → false, ensuring that the presence of ollama_model alone never toggles use_ollama. This avoids potential serde version differences across CI environments where the derived default behavior may vary.
18 tasks
Contributor
Author
|
The frontend companion for this is now up as a draft: iOfficeAI/AionUi#3602 — it adds the Guid-page Ollama Launch model selector gated on |
…rapping ollama launch QA on the frontend companion (AionUi draft PR) surfaced that spawning 'ollama launch <agent> --model <model> -y' as the ACP child process always fails with 'Initialize handshake timed out after 30s'. Root cause, verified empirically with a PATH shim on Ollama 0.32.0: 'ollama launch claude' resolves the claude binary on PATH and execs the agent's *interactive TUI* with provider env vars applied (and forwards --model to it). It is not an ACP server wrapper, so a headless spawn never answers the ACP initialize request on stdio. Fix: keep the agent's native ACP command (bridge) and inject the same environment that ollama launch injects, captured from 0.32.0: ANTHROPIC_BASE_URL=http://127.0.0.1:11434 ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_API_KEY= (cleared) ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU}_MODEL=<model> CLAUDE_CODE_SUBAGENT_MODEL=<model> Verified headless: the claude ACP bridge answers initialize immediately with this env applied. Scope: OLLAMA_LAUNCH_MAP is replaced by OLLAMA_COMPATIBLE_BACKENDS, restricted to 'claude' — the only backend whose env mapping is verified end-to-end. Other agents (codex, qwen, ...) need their own verified mappings (some use config files, not env) and can be added in follow-ups. A unit test enforces that every backend advertised as ollama_compatible has an env mapping, so the silent-fallback branch is unreachable.
The prompt turn failed with JSON-RPC -32603 (errorKind: model_not_found) whenever the user's ~/.claude settings carried a persisted settings.model: the claude ACP bridge resolves the session model with priority ANTHROPIC_MODEL > settings.model > default, and build_ollama_env set no ANTHROPIC_MODEL, so a provider model that does not exist on Ollama leaked into the session (verified by driving @agentclientprotocol/claude-agent-acp 0.58.1 headless over stdio). ollama launch itself does not set ANTHROPIC_MODEL because it runs the interactive TUI where users can pick a model; headless ACP has no picker, so the selected Ollama model must be pinned explicitly. Also add the six telemetry/nonessential-traffic suppression variables the real ollama launch claude injects (captured from Ollama 0.32.1 via a PATH shim) so the agent does not call Anthropic endpoints with the placeholder credentials. Verified end-to-end: aioncore --local conversation with use_ollama=true and ollama_model=qwen2.5-coder:7b returns a model response through the full ACP stack.
ollama launch qwen configures qwen-code with pure environment variables (captured from Ollama 0.32.1 via a PATH shim): OPENAI_API_KEY=ollama, OPENAI_BASE_URL=http://127.0.0.1:11434/v1 and OPENAI_MODEL=<model>, plus --model/--auth-type openai CLI flags that are redundant headless — with OPENAI_API_KEY set, qwen --acp resolves the openai auth path and OPENAI_MODEL pins the session model (verified end-to-end against qwen-code 0.19.10 with a clean HOME, and through aioncore --local where the prompt turn completed against a local Ollama model). The remaining ollama launch integrations were probed the same way and cannot be driven by environment alone, so they stay excluded and are documented in the OLLAMA_COMPATIBLE_BACKENDS doc comment: codex receives -c model_providers.* CLI overrides, kimi a --config <json> flag its acp subcommand does not accept, and pi/droid get no injection at all (their launchers rewrite user config files interactively).
ChristianLuciani
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lets compatible ACP agents run against a local Ollama server without provider API keys, by injecting the provider environment variables that
ollama launch <agent>injects — directly into the agent's native ACP command.Supported agents:
claude,qwen(each with an empirically verified env mapping).Related: iOfficeAI/AionUi#2384 (reusing locally configured models from ACP agents).
Why env injection instead of wrapping
ollama launchollama launch <agent>starts the agent's interactive TUI (verified with a PATH shim on Ollama 0.32.1): a TUI spawned without a TTY never answers the ACPinitializerequest on stdio, so the handshake times out. Injecting the same environment into the agent's native ACP command keeps the ACP transport intact while routing model calls to Ollama.Implementation
aionui-common:OLLAMA_COMPATIBLE_BACKENDS(claude,qwen) +is_ollama_supported_agent(), andOLLAMA_DEFAULT_BASE_URL.aionui-ai-agent: newollamamodule withbuild_ollama_env(backend, model)— the per-backend env mappings, captured empirically from Ollama 0.32.1:ANTHROPIC_BASE_URL→ local Ollama,ANTHROPIC_AUTH_TOKEN=ollama,ANTHROPIC_API_KEYcleared,ANTHROPIC_MODEL/ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU}_MODEL/CLAUDE_CODE_SUBAGENT_MODELpinned to the chosen model, plus the same telemetry-suppression varsollama launchsets.ANTHROPIC_MODELpinning is required for headless use: without it the ACP bridge falls back to~/.claude/settings.jsonand fails the turn with-32603 model_not_found(verified against@agentclientprotocol/claude-agent-acp0.58.1).OPENAI_API_KEY=ollama,OPENAI_BASE_URL=http://127.0.0.1:11434/v1,OPENAI_MODELpinned (verified end-to-end against qwen-code 0.19.10).aionui-api-types:AcpBuildExtragains opt-inuse_ollama: boolandollama_model: Option<String>(both#[serde(default)], derivedDeserialize);AgentMetadata/AgentManagementRowgain a computedollama_compatibleflag (#[serde(skip)]) for UI gating.use_ollama && ollama_compatible && ollama_model.is_some(), the resolved command spec gets the Ollama env appended last (so it overrides catalog/user vars). Any other combination falls back to the native launch with a warning — supplyingollama_modelalone never toggles the route.Why only claude and qwen
The other
ollama launchintegrations cannot be driven by environment alone (verified with a PATH shim on Ollama 0.32.1): codex receives-c model_providers.*CLI overrides, kimi a--config <json>flag itsacpsubcommand does not accept, and pi/droid get no injection at all (their launchers rewrite user config files interactively). Documented inaionui-common/src/constants.rs; the list can grow together with a verified mapping in theollamamodule (enforced by a unit test).Testing
No breaking changes —
ollama_compatibleis#[serde(skip)],use_ollamadefaults tofalse, and the whole path is opt-in per session via theextrapayload.Frontend companion (separate PR)
iOfficeAI/AionUi#3602 surfaces local Ollama models in the model selector for compatible ACP agents and sends
use_ollama/ollama_modelin the sessionextrapayload. This backend PR is complete and functional without it — callers can opt in via the API directly.Thanks for taking the time to review — happy to adjust anything to better fit the project's direction.