Vision-expert subagents (task/task_batch) should be able to use tools

[lowlorenz]

Summary

The vision-expert subagents spawned by task / task_batch currently cannot use tools — each expert turn is a single text/vision completion. They should be given a real tool loop so an expert can self-direct (e.g. zoom into a sub-region it identifies) instead of depending on the orchestrator to guess the right crop in advance.

Current behavior

runExpertTurn (chronos/tools/expert-turn.ts:110) makes one complete() call:

const response = await complete(
  resolved.model,
  { systemPrompt: pageExpertPrompt, messages: [...session.messages, userMessage] },
  { apiKey: resolved.apiKey, headers: resolved.headers },
);

The context is just { systemPrompt, messages } — no tools field — so the expert receives one (optionally pre-cropped) page image plus a text prompt and returns text only. It cannot crop, zoom, read another page, or call anything back.

Cropping today is caller-driven, not expert-driven: the main agent picks a bbox up front via the task/task_batch params (tools/view-page.ts:44-52, tools/task-batch.ts:52), and cropImageToBase64 (utils/crop-image.ts:21) crops the image before it is sent. The expert can't request a different region during its turn. "Follow-ups" via task_id (tools/expert-registry.ts) just replay stored session.messages into another single complete() — conversational memory, not agency.

Why this matters

Dense primary sources (tables, marginalia, faint or damaged ink) often need the reader to zoom into a sub-region it only identifies after seeing the page. Right now the expert must answer from a single fixed view, or the orchestrator has to predict the correct crop blind. Letting the expert pull in detail on demand should improve extraction accuracy and reduce orchestrator round-trips.

Proposed change

• Pass Context.tools to the expert (pi-ai's Context already supports tools?: Tool[] — pi-ai/dist/types.d.ts:177) and turn the single complete() in runExpertTurn into an agentic loop: handle tool_use blocks → execute → append tool_result → re-complete() until the model stops calling tools.
• Candidate tools, reusing existing utilities:
  • view_region / zoom — crop a normalized bbox of the current page at higher resolution (wraps cropImageToBase64).
  • view_page — load another page from the same source (the expert shares the SourceContext).

Considerations

• Persistence/consistency: intermediate tool calls + results must be stored in session.messages and in the appended-turn log (appendExpertTurn) so task_id follow-ups and agent-restart resume stay coherent.
• Iteration/cost cap: bound the loop (max tool turns, cost ceiling) so a confused expert can't spin.
• Model capability: only enable for models that support tool use; keep a clean fallback to the current single-shot path otherwise.
• Keep the caller bbox as the expert's initial view; tools augment it, they don't replace it.
• Model selection should stay configurable / unpinned, consistent with Remove Gemini-specific focus: stop prompting for Gemini API key, stop defaulting subagents to Gemini; move model recommendations to docs #7.