✨ Add structured response events to ResponseProtocol

[MFA-X-AI]

Step 2 of the AG-UI plan discussed (#38 is step 1). The goal is an AGUIAdapter that renders any xaibo agent in any AG-UI frontend with tool calls, steps, and reasoning visible. For that, adapters need something richer than final text to translate. The current ResponseProtocol carries text and file attachments only — everything else an agent does is invisible at the response seam.

Per the discussion, this extends the existing ResponseProtocol/Response rather than introducing a parallel streaming protocol.

If I Got It Correctly...

In practice, xaibo users write their own orchestrator modules. The ones we have now are starting templates, eg the SimpleOrchestrator and ReActOrchestrator. ResponseProtocol is the one channel every orchestrator, shipped or custom, has to the frontend, and today it only carries text. So any orchestrator that wants to surface its internals has to invent an ad-hoc encoding inside the response text. The ReActOrchestrator template demonstrates exactly this workaround (🛠️ **EXECUTING TOOL:** ... emoji prose), and every downstream project would have to rebuild some variant of it, each unparseable by any standard frontend.

This PR gives the seam a typed vocabulary instead, so custom orchestrators emit structured events once and every adapter can translate them uniformly.

Changes

• core/models/response.py - a typed event union. The semantics are aligned with AG-UI's taxonomy so the future adapter mapping stays thin, but the classes are xaibo's own rather than imports from the ag-ui-protocol SDK. Since the AG-UI is pre-1.0 and its spec still changes, keeping its types out of the core makes sense. Any future spec changes only needs to touch the adapter.
  • ToolCallEvent(id, name, arguments) / ToolResultEvent(id, name, success, result?, error?)
  • UsageEvent - subclasses the existing LLMUsage, so future usage fields (cached tokens, reasoning tokens) flow through without touching this model.
  • Response gains an events: List[ResponseEvent] field.
  • Only event types the reference implementation actually produces are included; reasoning/step events land when a template that emits them does.
• core/protocols/response.py - one new method: respond_event(event). Single entry point, so adding event types later doesn't grow the protocol surface a custom orchestrator programs against.
• primitives/modules/response.py - ResponseHandler accumulates events into the Response; respond(...) merges them. Events are turn-scoped: get_response() hands them off and resets, so a reused agent instance doesn't replay the previous turn's usage/tool events (text/attachment accumulation semantics are unchanged).
• orchestrator/simple_tool_orchestrator.py - updated as the reference for how an orchestrator uses the new surface: UsageEvent after each LLM call, ToolCallEvent before each tool execution, ToolResultEvent after (success, failure, and exception paths converge on a single emission site). This is the template users copy when writing their own orchestrators, and it doubles as the real producer the API is tested against. Tool results in the event are coerced with a json.dumps(..., default=repr) round-trip - the same treatment tool results already get on the conversation path in both orchestrator templates and that event payloads get in the debug trace listener (ui.py), so a raw copy here would be the inconsistency.
• Adapters (openai.py, openai_responses.py, livekit llm.py) - their inline response handlers implement respond_event as a no-op: they acknowledge events but don't forward them, since their wire formats have no lane for agent internals. Because the call still happens, it crosses the observability proxy - tool calls and usage stay visible in the debug trace when serving through these adapters. The AG-UI adapter will be the first consumer that actually forwards them.

Other technical details

• Whole-event granularity only - no deltas. Deliberate: respond_event calls go through the observability proxy like every module call, so delta-shaped methods would put per-token events into the event log (the log-volume concern). Whole events are one log entry per meaningful action. Token-level text deltas are a separate, later decision (needs the delta-aware event listener discussed in the standup).
• Events are pydantic models with a type discriminator. Serializable for free, both in the event log and for future SSE adapters.
• A side benefit of routing through the proxy: every respond_event call lands in the observability log with full envelope, so the debug UI sees tool calls and usage per agent run without any listener changes, for custom orchestrators too, automatically.
• The ReActOrchestrator template is intentionally left as-is: its emoji-prose output is user-visible behavior that people may have copied, and converting it to events could be updated in another PR once this shape is agreed.

Compatibility

• Agent YAMLs: no change. Response/ResponseHandler changes are additive.
• User-written orchestrators: unaffected until they opt in - nothing requires emitting events.
• Existing ResponseProtocol implementations: unaffected. Events are an optional capability (see below) — a handler that predates respond_event keeps working unchanged; it just doesn't receive events.

---

Update 1: Events are an optional capability (per the discussion)

Response handlers in xaibo support what they support - partial implementations are the norm (the adapters' inline handlers implement only respond_text + get_response, and nothing forces more). Per the discussion, respond_event should follow the same rule rather than becoming a hard requirement: the orchestrator template emits through a small guard (_emit_event) that checks the handler for respond_event and skips if absent. A handler that wants events extends to support them.

One behavior note: a skipped emission never crosses the observability proxy, so it can't appear in the debug event trace - the guard logs a logger.debug line instead so dropped events stay diagnosable. Handlers that implement respond_event, even as a no-op like the in-repo adapters, keep full trace visibility since the call happens and is recorded before being ignored.

A shared default base class (implementers get no-op event handling without writing it) is a natural follow-up once the AG-UI adapter exists as the first real event consumer to design it against.

Verification

uv run pytest tests/agents/test_response_events.py -v   # 8 new tests
uv run pytest --continue-on-collection-errors -q        # 145 passed, 18 skipped

New tests cover: ResponseHandler accumulation, merge, and per-turn event reset; a full agent run (MockLLM + stub tool provider through the real Exchange) asserting the event sequence [usage, tool_call, tool_result, usage] with payloads; failed-tool and exception paths; JSON-serializability of events carrying non-serializable tool results (datetime); a legacy response handler without respond_event running uncrashed through the event-emitting orchestrator.