feat: dedicated realtime-API protocol types for /v1/realtime

[GuanLuo]

Overview:

Slice 5/N of the bidirectional streaming-input feature — replaces the chat-completion placeholder shape that #9079 (1/N) shipped on /v1/realtime with the OpenAI Realtime API event surface. Clients now exchange real RealtimeClientEvent / RealtimeServerEvent JSON frames over the WebSocket, and the handler synthesizes a spec-compliant session.created server event on connect so a stock OpenAI Realtime client can complete its handshake.

The change is type-swap-shaped on purpose: engine-side semantics (VAD, audio buffering, transcription, response generation) belong to a downstream slice and are intentionally not in this PR. The mock EchoBidirectionalEngine is rewritten to demonstrate the round-trip without taking on those responsibilities.

async-openai 0.34's realtime-types feature already covers the full event surface (verified against RealtimeClientEvent / RealtimeServerEvent and confirmed against OpenAI's published event list). Per lib/protocols/CLAUDE.md's ownership rubric, no Dynamo-side overrides are needed today — the new dynamo_protocols::types::realtime module is a pure namespaced re-export.

Details:

Protocols crate:

• lib/protocols/Cargo.toml — + "realtime-types" to the async-openai features list.
• lib/protocols/src/types/realtime.rs (new) — glob re-export of async_openai::types::realtime::* behind a banner comment that names the ownership rubric.
• lib/protocols/src/types/mod.rs — pub mod realtime;. Namespaced (not flattened into types::*) because realtime exposes Session, ContentPart, Response, etc. that overlap with chat / responses surface area.

LLM crate — typedef:

• lib/llm/src/types.rs::generic::realtime::RealtimeBidirectionalEngine — repointed from BidirectionalStreamingEngine<NvCreateChatCompletionRequest, Annotated<NvCreateChatCompletionStreamResponse>> to BidirectionalStreamingEngine<RealtimeClientEvent, Annotated<RealtimeServerEvent>>. Drops the TODO (#9175) doc.

LLM crate — handler (/v1/realtime):

• Inbound JSON deserialization swapped to RealtimeClientEvent; mpsc channel typed accordingly.
• Merge-blocker resolution: synthesizes RealtimeServerEvent::SessionCreated { event_id: <uuid>, session: Session::RealtimeSession(Box::default()) } as the first wire frame on connect, before any engine event flows. The OpenAI spec mandates this; doing it handler-side (not engine-side) lets this slice ship without waiting on engine implementation.
• Outbound peels the Annotated carrier off — clients receive bare RealtimeServerEvent JSON, not the Dynamo-internal {"data": {...}, "id": "1"} envelope (this was a wire-shape bug that the new integration tests caught against the type-swap commit).
• Annotated::error is mapped to a synthesized RealtimeServerEvent::Error so engine errors stay visible on the wire as a spec-shaped error event.

LLM crate — mock engine:

• EchoBidirectionalEngine rewritten over the new types:
  • RealtimeClientEvent::SessionUpdate(req) → RealtimeServerEvent::SessionUpdated echoing req.session back.
  • All other variants → RealtimeServerEvent::Error with code = "echo_engine_unsupported" and a message naming the offending variant.
• client_event_variant_name helper keeps the rejection-message wire-tag stable across upstream additions to RealtimeClientEvent.

Tests:

• Two unit tests in engines::tests (session.update round-trip; unknown-event → error).
• Four integration tests in lib/llm/tests/http_websocket.rs:
  • realtime_websocket_emits_session_created_on_connect — merge-blocker assertion.
  • realtime_websocket_session_update_echoes_session_updated — end-to-end round-trip.
  • realtime_websocket_emits_close_after_client_close — fixture body swapped from chat JSON to session.update.
  • realtime_websocket_rejects_binary_frame — drains the on-connect session.created so the binary-reject is what triggers the close.
• The DYN_TOKEN_ECHO_DELAY_MS test setup is gone; the new echo engine doesn't sleep.

Server-frame coverage scope (intentional):

The OpenAI Realtime server contract emits ~43 frames across session, audio buffer, conversation, transcription, response, function-calling, MCP, and rate-limit categories. Only the connection-lifecycle group (session.created + session.updated + error) is in scope here. Everything else is engine-side and correctly deferred to the engine-implementation slice that comes after.

Where should the reviewer start?

Reading order, smallest mental model first:

1. lib/llm/tests/http_websocket.rs — what does this PR enable? The four tests show the on-connect handshake, the round-trip, the close path, and the binary-reject path.
2. lib/protocols/src/types/realtime.rs + lib/protocols/src/types/mod.rs — the new module. 10 lines; the question to satisfy yourself is should we own anything? Per lib/protocols/CLAUDE.md, the answer is no.
3. lib/llm/src/types.rs — typedef rename. Zero behavior, but the name shift signals the rest of the diff's intent.
4. lib/llm/src/engines.rs — EchoBidirectionalEngine. Pins the engine contract the handler has to satisfy.
5. lib/llm/src/http/service/realtime.rs — the bridge. Both the inbound deserialization swap and the on-connect session.created synthesis live here.

To verify locally:

cargo test -p dynamo-llm --test http_websocket
cargo test -p dynamo-llm --lib engines::tests
cargo check --workspace --all-targets
cargo clippy --no-deps --all-targets -- -D warnings

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Closes feat: dedicated realtime-API protocol types for bidirectional streaming #9175 — dedicated realtime-API protocol types for bidirectional streaming.
• Built on feat: bidirectional streaming-input WebSocket frontend #9079 (1/N) — base of this PR. Will retarget to main once feat: bidirectional streaming-input WebSocket frontend #9079 squash-merges.
• Relates to feat: ModelManager accessor for bidirectional engines (2/N) #9174 (2/N) — ModelManager accessor for bidirectional engines; orthogonal to this PR.

Internal Linear tickets DIS-1928 (this) and DIS-1858 (base) mirror the above GitHub issues (cross-linked per `.ai/linear-ticket-refs.md`).

🤖 Generated with Claude Code