feat: native image/audio/video content parts (LocalAI)

[localai-bot]

What

Adds native image / audio / video content parts to cogito, serialized by the LocalAI client, so agents can hand a model a voice note, image, or clip — not just text. go-openai's ChatMessagePart models only text/image_url; LocalAI's server already accepts image_url, audio_url, video_url, and input_audio — this closes the client-side gap.

How

Three commits, TDD:

1. fragment.go — a typed Multimedia (MediaKind{Image,Audio,Video} + Data()/Format(); plain Multimedia stays image, fully backward-compatible), a NativePart type, a transient Fragment.PendingNativeParts, and a NativePartsAware interface. AddMessage routes image parts to the existing image_url MultiContent path and audio/video parts to PendingNativeParts.
2. clients/localai_client.go — a mutex-guarded native-parts stash (SetPendingNativeParts, satisfying NativePartsAware); marshalRequest rebuilds the last message with input_audio/video_url parts when the stash is set (map-preserve, so every other request field and message is untouched); Ask sets its stash from the Fragment. Send-once (the result Fragment drops pending parts) so a multi-MB blob isn't re-uploaded every turn; the stash is read-not-cleared so stream retries resend.
3. tools.go — askWithStreaming and pickTool set the stash fresh from their Fragment via a NativePartsAware type-assert, covering the streaming response and tool-decision request paths leak-free (every request origin sets fresh from its Fragment, so no prior turn's audio leaks forward).

Design notes

• Scope: LocalAI-client + a small optional-interface hook. No LLM/StreamingLLM signature changes, GetMessages/OpenAIClient/Fragment.Messages untouched; non-LocalAI clients are unaffected (the type-assert simply misses).
• Images persist via MultiContent (unchanged, cheap); audio/video are send-once (large blobs).
• Wire forms: input_audio {format,data} (base64), video_url {url} (data URI), image_url {url}.

Testing

go test ./clients/ and the TestAddMessage* / TestExecuteToolsStreamingSerializesNativeAudio root tests pass; go build ./clients/ . and go vet clean. (The pre-existing fragment_e2e_test.go needs a live LLM and is not run here.)

A whole-branch review caught — and this branch fixes — a subtle cross-cutting bug: the usage-counting wrapper applied inside ExecuteTools embeds the LLM interface, which would have hidden SetPendingNativeParts from the seams' type-assertion and silently dropped native parts on the streaming path. Fixed with a forwarding method on the wrapper plus a regression test that drives the real wrapped/streaming seam (verified RED without the forwarder, GREEN with).

Consumes

Unblocks the nib multimodal-attachments work (nib's ContentPart implements TypedMultimedia).

🤖 Generated with Claude Code