Per-provider attachment validation: registry overrides + inline file-MIME gating#130
Draft
adambalogh wants to merge 2 commits into
Draft
Per-provider attachment validation: registry overrides + inline file-MIME gating#130adambalogh wants to merge 2 commits into
adambalogh wants to merge 2 commits into
Conversation
Attaching a PDF (or image) to deepseek-v4-pro/-flash was forwarded to
ByteDance ModelArk and came back as a raw 400 ("Model do not support
image input"). The attachment gate in validate_attachments fails open
for models without LangChain profile data, which covers every ModelArk
model routed through the generic OpenAI-compatible client.
Add per-model attachment capability overrides (image_inputs /
pdf_inputs) to ModelConfig, overlay them on the LangChain profile in
get_model_capabilities, and mark the text-only DeepSeek V4 models as
supporting neither — so the gateway now rejects these requests up front
with its clean "does not support document/image attachments" 400.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011rvJNkR8AK28peRgs4bVRu
Extends attachment validation beyond the boolean image/PDF modality gates to what each provider's chat endpoint actually accepts inline (per provider docs, July 2026): - Per-provider file-part MIME allowlists in validate_attachments: Anthropic and Gemini accept PDFs (Gemini also text-family types); OpenAI is unrestricted (its file-inputs cover Office/CSV/text/code formats); xAI, ByteDance, Nous and Z.ai have no inline file input on the chat path, so file parts get a clean 400 instead of the provider's raw error. - Registry capability overrides for the remaining text-only models: Hermes 4 405B/70B (Llama-3.1 text base) and GLM-5.2 (Z.ai restricts multimodal parts to its V-series models) now reject image and document attachments up front, like DeepSeek V4. - Seed models keep failing open for images: ModelArk documents image input on the chat API for seed-1.6/1.8/2.0-lite. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_011rvJNkR8AK28peRgs4bVRu
adambalogh
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Attaching a PDF to
deepseek-v4-proforwarded the file to ByteDance ModelArk, which rejected it with a raw 400 (Model do not support image input). The attachment gate fails open for models without LangChain profile data — which is every model routed through the generic OpenAI-compatible client (ByteDance, Nous, Z.ai) — and even for profiled models it only gated modalities (image/PDF), not which file types a provider's chat endpoint actually accepts inline.Changes
Registry capability overrides (
model_registry.py): optionalimage_inputs/pdf_inputsonModelConfig(None= defer to the LangChain profile, as before). Overlaid on the profile inget_model_capabilities. Set toFalseon the models confirmed text-only by their providers' docs:Seed models (1.6/1.8/2.0-lite) intentionally keep failing open for images — ModelArk documents image input on the Chat API for them.
Per-provider file-MIME allowlists (
llm_backend.py):validate_attachmentsnow also checks each inlinefilepart's MIME type against what the provider's chat endpoint documents (July 2026):application/pdfonly (base64 document blocks don't take Office/text formats — Anthropic's docs require converting those to text or PDF first)text/plain,text/csv,text/markdown, …) which Gemini text-extractsAll rejections raise
AttachmentValidationError→ the existing clean 400 ("Model 'grok-4.5' does not support application/pdf file attachments.") instead of a provider's raw error.Tests
New: provider MIME allowlist (docx→GPT passes, docx→Claude/Gemini rejected, csv→Gemini passes, PDF→Grok/Seed rejected), text-only overrides for DeepSeek/Hermes/GLM (both modalities), override-beats-profile, Seed image fail-open. Full suite: 195 passed, 3 skipped;
make lintclean.Notes
Companion chat-app PR (OpenGradient/chat-app#649) drives the same matrix in the UI (per-model accept lists, mismatch gating, and client-side text-file folding — text files reach the gateway as plain message text, not file parts).
🤖 Generated with Claude Code
https://claude.ai/code/session_011rvJNkR8AK28peRgs4bVRu