Skip to content

feat: support DeepSeek V4 Flash 0731 - #1364

Merged
raullenchai merged 4 commits into
mainfrom
agent/deepseek-v4-flash-0731
Jul 31, 2026
Merged

feat: support DeepSeek V4 Flash 0731#1364
raullenchai merged 4 commits into
mainfrom
agent/deepseek-v4-flash-0731

Conversation

@raullenchai

@raullenchai raullenchai commented Jul 31, 2026

Copy link
Copy Markdown
Owner

What changed

  • adds a deepseek-v4-flash-0731-mxfp4 alias for the 0731 MXFP4 MLX checkpoint
  • implements the checkpoint-specific prompt encoder and native DSML tool parser
  • translates the official checkpoint quantization paths to Rapid vendored-model paths
  • normalizes Hugging Face AddedToken metadata in the fallback tokenizer loader
  • prevents incompatible KV caches from retrying failed disk serialization every decode token
  • records the exact 156 GB checkpoint size, 192 GB memory floor, and parser coverage exemption

Why

This checkpoint intentionally ships a dedicated encoder and DSML wire format. Without explicit support, Rapid falls back to incompatible ChatML, misloads mixed MXFP4/MXFP8 tensor shapes, and cannot return structured tool calls.

Validation

  • official DeepSeek encoder fixtures 1 and 2 match byte-for-byte
  • focused DSV4/parser/checkpoint suite: 42 passed
  • parser coverage + streaming parity suite: 25 passed
  • complete local suite: 14,904 passed, 174 skipped; 13 environment/server-required failures and one parser coverage failure discovered, with the parser coverage failure fixed and revalidated
  • Ruff, type-check, diff check, wheel build, Python 3.10/3.11/3.12 CI, Apple Silicon CI, and PR validation
  • final wheel force-installed and run outside the repository
  • real 34-shard, 156 GB checkpoint loaded on a 256 GB Mac Studio
  • basic inference: HTTP 200, correct Chinese answer (法国的首都是巴黎)
  • long-prompt inference: HTTP 200 with 729 prompt tokens; failed KV serialization no longer hot-loops
  • public rapid-mlx share health check: ready=true, model_loaded=true

Known limitation

The community MXFP4 checkpoint answers normal chat correctly, but tool-use quality was unstable in local E2E trials: chat mode declined a requested call and thinking mode repeated thinking to the token limit. The official DSML encoding and Rapid parser are covered byte-for-byte/unit/streaming, but this PR does not claim that this particular quantization preserves reliable tool-selection quality.

@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

PR #1364 validation scorecard

Title: feat: support DeepSeek V4 Flash 0731
Author: raullenchai
Diff: 14 file(s), +659/-6 LOC, blast radius: high

Verdict: MERGE-SAFE

step status summary time
fetch PASS 14 files, +659/-6 LOC, blast=high 1.3s
test_plan_check PASS no checkbox-style test plan found 0.0s
cl_description_quality PASS title OK + body has rationale (2036 chars) 0.0s
supply_chain PASS no hooks touched, no suspicious patterns, deps clean 0.0s
test_env_check PASS installed trusted-pins (2 missing plugin(s) recovered) 4.2s
codex_review skip codex CLI not found on PATH (install: npm i -g @openai/codex) 0.0s
lint PASS clean (12 file(s)) 0.0s
targeted_tests PASS exit 2 (in 18 target file(s)) 5.1s
diff_coverage skip diff-cover not installed — advisory coverage unavailable 0.0s

@raullenchai
raullenchai marked this pull request as ready for review July 31, 2026 18:21
@raullenchai
raullenchai merged commit b65cddb into main Jul 31, 2026
15 of 16 checks passed
@raullenchai
raullenchai deleted the agent/deepseek-v4-flash-0731 branch July 31, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant