Skip to content

feat: support offline DFlash-family training#704

Merged
jiapingW merged 4 commits into
sgl-project:mainfrom
maocheng23:agent/pr678-offline-draft-training
Jul 21, 2026
Merged

feat: support offline DFlash-family training#704
jiapingW merged 4 commits into
sgl-project:mainfrom
maocheng23:agent/pr678-offline-draft-training

Conversation

@maocheng23

Copy link
Copy Markdown
Collaborator

Motivation

#678 adds unified-runtime offline consumers, but the production feature preparer is still EAGLE3-specific: it requires exactly three layers and writes hidden_state/aux_hidden_state. DFlash and Domino consume hidden_states, while DSpark additionally needs target_last_hidden_states.

This change makes the producer schema come from the same typed algorithm registry as the consumer, so offline preparation and training cannot silently drift.

Modifications

  • add algorithm-owned offline capture layouts describing the SGLang hook, auxiliary/final-state mapping, and passthrough tensors
  • generalize scripts/prepare_hidden_states.py for strategy-aware draft configuration, arbitrary layer counts, and exact persisted schemas
  • support DFlash and Domino feature production, plus DSpark offline storage, normalization, collation, and runtime wiring
  • dispatch through set_dflash_layers_to_capture for the DFlash family and set_eagle3_layers_to_capture for EAGLE3
  • add offline recipes for Qwen3 DFlash, Domino, and DSpark, with capability and data-preparation documentation

D-PACE and DTA inherit DFlash's capture/storage contract. P-EAGLE is intentionally unchanged because its target-head/tokenizer preparation is a separate algorithm path.

Related Issues

Follow-up to #678.

Accuracy Test

  • exact-commit DSpark CUDA/FSDP gate on H200: passed two training steps with finite parameters
  • exact-commit two-H200 DFlash capture on pinned SGLang 0.5.14: passed on both ranks with five captured layers and the expected 5x auxiliary width

Benchmark & Profiling

Not applicable. This changes offline data preparation and provider wiring; it does not change kernels, model forward math, or serving performance.

Validation

  • pre-commit run --all-files — passed
  • focused provider/composition/capture/config/architecture suite — 79 passed
  • adjacent config and offline-runtime suite — 58 passed, 3 CUDA-only skips on macOS
  • DSpark H200 runtime gate — passed
  • pinned SGLang 0.5.14 two-H200 DFlash capture gate — passed

Stack

This draft is stacked on #678 at its current head, ffe081c8074eaca7a079889737f82cb6649569de. Until #678 merges, GitHub will display its commits in this draft too; afterward the diff reduces to this offline DFlash-family follow-up.

Checklist

  • Format the code with repository pre-commit hooks.
  • Add unit and runtime tests.
  • Update documentation, recipes, and docstrings as needed.
  • Provide accuracy/performance validation as applicable.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@jiapingW
jiapingW force-pushed the agent/pr678-offline-draft-training branch from affeae5 to cf2dbde Compare July 21, 2026 09:46
@jiapingW
jiapingW merged commit 4e3f455 into sgl-project:main Jul 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants