Skip to content

perf(ds4): compute greedy argmax on GPU (monolithic and layer split)#531

Draft
Graffioh wants to merge 4 commits into
Luce-Org:codex/ds4-rocmfpx-serverfrom
Graffioh:codex/ds4-gpu-argmax
Draft

perf(ds4): compute greedy argmax on GPU (monolithic and layer split)#531
Graffioh wants to merge 4 commits into
Luce-Org:codex/ds4-rocmfpx-serverfrom
Graffioh:codex/ds4-gpu-argmax

Conversation

@Graffioh

@Graffioh Graffioh commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • compute DeepSeek4 greedy argmax in the GGML GPU graph for local/remote layer-split and all-hot monolithic execution
  • copy one token ID instead of the full vocabulary logits during greedy decode
  • keep full logits for sampling and penalties, with the hybrid MoE path retaining its CPU fallback
  • propagate sampler state and make CPU/CUDA/HIP argmax tie-breaking consistent

Validation

  • cached, dynamic, and fused output-path coverage on CPU and supported GPU backends
  • sampler propagation and request-reset coverage

Stacked on #503.

@Graffioh Graffioh changed the title perf(ds4): compute layer-split greedy argmax on GPU perf(ds4): compute greedy argmax on GPU across execution paths Jul 16, 2026
@Graffioh Graffioh changed the title perf(ds4): compute greedy argmax on GPU across execution paths perf(ds4): compute greedy argmax on GPU( monolithic and layer split) Jul 16, 2026
@Graffioh Graffioh changed the title perf(ds4): compute greedy argmax on GPU( monolithic and layer split) perf(ds4): compute greedy argmax on GPU (monolithic and layer split) Jul 16, 2026
@Graffioh
Graffioh force-pushed the codex/ds4-gpu-argmax branch from 2fd6f64 to 22a4cbe Compare July 16, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant