Skip to content

convert_hf_to_gguf.py: Refactor how --mtp / --no-mtp is enabled for a model#25641

Open
tarruda wants to merge 1 commit into
ggml-org:masterfrom
tarruda:refactor-convert-to-gguf-mtp-option
Open

convert_hf_to_gguf.py: Refactor how --mtp / --no-mtp is enabled for a model#25641
tarruda wants to merge 1 commit into
ggml-org:masterfrom
tarruda:refactor-convert-to-gguf-mtp-option

Conversation

@tarruda

@tarruda tarruda commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Overview

--no-mtp/--mtp doesn't currently work on Hy3 (#25395) despite the model supporting MTP.

Refactor how that those options are added by adding a flag to the ModelBase class and enabling for Qwen 3.x, Stepfun and Hy3.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: I used codex to show me how this mtp/no-mtp option was enabled and refactored the code myself.

- Add a supports_mtp_export capability to ModelBase so architectures can opt
  into --mtp and --no-mtp without extending a central class allowlist.
- Enable the capability for the existing Qwen3.5/3.6 and Step3.5/3.7
  implementations, and for HY V3, whose converter already supports
  filtering the appended MTP layers.
@satindergrewal

Copy link
Copy Markdown
Contributor

Confirming from the Hy3 side: HYV3Model already implements the full no_mtp / mtp_only filtering (the n_nextn hparams guard in set_gguf_parameters plus the filter_tensors checks in conversion/hunyuan.py), so the hardcoded gate in main() was the only thing blocking it. That was my omission in #25395, thanks for catching it. The supports_mtp_export class flag is also a cleaner shape than the growing issubclass chain, and the diff covers all three current MTP families. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants