spec: fix dflash target tokenizer mismatch during conversion

[ruixiang63]

Overview

Fix DFlash conversion issue. See #22105 (comment)

Root cause: DFlashDraftModel always used the Qwen vocab path, so target models needing different tokenizer handling (e.g. Gemma-4, which has no tokenizer.model) failed with "BPE pre-tokenizer was not recognized".

Solution: set set_vocab to read the target's architecture from its config.json and dispatches to that architecture's own set_vocab.

Tested with Gemma4 DFlash models, works as expected.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes, paired with cursor

[ruixiang63]

@ggerganov Now it should be fixed. The Gemma4 DFlash also works well.

[ggerganov]

Thanks. There is Python type check error that should be fixed.

On a similar note, I found that the following Eagle3 model does not convert:

• https://huggingface.co/openai/gpt-oss-120b
• https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-v3

Command:

python3 ./llama.cpp/convert_hf_to_gguf.py ./model-temp-gpt_oss_120b-EAGLE3 \
    --outtype bf16 --target-model ./model-temp-gpt_oss_120b-PRIMARY \
    --outfile ./upload-gpt_oss_120b/eagle3-gpt-oss-120b-BF16.gguf \
    --model-name gpt-oss-120b

Error:

Details

+ python3 ./llama.cpp/convert_hf_to_gguf.py ./model-temp-gpt_oss_120b-EAGLE3 --outtype bf16 --target-model ./model-temp-gpt_oss_120b-PRIMARY --outfile ./upload-gpt_oss_120b/eagle3-gpt-oss-120b-BF16.gguf --model-name gpt-oss-120b

INFO:hf-to-gguf:Loading model: model-temp-gpt_oss_120b-EAGLE3

INFO:hf-to-gguf:Model architecture: LlamaForCausalLMEagle3

INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors'

INFO:hf-to-gguf:EAGLE-3: renaming midlayer.* / layers.0.* to model.layers.0.*

INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only

INFO:hf-to-gguf:Detected EAGLE-3 draft model, switching to EAGLE3 architecture

WARNING:gguf.gguf_writer:Duplicated key name 'general.architecture', overwriting it with new value 'eagle3' of type STRING

INFO:hf-to-gguf:EAGLE-3: target_layers = [2, 18, 33] (target model has 36 layers)

INFO:hf-to-gguf:EAGLE-3: target_hidden_size = 2880 (from target model config)

INFO:hf-to-gguf:EAGLE-3: norm_before_residual = False

INFO:hf-to-gguf:Exporting model...

INFO:hf-to-gguf:fc.weight,                torch.bfloat16 --> BF16, shape = {8640, 2880}

Traceback (most recent call last):

  File "/tmp/convert/./llama.cpp/convert_hf_to_gguf.py", line 296, in <module>

    main()

  File "/tmp/convert/./llama.cpp/convert_hf_to_gguf.py", line 290, in main

    model_instance.write()

  File "/tmp/convert/llama.cpp/conversion/base.py", line 1025, in write

    self.prepare_tensors()

  File "/tmp/convert/llama.cpp/conversion/llama.py", line 320, in prepare_tensors

    super().prepare_tensors()

  File "/tmp/convert/llama.cpp/conversion/base.py", line 883, in prepare_tensors

    for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):

  File "/tmp/convert/llama.cpp/conversion/llama.py", line 280, in modify_tensors

    yield from super().modify_tensors(data_torch, name, bid)

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/tmp/convert/llama.cpp/conversion/base.py", line 618, in modify_tensors

    new_name = self.map_tensor_name(name)

               ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/tmp/convert/llama.cpp/conversion/base.py", line 611, in map_tensor_name

    raise ValueError(f"Can not map tensor {name!r}")

ValueError: Can not map tensor 'input_norm.weight'

[ruixiang63]

On a similar note, I found that the following Eagle3 model does not convert:

• https://huggingface.co/openai/gpt-oss-120b
• https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-v3

Command:

python3 ./llama.cpp/convert_hf_to_gguf.py ./model-temp-gpt_oss_120b-EAGLE3 \
    --outtype bf16 --target-model ./model-temp-gpt_oss_120b-PRIMARY \
    --outfile ./upload-gpt_oss_120b/eagle3-gpt-oss-120b-BF16.gguf \
    --model-name gpt-oss-120b

This EAGLE3 checkpoint appears to be relatively new, and its architecture or configuration may have changed.
The previous EAGLE3 checkpoint, https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-long-context, should work.

Supporting this new checkpoint would likely require a separate PR to add explicit support for it.

[ruixiang63]

On a similar note, I found that the following Eagle3 model does not convert:
https://huggingface.co/openai/gpt-oss-120b
https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-v3

@ggerganov I created another PR to add support for this eagle3-v3 model: #25794