Skip to content

Support Gemma 4 Unified (gemma4_unified) — kernel patching + fused linear cross-entropy #1308

Description

@lefft

Request

Add apply_liger_kernel_to_gemma4_unified (with MODEL_TYPE_TO_APPLY_LIGER_FN entries for gemma4_unified and gemma4_unified_text), mirroring the gemma4 omni support from #1203. The unified Gemma 4 checkpoints (e.g. google/gemma-4-12B-it) use model_type: gemma4_unified with their own modeling module, so the existing gemma4/gemma4_text patches don't apply — _apply_liger_kernel currently no-ops with no Liger kernels supported for model type: gemma4_unified.

Motivation: long-context full fine-tuning is memory-gated by the unfused cross-entropy loss. Concrete data point: gemma-4-12B-it FFT at 65,536-token sequences on 8×H100-class GPUs (transformers 5.10.1, torch 2.13.0, FSDP full-shard + flex_attention): the 64k forward completes at ~128.6/139.8 GiB used, then loss computation materializes the [65536 × ~262k-vocab] logits (~64 GiB fp32) and OOMs at step 1. Fused linear cross-entropy is exactly the missing piece.

Alternatives

Forcing the existing gemma4 patches onto gemma4_unified modules doesn't work — they target transformers.models.gemma4.* module objects, so cross-module application either no-ops or corrupts training (all-NaN checkpoints have been reported downstream from this approach).

Additional context

Related: #1186 (gemma4 omni support, added by #1203).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions