From 49ac869cad6098ad0e8e3839fda21be7f087d789 Mon Sep 17 00:00:00 2001
From: Loki Chen <dengcchi@meta.com>
Date: Mon, 6 Jul 2026 13:29:30 -0700
Subject: [PATCH] docs: document MXFP numerics (mxfp8/mxfp4/nvfp4) and hardware
 dispatch
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Qwix supports mxfp8, mxfp4, and nvfp4 as user-facing weight/activation
qtypes, but they are absent from the user documentation. This adds a docs
page covering:

- Block/tile sizes (32 for mxfp8/mxfp4, 16 for nvfp4)
- Scale format (e8m0fnu vs e4m3fn)
- Hardware dispatch table (GPU fused via scaled_matmul; TPU/CPU native
  fp8 dot_general fallback)
- Performance note about activation-quant overhead

No code change — docs only.
---
 docs/source/mxfp.md | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100644 docs/source/mxfp.md

diff --git a/docs/source/mxfp.md b/docs/source/mxfp.md
new file mode 100644
index 0000000..b1a4e94
--- /dev/null
+++ b/docs/source/mxfp.md
@@ -0,0 +1,38 @@
+# MXFP quantization (mxfp8 / mxfp4 / nvfp4)
+
+Qwix supports microscaled floating-point (MXFP) weight/activation qtypes.
+Set `weight_qtype` / `act_qtype` in a `QtRule` or PTQ config to one of:
+`mxfp8`, `mxfp4`, `nvfp4`.
+
+## Block / tile sizes
+
+- `mxfp8`, `mxfp4`: block size 32 (per-32-element scale).
+- `nvfp4`: block size 16.
+
+## Scale format
+
+- `mxfp8` / `mxfp4`: `float8_e8m0fnu` block scales.
+- `nvfp4`: `float8_e4m3fn` block scales.
+
+## Hardware dispatch
+
+`dot_general` on MXFP operands dispatches by platform:
+
+| Platform | Path | Notes |
+|----------|------|-------|
+| GPU (Blackwell) | Fused `jax.nn.scaled_matmul` (cuDNN) | `mxfp_dot._gpu_mxfp_dot` |
+| GPU (legacy) | `scaled_matmul` emulated/decomposed | |
+| TPU / CPU | Native tiled fp8 `dot_general` | `mxfp_dot_general` returns `None` → fallback to `_fast_dot_general` |
+
+When `mxfp_dot_general` returns `None` (non-GPU platforms), the caller
+falls through to the native quantized `dot_general` path, which is correct
+and MXU-accelerated on TPU.
+
+## Performance note
+
+The fp8 matmul itself is fast; the activation-quantization + cast plumbing
+around it can dominate in full forward passes. Weight caching (PTQ: quantize
+the weight once) amortizes weight-quant cost across reuses, but does not
+amortize per-call activation quantization. Expect fp8 gains primarily in
+large GEMMs with high weight reuse; a full small-model forward may not
+benefit at current XLA lowering.