diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 49c58586674..4c952ad0650 100755 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -75,6 +75,7 @@ Changelog - Fix INT8 entropy calibration of fp16 ONNX models raising ``ValueError: Too many bins for data range`` on numpy >= 2.0. ``_collect_value`` in ``modelopt.onnx.quantization.ort_patching`` now casts the histogram range endpoints to Python float so bin edges are computed in float64, instead of inheriting the fp16 dtype of an activation tensor with a small range (which collapsed the 128-bin linspace under NEP-50 promotion). - Fix the GPT-OSS MXFP4 → NVFP4 PTQ path in ``examples/llm_ptq/hf_ptq.py`` (used with ``--cast_mxfp4_to_nvfp4``). ``get_model`` now loads native MXFP4 checkpoints (``openai/gpt-oss-*``) dequantized to BF16 ``GptOssExperts`` via ``Mxfp4Config(dequantize=True)`` on a sequential device map. This fixes a CUDA illegal-memory access during the multi-GPU dequant load and the ``NotImplementedError`` for experts type ``Mxfp4GptOssExperts`` during unified HF export (the packed-kernel experts wrapper, used when the optional ``kernels`` package is installed, is unsupported by export); ``kernels`` is no longer required. The ``--cast_mxfp4_to_nvfp4`` step now also resolves a HF Hub ID ``--pyt_ckpt_path`` to its local snapshot directory instead of failing with ``FileNotFoundError``. - Fix ``_QuantGptOssExperts`` / ``_QuantLlama4TextExperts`` static-block NVFP4 weight calibration raising ``ValueError: Input shape has changed`` during the calibration forward. These experts quantize their weights transposed (``_transposed_quantize``); ``iter_weights_for_calibration`` now yields the same transposed view so weight-only calibration and the forward agree on the block-quant shape (and the export ``_amax`` orientation). +- Fix unified HF checkpoint export for Llama4 MoE models. The uncalibrated-experts input-quantizer ``amax`` fallback in ``_export_transformers_checkpoint`` special-cased only ``QuantGptOssExperts``; ``QuantLlama4TextExperts`` uses the same fused ``gate_up_proj`` / ``down_proj`` layout and is now handled by the same branch, fixing the export failure. **Deprecations** diff --git a/modelopt/torch/export/unified_export_hf.py b/modelopt/torch/export/unified_export_hf.py index ef5757aa0cb..892b8c42cad 100644 --- a/modelopt/torch/export/unified_export_hf.py +++ b/modelopt/torch/export/unified_export_hf.py @@ -859,9 +859,14 @@ def _export_transformers_checkpoint( elif hasattr(sub_module.experts, "gate_up_proj_weight_quantizers"): # _QuantFusedExperts: amax fallback is handled in _export_fused_experts break - elif "QuantGptOssExperts" in type(sub_module.experts).__name__: - # Handle GPT-OSS experts specifically - # GPT-OSS experts use gate_up_proj and down_proj + elif ( + "QuantGptOssExperts" in type(sub_module.experts).__name__ + or "QuantLlama4TextExperts" in type(sub_module.experts).__name__ + ): + # Handle GPT-OSS / Llama4 fused experts specifically. + # Both use gate_up_proj and down_proj with singular input quantizers + # (gate_up_proj_input_quantizer/down_proj_input_quantizer); the actual + # amax fallback and weight export is performed in _process_quantized_modules. gpt_oss_linear_names = ["gate_up_proj", "down_proj"] for linear_name in gpt_oss_linear_names: if hasattr(sub_module.experts, linear_name):