Found while implementing #241.
LoadMiniMaxH3DitFromGgufBf16 (src/vllm/model_executor/models/minimax_h3_gguf.cpp) dequantizes every tensor to bf16, including rope.inv_freq. The forward then reads it as f32:
// src/vllm/model_executor/models/minimax_h3.cpp:524
const std::vector<float> inv_freq(weights.rope_inv_freq.Ptr<float>(), ...);
vt::Tensor::Ptr<T>() is an unchecked static_cast, so the bf16 bit pattern is reinterpreted as float rather than converted — the RoPE inverse frequencies are garbage on that arm.
Scope: the CPU --dequant-bf16 fallback only (cuda == false). The production GPU path is StreamMiniMaxH3DitToDeviceBf16, which reads rope.inv_freq into rope_inv_freq_host as f32 and is unaffected; LoadMiniMaxH3DitFromGguf (keep-quant) dequantizes to f32 and is unaffected.
#241 keeps rope.inv_freq and adaln_t_table in f32 in that loader, which removes this instance. This issue tracks the wider hazard: the fp32-island rule (MiniMaxH3IsFp32IslandTensor, single-sourced in minimax_h3_sharded.cpp) is documented as binding on all four staging paths, and this loader does not consult it at all — the patch projections and output heads are still rounded there. Straightening that out means checking the activation dtype each of those GEMMs is fed, which is why #241 did not widen into it.
Kind: bug.
Found while implementing #241.
LoadMiniMaxH3DitFromGgufBf16(src/vllm/model_executor/models/minimax_h3_gguf.cpp) dequantizes every tensor to bf16, includingrope.inv_freq. The forward then reads it as f32:vt::Tensor::Ptr<T>()is an uncheckedstatic_cast, so the bf16 bit pattern is reinterpreted as float rather than converted — the RoPE inverse frequencies are garbage on that arm.Scope: the CPU
--dequant-bf16fallback only (cuda == false). The production GPU path isStreamMiniMaxH3DitToDeviceBf16, which readsrope.inv_freqintorope_inv_freq_hostas f32 and is unaffected;LoadMiniMaxH3DitFromGguf(keep-quant) dequantizes to f32 and is unaffected.#241 keeps
rope.inv_freqandadaln_t_tablein f32 in that loader, which removes this instance. This issue tracks the wider hazard: the fp32-island rule (MiniMaxH3IsFp32IslandTensor, single-sourced inminimax_h3_sharded.cpp) is documented as binding on all four staging paths, and this loader does not consult it at all — the patch projections and output heads are still rounded there. Straightening that out means checking the activation dtype each of those GEMMs is fed, which is why #241 did not widen into it.Kind: bug.