fix(minimax-h3): NVFP4 ref2va grid root-caused — high-nibble-first fp4 packing (loader fix) - #94
Closed
localai-bot wants to merge 2 commits into
Closed
fix(minimax-h3): NVFP4 ref2va grid root-caused — high-nibble-first fp4 packing (loader fix)#94localai-bot wants to merge 2 commits into
localai-bot wants to merge 2 commits into
Conversation
…4 packing The ref2va NVFP4 grid (#93, checkpoint-correlated) is a LOADER bug, found by an independent-oracle loader diff, NOT a file defect and NOT the ref2va assembly. Method (no new download). An independent CPU dequant of `minimax_h3_ref2va_nvfp4_full.safetensors` (own fp8-e4m3fn + E2M1 + bf16 math, each primitive verified EXACT vs torch) diffed against the coherent FL2VA GGUF (same base DiT: the islands condition_proj/time_embedder/patch_proj/norms and the per-block norm1/q_norm are BYTE-IDENTICAL between the two files -> shared weights, so the GGUF is a per-tensor oracle for the projections too). Result: the quantized projections have IDENTICAL value distributions (absmax/std match ~1%) but ZERO elementwise correlation reading the file low-first. Unpacking the fp4 nibbles HIGH-first restores it: sign-agreement 1.000 over 115M+ weights, corr 0.85->0.94 rising with |w| (the NVFP4-vs-Q3_K quant-noise floor). Low-first corr 0.000. Root cause. The community checkpoint (metadata converted_by "Star Ultimate Model Converter Pro") packs element 2i in the HIGH nibble and 2i+1 in the LOW nibble -- the OPPOSITE of the modelopt standard our DequantNvfp4ToBf16 and the Marlin W4A16 path assume. Read low-first, every adjacent fp4 pair is swapped, scrambling each projection so the DiT cannot denoise and every render grids. Islands are bf16 (not nibble-packed) so they were unaffected -> byte-identical -> the misattribution. Fix. Swap the two nibbles of every packed byte at load ((b>>4)|(b<<4)) in the three H3 NVFP4 loaders -- LoadMiniMaxH3DitFromNvfp4 (reference), and the bf16 and fp4-resident device streamers -- turning the file's high-first bytes into the standard low-first the bf16 dequant AND Marlin both expect (one transform, both arms). H3-scoped: the shared DequantNvfp4ToBf16 stays low-first for the modelopt checkpoints (Laguna / DeepSeek-V4 / Qwen3-32B), which are coherent and untouched. Default ON; VT_H3_NVFP4_LOWNIBBLE=1 restores the pre-fix read for A/B. The synthetic NVFP4 gates compare our paths to each other (streamer vs reference, bf16 vs fp4 wiring), so the uniform swap keeps them green. GB10 render A/B pending on this commit. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
…ND+FIXED, 2nd defect isolated (#94) Records for row/H3-NVFP4-LOADER-DIFF (fix in the preceding commit): spec §8.11 + §8.2 supports-audit row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record entry, ENVIRONMENT (VT_H3_NVFP4_LOWNIBBLE), NOW, state. Independent-oracle loader diff (no download) root-caused #93's residual: the community ckpt packs fp4 HIGH-first (converter "Star Ultimate Model Converter Pro"); our loader read low-first -> every adjacent pair swapped -> scrambled projections. Vs the coherent FL2VA GGUF (same base model: islands byte-identical, all sampled projections sign-agree 1.000): low-first corr 0.000, high-first sign-agree 1.000 over 115M weights. Fixed by a load-time nibble swap in the 3 H3 NVFP4 loaders; BYTE-VERIFIED on GB10 (binary streamer dump == oracle high-first == GGUF-sign; params identical). #93's three guesses all missed the nibble order. HONEST residual: the render STILL grids (t2va, ref2va, keyframe-pinned all grid) while the FL2VA-GGUF control renders coherent on IDENTICAL weights+params -> a SECOND, independent defect in the NVFP4 render PATH (device stream/forward), NOT the checkpoint content, the nibble order, fp4, or free-generation. fp4-resident Marlin arm grids differently again (3rd, Marlin-specific). The nibble fix is the objectively-correct dequant and lands default-ON (VT_H3_NVFP4_LOWNIBBLE=1 reverts), byte-verified but not yet render-validated. Next: layer activation diff NVFP4 vs GGUF stream. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
… packing unscrambled (#94) row/H3-NVFP4-LOADER-DIFF squash. Independent-oracle loader diff (own fp8-e4m3fn/E2M1/bf16-RNE math, primitives byte-verified vs torch; cross-checked against the coherent FL2VA GGUF): the community ref2va NVFP4 checkpoint ("Star Ultimate Model Converter Pro") packs the two fp4 elements per byte HIGH-first — opposite the modelopt standard our dequant assumes. Read low-first, every adjacent weight pair swaps: corr 0.000 (scrambled) -> sign-agreement 1.000 over 115M+ weights with the swap (corr 0.85-0.94 = pure NVFP4-vs-Q3_K quant noise); fp32/bf16 islands were byte-identical all along (same base model, not nibble-packed). Fix: nibble-swap at load in the three H3 NVFP4 loaders (H3-scoped; the shared DequantNvfp4ToBf16 stays low-first for Laguna/DS4/Qwen3), default ON, VT_H3_NVFP4_LOWNIBBLE=1 reverts. #93's three suspects (islands/weight_scale_2/name-map) all cleared. HONEST: byte-correct weights do NOT un-grid the render — dark grid becomes pale grid; keyframe-pinning does not rescue; the FL2VA-GGUF control stays coherent in the same build. Since checkpoint content now provably matches the coherent GGUF's model, the residual is a SECOND defect in the NVFP4 render PATH (device stream/forward; the fp4-resident Marlin arm grids differently again — a third, wiring-gated-only issue). Next: layer-by-layer activation diff of the NVFP4-bf16 stream vs the GGUF-bf16 stream — identical weights, differing only in dequant source and island read width. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
Collaborator
Author
|
Landed on main as the squash (mudler authorship). The NVFP4-vs-GGUF identical-weights activation diff runs next — the second defect's decisive experiment. 🤖 Generated with Claude Code |
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…iscrete load-path defect Ran #94's prescribed identical-weights activation diff (NVFP4-bf16 stream vs the FL2VA-GGUF-bf16 control, byte-identical inputs via an encode-once pe.f32) plus direct WEIGHT fingerprints, through an env-gated per-stage hook in MiniMaxH3DitForwardDevice (VT_H3_ACT_DUMP; byte-inert unset). The result REFUTES the "second NVFP4-render-path defect" hypothesis. Instrument (byte-inert): a reusable H3DumpFingerprint over every embed/scatter/time stage, every block's adaln+post-attn+post-mlp, the final heads, the RoPE cos/sin cache, the block-0 attention internals (qkv/split/qknorm/rope/core/out_proj), and input-independent fingerprints for every island, bias, output head, q/k-norm, and block-0/refiner-0 projection — the classes #94's oracle never sampled. Finding: EVERY weight, bias, fp32 island, output head and q/k-norm loads quant-noise-close to the coherent GGUF (qkv/out/fc/adaln 2-6% rms = Q3K-vs-NVFP4 quantizer variance, sign-correct; all biases <0.1% apart incl adaln_b), and the RoPE cache is BYTE-IDENTICAL — no scramble/transpose/mis-stride/wrong-dtype/ wrong-shape. Both arms run identical code, so the grid is 100% the per-weight NVFP4-vs-Q3K quant delta on the SAME weights. Divergence first appears at the token refiner + the block-0 attention INPUT (not RoPE, not a GEMM/norm weight) and amplifies chaotically through the 50-block stack, driven by the Qwen massive text activation (condition_proj absmax ~7.4e4); final latents are DECORRELATED (sample-rel-L2 >1, not scale). Render A/B re-confirmed in the same byte-inert build: NVFP4 t2va = pale patch grid, FL2VA-GGUF t2va = coherent orange cat. Conclusion: not a loader fix. The residual is the community NVFP4 checkpoint's quantization fidelity (Star Ultimate Model Converter Pro lineage; corr 0.85-0.94 to the coherent Q3_K) times the DiT's massive-activation sensitivity. Clean- reference disambiguation is blocked (bf16 132GiB = OOM on one GB10; a same-finetune REF2VA-GGUF control = 23G-disk-blocked); the path forward is an official modelopt-NVFP4 checkpoint. The #94 nibble fix stands. The fp4-resident Marlin arm's separate grid is untouched (wiring-gated-only residual). Records: spec §8.12 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW, state. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…quant fidelity (#95) row/H3-NVFP4-STREAM-DIFF squash. Identical-weights activation diff: divergence enters at the token refiner and amplifies chaotically, but the guilty-class audit finds NO loader bug — every tensor class loads quant-noise-close to the coherent GGUF, RoPE byte-identical, no scramble/transpose/stride/dtype defect; both arms run identical code. The grid is the community NVFP4 file's own quantization fidelity (corr 0.85-0.94, same converter lineage as the #94 nibble bug) times the DiT's massive-activation sensitivity. Path forward: an official modelopt-NVFP4 checkpoint, not a loader change. #94 nibble fix stands; the fp4-resident Marlin arm grid stays the named wiring residual. The byte-inert VT_H3_ACT_DUMP fingerprint hook ships as a permanent instrument. Merge also anchors a parallel session's anchorless startup-latency state entry. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ref2va NVFP4 grid — fp4 nibble-order LOADER bug root-caused + FIXED (byte-verified), but a 2nd NVFP4-path defect remains
Independent-oracle loader diff on #93's residual, NO new download.
Method. An independent CPU dequant of
minimax_h3_ref2va_nvfp4_full.safetensors(own fp8-e4m3fn + E2M1 + bf16-RNE math, each primitive byte-EXACT vs torch) diffed against the coherent FL2VA GGUF (maintainedgguf.quants). The two files are ONE base model: islands (condition_proj,time_embedder, patch projections, norms,rope.inv_freq) byte-identical, every sampled projection (qkv/out/fc1/fc2/adaln, blocks 0..45 + both refiners) sign-agreement 1.000 — so the coherent GGUF is a per-tensor oracle for the ref2va projections.Root cause (fixed). The community checkpoint (metadata
converted_by: "Star Ultimate Model Converter Pro") packs fp4 HIGH-first (element2iin the high nibble) — opposite the modelopt standard ourDequantNvfp4ToBf16+ Marlin assume. Read low-first, every adjacent pair is swapped → each projection scrambled. Vs the GGUF: low-first corr 0.000; high-first sign-agreement 1.000 over 115M+ weights, corr 0.85→0.94 rising with |w| (NVFP4-vs-Q3_K quant-noise floor). Islands are bf16 (not nibble-packed) → byte-identical → why #86/#93 misattributed. #93's three guesses (island preservation / weight_scale_2 / name mapping) all missed it. Fix: nibble-swap(b>>4)|(b<<4)at load in the 3 H3 NVFP4 loaders; H3-scoped; default ON,VT_H3_NVFP4_LOWNIBBLE=1reverts.Byte-verified on GB10. An env-gated dump of the actual streamer: with the fix,
blocks.0.attn.qkv_proj[0:16]== the oracle HIGH-first row exactly, sign-identical to the GGUF; derived params identical between the two files.HONEST residual — the fix does NOT un-grid. GB10 A/B (256×256/22f/12steps): the nibble fix changes the output (low-first severe grid → high-first pale grid, so the swap IS applied) yet every NVFP4 render still grids — t2va, ref2va, and even fl2va-with-keyframe (output-pinned) — while the FL2VA-GGUF control renders a coherent orange cat on identical weights + params in the same build. So the checkpoint content is sound, the loader dequant is byte-correct, params are identical, and pinning doesn't rescue it → the residual grid is a SECOND, independent defect in the NVFP4 render PATH (device stream / forward), NOT the checkpoint, the nibble order, fp4, or free-generation. The fp4-resident Marlin arm grids differently again (a 3rd, Marlin-specific issue; that path was only ever wiring-gated). The nibble fix is the objectively-correct dequant and lands default-ON, byte-verified but not yet render-validated. Next: layer-by-layer activation diff of the NVFP4-bf16 vs GGUF-bf16 stream (identical weights).
Synthetic CPU/CUDA NVFP4 gates compare our paths to each other, so the uniform swap keeps them green. Full detail + the A/B table in the benchmark record + spec §8.11.
🤖 Generated with Claude Code
https://claude.ai/code/session_014fZAcg1WcU8V629k6HWKys