mudler · localai-bot · Aug 7, 2026 · Aug 7, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -17,7 +17,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | Laguna NVFP4 / DeepSeek-V4 decode | **Both CLOSED, byte-exact, default-ON**: 1.03x vLLM, 1.144x ds4 | Laguna vLLM K-run when convenient |
 | f32-out GEMV audit | Only laguna + ds4 bf16 tower affected; gate models unaffected | Re-verify ds4 tower same-tool |
 | Invocation-parity prevention | CI guard + checklist landing | Merge; build-verify `kGemvHeuristicAlgos` on dgx |
-| MiniMax-H3 lane | **fl2va COHERENT; ref2va assembly bug FIXED+gated** (`H3-REF2VA-ASSEMBLY`) | grid RE-ATTRIBUTED to the ref2va NVFP4 checkpoint/loader, NOT assembly/fp4 (§8.10) |
+| MiniMax-H3 lane | **fl2va COHERENT; ref2va NVFP4 fp4-nibble loader bug FIXED** (byte-verified, #94) | grid PERSISTS: 2nd NVFP4-render-path defect (ckpt/params byte-match coherent GGUF) §8.11 |
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | **e2e RUNS** (bf16-resident §13): 13/13·656. Token gate **NEAR-TIE 106/128** | device GDN/MLA islands; 1.59 tok/s; default OFF |
 | 35B fresh grid | **BOUND** @`1ea26427`: 0.93-1.03x, c16 0.93x. INTAKE + Option A both NEGATIVE | Lever left: prefill glue (#61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |

diff --git a/.agents/benchmark-record.md b/.agents/benchmark-record.md
@@ -14400,3 +14400,28 @@ so both misattributed a checkpoint/loader defect. True residual = the NVFP4 DiT
 Synthetic-NVFP4 gates proved the dequant MATH byte-exact but never loaded THIS file vs a coherent oracle.
 Next: a REF2VA GGUF (bf16, known-good loader) as checkpoint oracle — dgx-disk-blocked (23 GiB free).
 Artifacts `~/h3fp4/out_{vs_ref2va,bf16_ref2va,t2va_nvfp4,vs_fl2va}.mp4`.
+
+## MiniMax-H3 NVFP4 ref2va grid — independent-oracle loader diff: fp4 nibble-order bug FIXED, but a 2nd NVFP4-path defect remains (`row/H3-NVFP4-LOADER-DIFF` PR #94, 2026-08-07, dgx sm_121a)
+
+Followed #93's residual (the ref2va NVFP4 checkpoint/loader) with an independent-oracle loader diff — NO new download. Root-caused + fixed a REAL loader bug #93 mis-guessed, verified byte-exact, but the render still grids from a SECOND, independent defect that is NOT the checkpoint content and NOT the loader dequant.
+
+**Method.** An independent CPU dequant of `minimax_h3_ref2va_nvfp4_full.safetensors` (own fp8-e4m3fn + E2M1 + bf16 math; each primitive byte-EXACT vs torch, verified over all 256 fp8 bytes and 200k bf16 RNE cases) diffed against the coherent FL2VA GGUF via the maintained `gguf.quants` dequant (trusted, independent). The two files share the SAME base DiT: the islands (`condition_proj`, `time_embedder`, both patch projections, norms, `rope.inv_freq`) are BYTE-IDENTICAL, and every sampled projection (`qkv`/`out`/`fc1`/`fc2`/`adaln`, blocks 0..45 step 5 + both token_refiners) has sign-agreement 1.000 — so fl2va and ref2va are one model, and the coherent GGUF is a per-tensor oracle for the ref2va projections too.
+
+**Root cause (fixed).** The community checkpoint (metadata `converted_by: "Star Ultimate Model Converter Pro"`) packs the two fp4 elements per byte HIGH-first (element 2i in the high nibble, 2i+1 in the low) — the opposite of the modelopt standard our `DequantNvfp4ToBf16` + Marlin assume. Read low-first, every adjacent fp4 pair is swapped, so each projection matrix is internally scrambled: vs the coherent GGUF, low-first gives elementwise corr **0.000**; HIGH-first gives **sign-agreement 1.000 over 115M+ weights** and corr 0.85→0.94 rising with |w| (the NVFP4-vs-Q3_K quant-noise floor). Islands are bf16 (not nibble-packed) → byte-identical → exactly why #93/#86 misattributed. #93's three guesses (island preservation / weight_scale_2 / name mapping) were all wrong; none named the nibble order. Fix: swap the two nibbles of every packed byte at load (`(b>>4)|(b<<4)`) in the three H3 NVFP4 loaders (reference + bf16 streamer + fp4-resident Marlin streamer) → the file's high-first bytes become the standard low-first both arms expect. H3-scoped; shared `DequantNvfp4ToBf16` untouched (Laguna/DS4/Qwen3 stay low-first). Default ON; `VT_H3_NVFP4_LOWNIBBLE=1` reverts.
+
+**Fix BYTE-VERIFIED on GB10.** An env-gated dump of the actual streamer output: the binary's `blocks.0.attn.qkv_proj[0:16]` with the fix = `-0.0859 -0.0430 -0.1289 -0.0430 -0.0859 0.1719 -0.1289 0.2578 ...` = EXACTLY the independent oracle's HIGH-first row, sign-identical to the GGUF (`-0.0679 -0.0679 -0.1357 ...`). Derived params IDENTICAL between the two files (L=50 refL=2 H=5376 heads=56 ffn=14336 txt=5120 lat=24 alat=32 adaln=96768 …).
+
+**GB10 render A/B (256×256/22f/12steps).**
+
+| Render | Checkpoint | Quant | nibble | Result |
+|---|---|---|---|---|
+| t2va (no refs, `--partition fl2va`) | ref2va NVFP4 | bf16 | LOW (=1) | severe dark GRID (= #93 baseline) |
+| t2va (no refs, `--partition fl2va`) | ref2va NVFP4 | bf16 | HIGH (fix) | pale GRID (weights now correct; STILL grids) |
+| ref2va (`--ref-image`+`--cond-image`) | ref2va NVFP4 | bf16 & fp4 | HIGH (fix) | GRID |
+| fl2va + keyframe (`--first-frame`) | ref2va NVFP4 | bf16 | HIGH (fix) | GRID (output-pinning does NOT rescue) |
+| t2va (no refs, `--partition fl2va`) | FL2VA GGUF | bf16 | — | COHERENT orange cat (control, this build) |
+| fl2va + keyframe (`--first-frame`) | FL2VA GGUF | bf16 | — | COHERENT (#93) |
+
+**RE-ATTRIBUTION (corrects #93 again).** The nibble fix CHANGES the output (low-first severe grid → high-first pale grid, so the swap IS applied) yet EVERY NVFP4 render grids — t2va, ref2va, AND fl2va-with-keyframe — while the fl2va-GGUF control (SAME weights, SAME params, verified byte-for-byte) renders a coherent cat in the same build and same task. So: the checkpoint CONTENT is sound (byte-matches the coherent GGUF), the loader dequant is now byte-correct (binary dump == oracle == GGUF-sign), the derived params are identical, and output-pinning (keyframe) does NOT rescue it (rules out free-generation divergence). The residual grid is therefore a SECOND, independent defect in the NVFP4 render PATH itself — the device stream / forward, NOT the checkpoint, NOT the fp4 nibble order, NOT free-gen. The fp4-resident Marlin arm additionally grids differently (a THIRD, Marlin-specific issue — that path was only ever wiring-gated, never correctness-gated). The nibble fix is the objectively-correct dequant (the file IS high-first) and lands default-ON (A/B via `VT_H3_NVFP4_LOWNIBBLE=1`); it is byte-verified but not yet render-validated, blocked on the second defect. Next diagnostic: layer-by-layer intermediate-activation diff of the NVFP4-bf16 stream vs the GGUF-bf16 stream (identical weights) to locate where they diverge — the streamers are structurally identical except the dequant source and the island read (bf16-disk vs f16-disk), so the divergence is a candidate.
+
+Artifacts `~/h3fp4/{t2va_fix,t2va_nofix,ctrl_t2va,ab_bf16fix,ab_fix,ab_nofix,kf_nvfp4}.mp4`.
diff --git a/.agents/specs/minimax-h3.md b/.agents/specs/minimax-h3.md
@@ -400,7 +400,7 @@ vLLM-Omni H3 modules at `vllm_omni/diffusion/models/minimax_h3/`; serving in
 | WebSocket `/v1/video/chat/stream`, `/v1/realtime/video` | `api_server.py:1593,1610` | — | **MISSING** (streaming/realtime) |
 | Request schema (prompt, size/w/h, num_frames, fps, seed, steps, refs) | `protocol/videos.py:97-249` | request contract (W7) | **PARTIAL** (core fields; frame-interp/lora/generate_sound absent) |
 | H3 knobs via `extra_params.{task,duration,flow_shift,audio_flow_shift}` | `pipeline:1034,403,1157-1158` | planner reads task/duration/shift | **DONE** |
-| Modalities in: text/image/video/audio | `pipeline:1036-1104` | t2va (text) done; vision tower LOADS real `visual.*` + runs; merged→prompt_embeds scatter + DeepStack→device text tower WIRED 1:1 + gated (§8.9); fl2va COHERENT via BOTH the VAE-keyframe AND the encoder vision path; ref2va reference-row assembly FIXED + gated (§8.10, the block-dim double-division) — but ref2va still grids, now RE-ATTRIBUTED to the ref2va NVFP4 CHECKPOINT/loader (t2va-zero-assembly grids too), NOT the assembly | **PARTIAL** (vision→conditioning scatter + ref2va assembly DONE; residual = the NVFP4 DiT loader for the ref2va checkpoint, §8.10) |
+| Modalities in: text/image/video/audio | `pipeline:1036-1104` | t2va (text) done; vision tower LOADS real `visual.*` + runs; merged→prompt_embeds scatter + DeepStack→device text tower WIRED 1:1 + gated (§8.9); fl2va COHERENT via BOTH the VAE-keyframe AND the encoder vision path; ref2va reference-row assembly FIXED + gated (§8.10) — ref2va still grids; the NVFP4 fp4 nibble-order loader bug is now FOUND+FIXED (byte-verified, §8.11) but the grid PERSISTS from a 2nd NVFP4-render-path defect (checkpoint content + loader dequant + params all byte-match the coherent GGUF) | **PARTIAL** (vision→conditioning scatter + ref2va assembly + NVFP4 nibble loader DONE; residual = the 2nd NVFP4-render-path defect, §8.11) |
 | Output: joint video+audio, 24 fps, 32 kHz stereo | `pipeline:106-111,1187` | frames + WAV + MP4 mux (W7) | **DONE** |
 | Scheduler: euler-ancestral rectified flow (single) | `scheduling_...euler_ancestral.py`; `time_request.py:34-61` | `MiniMaxH3EulerEta0Step` / `MiniMaxH3TimeShiftSigmas` | **DONE** |
 | CFG: distilled, no CFG (guidance params accepted+ignored; `cfg_parallel_size==1`) | `pipeline:250,275-276` | no CFG branch | **DONE** (matches) |
@@ -790,3 +790,15 @@ anchor.
 | `model` | `VideoRequest::model` | Recorded + echoed; an unserved name is a job `warning`, never a rejection (a Sora client cannot know the local model's name) |
 | `size` | `width`, `height` | `"<w>x<h>"`, whole positive pixels, one `x`/`X` |
 | `seconds` | `duration_seconds` | Number OR numeric string — OpenAI types it as a string enum ("4"/"8"/"12") |
+
+## 8.11 NVFP4 LOADER DIFF — fp4 nibble-order bug FIXED (byte-verified); the grid's 2nd defect isolated (2026-08-07, `row/H3-NVFP4-LOADER-DIFF` PR #94)
+
+Resolves #93's residual (§8.10 → the ref2va NVFP4 checkpoint/loader) with an **independent-oracle loader diff**, NO new download. Root-causes and fixes a REAL loader bug #93 mis-guessed, byte-verified on GB10 — but the render still grids, so the grid is RE-ATTRIBUTED a third time.
+
+**The oracle (breaks the shared-helper trap).** An independent CPU dequant of `minimax_h3_ref2va_nvfp4_full.safetensors` with its OWN fp8-e4m3fn + E2M1 + bf16-RNE math (each primitive verified byte-EXACT vs torch over all 256 fp8 bytes + 200k RNE cases), diffed against the coherent FL2VA GGUF dequantized by the maintained `gguf.quants` package (a genuinely independent second implementation). **The two files are ONE base model:** the islands (`condition_proj`, `time_embedder`, both patch projections, all norms, `rope.inv_freq`) are BYTE-IDENTICAL and every sampled quantized projection (`qkv`/`out`/`fc1`/`fc2`/`adaln` at blocks 0..45 step 5 + both token_refiners) has sign-agreement **1.000** — so the coherent GGUF is a per-tensor oracle for the ref2va projections too.
+
+**Root cause = fp4 nibble order.** The community converter (metadata `converted_by: "Star Ultimate Model Converter Pro"`) packs the two fp4 elements per byte HIGH-first (element 2i in the HIGH nibble, 2i+1 in the LOW) — the opposite of the modelopt standard `DequantNvfp4ToBf16` and the Marlin W4A16 path assume. Read low-first, every adjacent fp4 pair is read swapped, scrambling each projection matrix. Vs the coherent GGUF: **low-first corr 0.000; high-first sign-agreement 1.000 over 115M+ weights**, corr 0.85→0.94 rising with |w| (the pure NVFP4-vs-Q3_K quant-noise floor). Islands are bf16 (not nibble-packed) → unaffected → byte-identical → why #86/#93 misattributed. **#93's three named suspects (fp32-island preservation, `weight_scale_2` double-dequant, 1051-tensor name mapping) were ALL wrong; none named the nibble order.**
+
+**Fix + byte-verification.** Swap the two nibbles of every packed U8 byte at load (`(b>>4)|(b<<4)`) in the three H3 NVFP4 loaders — `LoadMiniMaxH3DitFromNvfp4`, `StreamMiniMaxH3Nvfp4ToDeviceBf16`, `StreamMiniMaxH3Nvfp4ToDeviceFp4` — turning the file's high-first bytes into the standard low-first that both the bf16 dequant AND Marlin expect (one transform, both arms). H3-scoped: the shared `DequantNvfp4ToBf16` stays low-first for the modelopt checkpoints (Laguna / DeepSeek-V4 / Qwen3-32B, all coherent, untouched). Default ON; `VT_H3_NVFP4_LOWNIBBLE=1` reverts. An env-gated dump of the ACTUAL streamer output confirmed the fix byte-exact: the binary's `blocks.0.attn.qkv_proj[0:16]` == the oracle's HIGH-first row exactly, sign-identical to the GGUF; derived params IDENTICAL between the two files. The synthetic CPU/CUDA NVFP4 gates compare our paths to each other (streamer vs reference, bf16 vs fp4 wiring), so the uniform swap keeps them green.
+
+**But the render STILL grids — 2nd defect isolated.** GB10 A/B (256×256/22f/12steps): the nibble fix CHANGES the output (low-first severe grid → high-first pale grid, so the swap IS applied), yet EVERY NVFP4 render grids — t2va, ref2va, and even fl2va-WITH-keyframe (output-pinned) — while the FL2VA-GGUF control (SAME weights, SAME params, verified) renders a coherent orange cat in the same build and same task. With the checkpoint content sound, the loader dequant byte-correct, the params identical, and output-pinning not rescuing it (rules out free-gen divergence), the residual grid is a **SECOND, independent defect in the NVFP4 render PATH** (the device stream / forward), NOT the checkpoint and NOT the nibble order. The fp4-resident Marlin arm grids differently again (a THIRD, Marlin-specific issue; that path was only ever wiring-gated). The nibble fix is objectively correct (the file IS high-first) and lands default-ON, byte-verified but not yet render-validated (blocked on the 2nd defect). Next diagnostic: layer-by-layer activation diff of the NVFP4-bf16 stream vs the coherent GGUF-bf16 stream (identical weights) — they are structurally identical except the dequant source and the island read (bf16-disk vs f16-disk).
diff --git a/.agents/state.md b/.agents/state.md
@@ -40623,3 +40623,25 @@ a coherent oracle. NEXT: a REF2VA GGUF (bf16, known-good loader) as checkpoint o
 disk (23 GiB free, 100% full; large dirs belong to other campaigns, not prunable). Fix + gates LAND
 regardless; box left clean (renders exited, gpu.lock released, worker stays parked, ckpts kept). Records:
 spec §8.10 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW.
+
+
+[2026-08-07] H3-NVFP4-LOADER-DIFF (#94) — fp4 nibble-order loader bug FOUND+FIXED (byte-verified);
+grid 2nd defect isolated. Independent-oracle loader diff (no download): own fp8-e4m3fn+E2M1+bf16 math
+(each primitive byte-exact vs torch) vs the coherent FL2VA GGUF (gguf.quants). The two files are ONE base
+model — islands BYTE-IDENTICAL, every sampled projection (qkv/out/fc1/fc2/adaln blocks 0..45 + refiners)
+sign-agree 1.000. ROOT CAUSE: the community ckpt (converted_by "Star Ultimate Model Converter Pro") packs
+fp4 HIGH-first (element 2i in the high nibble) vs the modelopt low-first our DequantNvfp4ToBf16+Marlin
+assume; read low-first every adjacent pair is swapped -> scrambled matrix. Low-first corr 0.000; HIGH-first
+sign-agree 1.000 over 115M weights, corr 0.85->0.94 with |w| (NVFP4-vs-Q3K quant-noise floor). #93 three
+guesses (island preservation/weight_scale_2/name mapping) ALL wrong. FIX: nibble-swap (b>>4)|(b<<4) at load
+in the 3 H3 NVFP4 loaders (reference+bf16+fp4 streamers); H3-scoped, shared DequantNvfp4ToBf16 untouched
+(Laguna/DS4/Qwen3 stay low-first); default ON, VT_H3_NVFP4_LOWNIBBLE=1 reverts. BYTE-VERIFIED on GB10: the
+binary streamer dump of blocks.0.attn.qkv_proj[0:16] == the oracle HIGH-first row exactly, sign-matches the
+GGUF; params IDENTICAL between the files. BUT the render STILL grids: t2va, ref2va, AND fl2va-with-keyframe
+(output-pinned) all grid, while the FL2VA-GGUF control renders a coherent cat on IDENTICAL weights+params in
+the same build. So checkpoint content sound + loader dequant byte-correct + params identical + pinning does
+NOT rescue -> a SECOND, independent defect in the NVFP4 render PATH (device stream/forward), NOT the ckpt,
+nibble, fp4, or free-gen. fp4-resident Marlin arm grids differently again (3rd, Marlin-specific, only ever
+wiring-gated). NEXT: layer-by-layer activation diff of the NVFP4-bf16 stream vs the GGUF-bf16 stream
+(identical weights; differ only in dequant source + island read bf16-disk vs f16-disk). Records: spec
+§8.11 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW. Box left clean.
diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md
@@ -313,7 +313,7 @@ built on it rather than keeping the flattering one.
 | vLLM 0.26 re-benchmark | Pending | Re-run the binding grids on the advanced pin |
 | MiniMax-H3 FP4 speed (W-FP4a) | **Measured GB10 (`row/H3-FP4-GPU-E2E`).** Marlin W4A16 byte-exact vs bf16; fp4 a memory win, 0.8x bf16/forward. Real-ckpt fp4-resident e2e RUNS (mp4/wav) | fp4 speed CLOSED. Detail: benchmark-record + spec §8 |
 | MiniMax-H3 render coherence (`row/H3-RENDER-CLOSE` #77) | **CLOSED: a COHERENT scene on GB10.** #70/#74 white was wrong-PARTITION usage (t2va on the ref2va ckpt); t2va on the FL2VA GGUF renders a prompt-matched orange cat (adj-cos 0.95 vs 0.06, no patch-grid) | Verified first: t2va inputs byte-exact vs upstream; CUDA device==host at seq 1920. Follow-up `H3-TASK-PARTITION-GUARD`: the task/partition mismatch now RAISES 1:1 with `_resolve_task` (spec §8.6-8.7) |
-| MiniMax-H3 image conditioning (`row/H3-CONDITIONED-E2E`, `row/H3-VISION-SCATTER`, `row/H3-REF2VA-ASSEMBLY`) | **fl2va COHERENT; ref2va assembly bug FIXED+gated.** vision→cond scatter gated; ref2va block-dim double-division fixed + RED-first gated (128 vs 512) + a permanent ref2va DiT-forward rung (§8.10) | grid RE-ATTRIBUTED: with the fix ref2va grids in fp4 AND bf16, and t2va with no refs on the ref2va NVFP4 also grids while FL2VA-GGUF renders, so it is the **NVFP4 checkpoint/loader**, NOT assembly/fp4 (§8.10) |
+| MiniMax-H3 image conditioning (`row/H3-CONDITIONED-E2E`, `row/H3-VISION-SCATTER`, `row/H3-REF2VA-ASSEMBLY`, `row/H3-NVFP4-LOADER-DIFF`) | **fl2va COHERENT; ref2va assembly + NVFP4 fp4-nibble loader bug FIXED (byte-verified).** File packs fp4 HIGH-first, loader read low-first→scrambled (corr 0.000→1.000 sign-agree vs coherent GGUF); nibble-swap (§8.11) | grid RE-ATTRIBUTED AGAIN: with the fix the render STILL grids (t2va/ref2va/keyframe) while FL2VA-GGUF renders coherent on IDENTICAL weights+params → residual = a 2nd NVFP4-render-path defect, NOT ckpt/nibble/fp4 (§8.11) |
 | MXFP4 Qwen3-8B (W4A16 Marlin) | **`KERNEL-MARLIN-DENSE-EXEC` x3 (dense-ON default): c1 1.020, c2/c4/c8 0.962/0.966/0.969, GPU mem 2.63x less** (beats #51 1.005/0.925/0.939/0.953 EVERY axis); #44 3/3, 32B-NVFP4A16 6/6; -Werror test-guard fixes x2 | **VT_MARLIN_DENSE default-ON** (+951us). `FLASH-PTXAS` #82: cuModule A/B ties our+vLLM PTX across ptxas 13.0/13.2/driver-JIT (~144us); +10us is engine CONTEXT not codegen, no ptxas lever/flip (retires #75) |
 | Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | **NOT APPLICABLE: no number measured, claimed or owed.** Vulkan registers 8 of the CPU backend's 83 ops and runs no model. The 2026-08-06 campaign spec is PLAN ONLY. [Detail](../.agents/specs/vulkan-full-support.md) | `VK-E`: build llama.cpp `-DGGML_VULKAN=ON` at pin `237ad9b96` on dgx, `llama-bench` on the same GGUF, record three columns (ours-Vulkan, llama.cpp-Vulkan, ours-CUDA). Blocked by `VK-B` and by `glslc` on the gate box |
 | Memory footprint vs declared workload (`ROAD-V1-MEM`, #83) | **Never measured, and not measurable today**: there is no auto-sizing to compare against, because the KV pool is a hand-typed `--num-blocks`, so "what the run actually needed" has no number | Once M1's `MemoryBudget` lands: predicted-vs-actual bytes per allocation class, then peak footprint ours-auto vs vLLM at its 0.9 default on the same model and config |