mudler · localai-bot · Aug 7, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -18,7 +18,7 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | DeepSeek-V4-Flash decode | **Closed: beats ds4 1.144x** (`VT_V4_RESIDENT_W`, byte-exact); phase-2 residency NEG, default-OFF | — |
 | f32-out GEMV audit | Only laguna + ds4 bf16 tower affected; gate models unaffected | Re-verify ds4 tower same-tool |
 | Invocation-parity prevention | CI guard + AGENTS.md checklist landing | Merge; build-verify `kGemvHeuristicAlgos` on dgx |
-| MiniMax-H3 lane | **vision→cond scatter WIRED+gated; fl2va COHERENT via encoder path** (`H3-VISION-SCATTER` PR#90; indices {8,16,24} confirmed) | ref2va grids; residual = ref2va ref-row assembly (§8.9) |
+| MiniMax-H3 lane | **fl2va COHERENT; ref2va assembly bug FIXED+gated** (`H3-REF2VA-ASSEMBLY`) | grid RE-ATTRIBUTED to the ref2va NVFP4 checkpoint/loader, NOT assembly/fp4 (§8.10) |
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | **e2e RUNS** (bf16-resident §13): 13/13·656. Token gate **NEAR-TIE 106/128** | device GDN/MLA islands; 1.59 tok/s; default OFF |
 | 35B fresh grid | **BOUND** @`1ea26427`: 0.93-1.03x, c16 0.93x. INTAKE + Option A both NEGATIVE | Lever left: prefill glue (#61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |

diff --git a/.agents/benchmark-record.md b/.agents/benchmark-record.md
@@ -14362,3 +14362,41 @@ rows. So the residual is the **ref2va reference-row conditioning ASSEMBLY**
 noised-anchor layout + how the denoise loop conditions the un-pinned target rows), NOT the prompt and
 NOT the DiT forward. Next diagnostic: dump the ref2va target-row VAE-input adjacency-cosine (like #77's
 0.95 for coherent fl2va) to confirm the target rows are white, and A/B the reference-row condition-noise.
+
+## MiniMax-H3 ref2va ASSEMBLY fix + grid RE-ATTRIBUTED to the NVFP4 checkpoint (`row/H3-REF2VA-ASSEMBLY` PR #93, 2026-08-07, dgx sm_121a)
+
+Followed §8.9's residual (ref2va reference-row assembly), diffed it vs upstream, fixed a real bug,
+gated it, re-rendered — and the render DISPROVED the §8.9 attribution.
+
+**Real bug fixed.** `MiniMaxH3EncodeReferenceImages`/`Video` emitted PATCHED ref-block dims
+(`ls.h/patch_size_h`); `BuildMiniMaxH3PackedSequenceRef2va` divides `block.latent_h/kPatchH` AGAIN
+(mirroring upstream `packed_sequence.py:328-330`, which takes the UNPATCHED latent; upstream feeds the
+raw latent `visual_shape=(1,height//16,width//16)`, `pipeline_minimax_h3.py:1141-1145`). Double-division
+under-allocated the reference span by patch_h*patch_w (=4); the denoise pin-loop `>=` check silently
+TRUNCATED the oversized encoded reference to its first quarter. Fix = emit raw `ls.{t,h,w}`.
+
+**Gates.** goldens section 5c (ref2va-shaped DiT-forward rung: image + video+audio ref blocks, 8×8
+geometry, ref2va timestep partition + audio update mask, RefDiT) → C++ case "DiT-forward REF2VA rung
+matches upstream (reference rows, mixing)"; RED-first encoded-vs-layout row-count invariant
+(reintroducing the bug fails 128==512 = 16 encoded vs 4 allocated; restored green). Suite 69/69, 52377
+assertions. Goldens regen purely additive (635 inserts, 0 deletes).
+
+**GB10 render A/B + ISOLATION (256×256/22f/12steps, fixed incremental binary).**
+
+| Render | Checkpoint | Quant | Assembly | Result |
+|---|---|---|---|---|
+| ref2va (`--ref-image`+`--cond-image`) | ref2va NVFP4 | fp4-resident | full | GRID |
+| ref2va (`--ref-image`+`--cond-image`) | ref2va NVFP4 | bf16 dequant | full | GRID (fp4 eliminated) |
+| t2va (no refs, no cond-image, `--partition fl2va`) | ref2va NVFP4 | bf16 dequant | NONE | GRID (assembly eliminated) |
+| fl2va (keyframe + `--cond-image`) | FL2VA GGUF | bf16 | keyframe | COHERENT (cat→windowsill; no regression) |
+
+**Re-attribution (corrects §8.9 + §8.6).** The grid is NOT the ref2va assembly and NOT fp4 — it appears
+with the ENTIRE reference assembly removed, in both fp4 and bf16. It correlates 1:1 with the ref2va
+NVFP4 CHECKPOINT: every render loading `minimax_h3_ref2va_nvfp4_full` grids; every FL2VA-GGUF render is
+coherent. §8.9's A/B varied only the prompt, §8.6's only the task — neither varied the checkpoint/quant,
+so both misattributed a checkpoint/loader defect. True residual = the NVFP4 DiT loader for this file
+(`StreamMiniMaxH3Nvfp4ToDeviceBf16/Fp4`): suspect fp32-island preservation (patch/time/output layers,
+`minimax_h3_transformer.py:898-904`), `weight_scale_2` double-dequant, or the 1051-tensor name mapping.
+Synthetic-NVFP4 gates proved the dequant MATH byte-exact but never loaded THIS file vs a coherent oracle.
+Next: a REF2VA GGUF (bf16, known-good loader) as checkpoint oracle — dgx-disk-blocked (23 GiB free).
+Artifacts `~/h3fp4/out_{vs_ref2va,bf16_ref2va,t2va_nvfp4,vs_fl2va}.mp4`.
diff --git a/.agents/specs/minimax-h3.md b/.agents/specs/minimax-h3.md
@@ -400,7 +400,7 @@ vLLM-Omni H3 modules at `vllm_omni/diffusion/models/minimax_h3/`; serving in
 | WebSocket `/v1/video/chat/stream`, `/v1/realtime/video` | `api_server.py:1593,1610` | — | **MISSING** (streaming/realtime) |
 | Request schema (prompt, size/w/h, num_frames, fps, seed, steps, refs) | `protocol/videos.py:97-249` | request contract (W7) | **PARTIAL** (core fields; frame-interp/lora/generate_sound absent) |
 | H3 knobs via `extra_params.{task,duration,flow_shift,audio_flow_shift}` | `pipeline:1034,403,1157-1158` | planner reads task/duration/shift | **DONE** |
-| Modalities in: text/image/video/audio | `pipeline:1036-1104` | t2va (text) done; vision tower LOADS real `visual.*` + runs; merged→prompt_embeds scatter + DeepStack→device text tower WIRED 1:1 + gated (§8.9); fl2va COHERENT via BOTH the VAE-keyframe AND the encoder vision path; ref2va still grids (residual RE-ATTRIBUTED to the ref2va reference-row assembly, NOT the conditioning) | **PARTIAL** (vision→conditioning scatter DONE §8.9; residual = ref2va reference-row assembly) |
+| Modalities in: text/image/video/audio | `pipeline:1036-1104` | t2va (text) done; vision tower LOADS real `visual.*` + runs; merged→prompt_embeds scatter + DeepStack→device text tower WIRED 1:1 + gated (§8.9); fl2va COHERENT via BOTH the VAE-keyframe AND the encoder vision path; ref2va reference-row assembly FIXED + gated (§8.10, the block-dim double-division) — but ref2va still grids, now RE-ATTRIBUTED to the ref2va NVFP4 CHECKPOINT/loader (t2va-zero-assembly grids too), NOT the assembly | **PARTIAL** (vision→conditioning scatter + ref2va assembly DONE; residual = the NVFP4 DiT loader for the ref2va checkpoint, §8.10) |
 | Output: joint video+audio, 24 fps, 32 kHz stereo | `pipeline:106-111,1187` | frames + WAV + MP4 mux (W7) | **DONE** |
 | Scheduler: euler-ancestral rectified flow (single) | `scheduling_...euler_ancestral.py`; `time_request.py:34-61` | `MiniMaxH3EulerEta0Step` / `MiniMaxH3TimeShiftSigmas` | **DONE** |
 | CFG: distilled, no CFG (guidance params accepted+ignored; `cfg_parallel_size==1`) | `pipeline:250,275-276` | no CFG branch | **DONE** (matches) |
@@ -774,3 +774,73 @@ the un-pinned target rows on them), NOT the prompt_embeds and NOT the DiT forwar
 dump the ref2va target-row VAE-input latent adjacency-cosine (like #77 did for the coherent fl2va,
 0.95) to confirm the target rows are white, and A/B the reference-row condition-noise vs a clean
 anchor.
+
+## 8.10 REF2VA ASSEMBLY — a real assembly bug FIXED + gated, and the grid RE-ATTRIBUTED to the CHECKPOINT (2026-08-07, `row/H3-REF2VA-ASSEMBLY` PR #93)
+
+Took §8.9's residual (the ref2va reference-row assembly) at face value, diffed the whole assembly
+against upstream on CPU, found + fixed a real bug, extended the permanent gate, then re-rendered on
+real weights — and the render **DISPROVED §8.9's attribution**: the grid is NOT the assembly.
+
+**The real assembly bug FOUND + FIXED (suspect #1, `MiniMaxH3EncodeReferenceImages`).**
+`MiniMaxH3EncodeReferenceImages` (`minimax_h3_pipeline.cpp:162`) and `MiniMaxH3EncodeReferenceVideo`
+(`:83`) populated the `MiniMaxH3RefBlock` with **PATCHED** dims (`ls.h / patch_size_h`), but
+`BuildMiniMaxH3PackedSequenceRef2va` (`minimax_h3_packing.cpp:453`) divides `block.latent_h / kPatchH`
+**again** — it mirrors upstream `minimax_h3_packed_sequence_ref2va_blocks` (`packed_sequence.py:328-330`),
+which takes the **UNPATCHED** latent and divides by `_PATCH_H` once (upstream feeds it the raw latent:
+`pipeline_minimax_h3.py:1141-1145` sets `visual_shape = (1, height//16, width//16)`, and the target
+`latent_h = height//16` is fed to the same builder). The double-division under-allocated the reference
+span by `patch_h*patch_w` (=4): the layout claimed **1/4** the reference rows the VAE encode produced,
+and the denoise loop's `keyframe_cond_rows.size() >= …` check is a `>=`, so the oversized encoded
+reference was **silently truncated to its first quarter** — no throw, a coherence-mangling layout bug.
+Fix: emit the RAW `ls.{t,h,w}` in both encode functions (byte-for-byte upstream's `visual_shape`). The
+other three suspects were CLEARED by the same read: the per-token timesteps mirror `denoise_loop.py:109-118`
+exactly (pinned refs at `max(t_v,0.999)` / `max(t_a,1.0)` — `minimax_h3.cpp:843-854`); the packed layout,
+position grid and token tags are byte-exact vs upstream (gated below); the #77 output bookkeeping is
+correct (upstream `video_rows[update_mask]` == our trailing-target slice, refs front-loaded).
+
+**Permanent gates (the recurring blind-spot class closed).**
+1. `scripts/gen-minimax-h3-goldens.py` grew **section 5c**: a REF2VA-shaped DiT-forward rung built from
+   the upstream `minimax_h3_packed_sequence_ref2va_blocks` (an image reference + a video+audio reference
+   prepended to the target) at 8×8 spatial geometry, forwarded through `RefDiT` with the ref2va per-token
+   timestep partition (target `t_v`/`t_a`, pinned visual refs at 0.999, pinned audio refs at 1.0) and its
+   audio update mask. The C++ case `test_minimax_h3.cpp :: "DiT-forward REF2VA rung matches upstream
+   (reference rows, mixing)"` rebuilds it with `BuildMiniMaxH3PackedSequenceRef2va` and gates the layout,
+   host+device video/audio logits (≤2e-5), the reference-row output masking, and the target-row
+   spatial-mixing probe (mix_fraction 1.0). No prior DiT-forward gate ever forwarded a ref2va layout —
+   the §8.9 blind spot. Needed a 1-line RefDiT extension (honor `audio_update_mask`, mirroring
+   `minimax_h3_transformer.py:1099-1101`; our port already did). Goldens regen is purely additive (635
+   inserts, 0 deletes — no drift).
+2. A RED-first **encoded-vs-layout row-count invariant** in the ref2va image + video subcases: after
+   `MiniMaxH3EncodeReferenceImages`, `ref_visual_rows * video_row_width == ref_rows.size()`. With the bug
+   reintroduced it fails `128 == 512` (16 encoded rows vs 4 allocated — the exact 4× patch double-count);
+   restored → green. Nothing else in the suite couples encoded rows to layout rows (the section-2 layout
+   gate hand-builds unpatched blocks; the denoise round-trip only checks finiteness/motion). Suite
+   **69/69, 52377 assertions** (CPU), the fix RED→GREEN proven.
+
+**GB10 render A/B + isolation (2026-08-07, dgx sm_121a, 256×256/22f/12steps, fixed binary).**
+- **ref2va, fp4-resident** (`--ref-image` + `--cond-image` + `--partition ref2va`): still a multicolour
+  patch grid. `~/h3fp4/out_vs_ref2va.mp4`.
+- **ref2va, bf16** (same, NVFP4→bf16 dequant, no `--fp4-resident`): **also grids** → fp4 is NOT the cause.
+  `~/h3fp4/out_bf16_ref2va.mp4`.
+- **t2va with ZERO reference assembly** (ref2va NVFP4, bf16, `--partition fl2va` to bypass the guard, no
+  refs, no cond-image, plain text prompt): **still grids** → the reference-row assembly is NOT the cause;
+  the grid appears with the entire assembly removed. `~/h3fp4/out_t2va_nvfp4.mp4`.
+- **fl2va control** (FL2VA GGUF `--dequant-bf16` + keyframe + `--cond-image`): **COHERENT** — frame 0 a
+  photorealistic orange cat matching the keyframe, frame 21 the cat on a windowsill in warm sunlight
+  (evolved toward the prompt). No regression from the fix. `~/h3fp4/out_vs_fl2va.mp4`.
+
+**RE-ATTRIBUTION (corrects §8.9 and §8.6).** The grid is **NOT the ref2va reference-row assembly** and
+**NOT fp4**. It correlates 1:1 with the **ref2va NVFP4 checkpoint** (`minimax_h3_ref2va_nvfp4_full`):
+EVERY render that loads it grids (t2va #74, ref2va-fp4, ref2va-bf16, t2va-zero-assembly), while EVERY
+render that loads the FL2VA GGUF is coherent (t2va #77, fl2va #90, fl2va here). §8.9's A/B varied only the
+prompt and §8.6's only the task — neither ever varied the checkpoint or the quant, so both misattributed
+a checkpoint/loader defect (to "assembly" and to "wrong partition"). The DiT MATH, the ref2va assembly
+math, and the vision scatter are now all gate-proven correct vs upstream; the real residual is the
+**NVFP4 DiT LOADER for this checkpoint** (`StreamMiniMaxH3Nvfp4ToDeviceBf16/Fp4`, `minimax_h3_nvfp4*`) —
+suspect the fp32-island preservation (patch/time/output layers, `minimax_h3_transformer.py:898-904`),
+the `weight_scale_2` double-dequant, or the tensor-name mapping for this specific 1051-tensor file. The
+synthetic-NVFP4 gates (§8.4/§8.6) proved the dequant MATH byte-exact but never loaded THIS file end to
+end against a coherent oracle. Next: a REF2VA GGUF (bf16, known-good loader) as the checkpoint oracle —
+if ref2va-on-REF2VA-GGUF is coherent, the NVFP4 loader for this file is the bug; else the checkpoint
+itself. The REF2VA GGUF download is dgx-disk-blocked (23 GiB free, 100% full; large dirs belong to other
+campaigns — not prunable).
diff --git a/.agents/state.md b/.agents/state.md
@@ -40304,3 +40304,50 @@ cosine (vs #77's 0.95 for coherent fl2va) + A/B the reference-row condition-nois
 
 Box left clean (renders exited, GPU idle, gpu.lock released, worker stays parked, temp PNGs pruned,
 ckpts kept). Records: spec §8.9 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW.
+
+## 2026-08-07T05:30 - MiniMax-H3 ref2va assembly bug FIXED + gated; render grid RE-ATTRIBUTED to the ref2va NVFP4 CHECKPOINT (NOT the assembly) (row/H3-REF2VA-ASSEMBLY, helper, PR #93)
+<!-- state: 2026-08-07T05:30 -->
+
+Took §8.9's residual (the ref2va reference-row assembly) and ran the §70 latent-bisection discipline at
+the assembly boundary: diffed the whole assembly vs upstream on CPU, found + fixed a real bug, extended
+the permanent gate to a ref2va-shaped rung, re-rendered on real weights — and the render DISPROVED
+§8.9's attribution.
+
+**REAL BUG FIXED (suspect #1).** `MiniMaxH3EncodeReferenceImages` (minimax_h3_pipeline.cpp:162) and
+`MiniMaxH3EncodeReferenceVideo` (:83) emitted the ref block with PATCHED dims (`ls.h/patch_size_h`), but
+`BuildMiniMaxH3PackedSequenceRef2va` (minimax_h3_packing.cpp:453) divides `block.latent_h/kPatchH` AGAIN,
+mirroring upstream packed_sequence.py:328-330 which takes the UNPATCHED latent (upstream feeds the raw
+latent: pipeline_minimax_h3.py:1141-1145 `visual_shape=(1,height//16,width//16)`). Double-division
+under-allocated the reference span by patch_h*patch_w (=4); the denoise pin-loop's `>=` check let the
+oversized encoded reference be SILENTLY TRUNCATED to its first quarter. Fix: emit raw `ls.{t,h,w}` in
+both encode fns. The other 3 suspects CLEARED by the same read: per-token timesteps mirror
+denoise_loop.py:109-118 exactly (minimax_h3.cpp:843-854); layout/grid/tags byte-exact vs upstream; #77
+output bookkeeping correct (upstream `video_rows[update_mask]` == our trailing-target slice).
+
+**GATES.** (1) goldens section 5c: a ref2va-shaped DiT-forward rung (image + video+audio reference blocks,
+8×8 geometry) forwarded through RefDiT with the ref2va timestep partition + audio update mask; C++ case
+"DiT-forward REF2VA rung matches upstream (reference rows, mixing)" gates layout + host/device logits
+(≤2e-5) + ref-row masking + target mixing (frac 1.0). Needed a 1-line RefDiT extension (honor
+audio_update_mask, upstream minimax_h3_transformer.py:1099-1101; our port already did). Goldens regen
+purely additive (635 inserts, 0 deletes). (2) RED-first encoded-vs-layout row-count invariant in the
+ref2va image+video subcases: reintroducing the bug fails 128==512 (16 encoded vs 4 allocated), restored →
+green. Suite 69/69, 52377 assertions (CPU Release, standalone — no line-3503 flake).
+
+**GB10 render A/B + ISOLATION (256×256/22f/12steps, fixed binary, incremental build in ~/h3fp4/src).**
+ref2va fp4-resident → grids; ref2va bf16 (no --fp4-resident) → ALSO grids (fp4 eliminated); t2va with
+ZERO reference assembly on the ref2va NVFP4 (--partition fl2va bypass, no refs/cond-image) → STILL grids
+(assembly eliminated); fl2va control (FL2VA GGUF + keyframe + --cond-image) → COHERENT (frame 0 orange
+cat matching keyframe, frame 21 cat on a windowsill in warm sunlight — no regression from the fix).
+Artifacts ~/h3fp4/out_{vs_ref2va,bf16_ref2va,t2va_nvfp4,vs_fl2va}.mp4.
+
+**RE-ATTRIBUTION (corrects §8.9 + §8.6).** The grid is NOT the assembly and NOT fp4. It correlates 1:1
+with the ref2va NVFP4 CHECKPOINT (minimax_h3_ref2va_nvfp4_full): every render loading it grids, every
+FL2VA-GGUF render is coherent. §8.9 varied only the prompt; §8.6 only the task — neither varied the
+checkpoint/quant, so both misattributed a checkpoint/loader defect. True residual = the NVFP4 DiT LOADER
+for this file (StreamMiniMaxH3Nvfp4ToDeviceBf16/Fp4): suspect fp32-island preservation (patch/time/output
+layers, minimax_h3_transformer.py:898-904), weight_scale_2 double-dequant, or tensor-name mapping for the
+1051-tensor file. Synthetic-NVFP4 gates proved the dequant MATH byte-exact but never loaded THIS file vs
+a coherent oracle. NEXT: a REF2VA GGUF (bf16, known-good loader) as checkpoint oracle — blocked on dgx
+disk (23 GiB free, 100% full; large dirs belong to other campaigns, not prunable). Fix + gates LAND
+regardless; box left clean (renders exited, gpu.lock released, worker stays parked, ckpts kept). Records:
+spec §8.10 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW.