feat(minimax-h3): the ORIGINAL bf16 DiT release is indexable — 13 shards, 66.3 GB

.agents/NOW.md

@@ -17,7 +17,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | Laguna NVFP4 / DeepSeek-V4 decode | **Both CLOSED, byte-exact, default-ON**: 1.03x vLLM, 1.144x ds4 | Laguna vLLM K-run when convenient |
 | f32-out GEMV audit | Only laguna + ds4 bf16 tower affected; gate models unaffected | Re-verify ds4 tower same-tool |
 | Invocation-parity prevention | CI guard + checklist landing | Merge; build-verify `kGemvHeuristicAlgos` on dgx |
-| MiniMax-H3 lane | **fl2va COHERENT; ref2va NVFP4 grid DIAGNOSED (#95): NO loader bug** | weights/islands/RoPE all quant-noise-close to coherent GGUF; residual = community-NVFP4 quant fidelity §8.12 |
+| MiniMax-H3 lane | **fl2va COHERENT; ref2va grid DIAGNOSED (#95): NO loader bug; bf16 13-shard DiT INDEXES** | residual = community-NVFP4 quant fidelity §8.12; no bf16 render yet |
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | bf16 knobs **106→120/128**, NOT STRICT (§14, `row/KIMI-LINEAR-STRICT-SPEED`); default OFF | residual = device islands; 1.30 tok/s |
 | 35B fresh grid | **BOUND** @`1ea26427`: 0.93-1.03x, c16 0.93x. INTAKE + Option A both NEGATIVE | Lever left: prefill glue (#61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |

.agents/parity-ledger.md

@@ -916,3 +916,4 @@ Columns:
 | 2026-08-06 (`row/H3-FP4-SPEED`; `ROAD-V1-H3`; model `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit`; lifecycle unchanged) | **MiniMax-H3 W-FP4a — fp4-RESIDENT NVFP4 routing for the device DiT forward (NO new quant code).** Until now both NVFP4 loaders dequantized the packed FP4 projections to bf16 and the device forward ran `vt::MatmulBT`, so the sm_121a FP4 tensor-core route never ran for H3. Adds `Nvfp4Weight` carriers to `MiniMaxH3DitBlockWeights`/`MiniMaxH3DitWeights`, a fp4-resident streamer `StreamMiniMaxH3Nvfp4ToDeviceFp4` (keeps the compressed-tensors triple host-resident; the shared dispatcher uploads+repacks lazily then frees the fp4 originals, so peak device memory is ~1/4 of the bf16 arm), and a `LinearDev` dispatch that routes a non-Empty fp4 projection through `dense_nvfp4::MatmulNvfp4W4A16D`. | The routing is vLLM's OWN forced-Marlin-for-a16 selection: the checkpoint is weight-only NVFP4 (no `input_activations`, `IsTrueW4A4()==false`), so `kernels/linear/__init__.py:879-881` forces the Marlin W4A16 kernel, mirrored by `include/vllm/model_executor/models/dense_nvfp4_gemm.h:12-22,505-549` (`MatmulNvfp4W4A16D` -> single-expert `vt::MoeGroupedGemmNvfp4Marlin`, the SAME kernel Laguna routed-experts + dense Qwen3-32B NVFP4 use). Not cutlass-fp4/W4A4 (needs fp4 activations, private to `qwen3_5.cpp`). fc1 is already merged `[gate;up]` -> one W4A16 GEMM + `vt::SiluAndMul`. | **CPU-GATED (wiring), verified**: `test_minimax_h3` 62/62 cases / 30039 assertions, 0 failed. The synthetic-NVFP4 case streams the fp4 twin, asserts the loader kept the projections PACKED (fp4 slot set / bf16 slot Empty, and the inverse for the bf16 loader), runs fp4 + bf16 device forwards on the SAME file, asserts the W4A16 dispatcher executed ALL 11 quantized GEMMs (the `Nvfp4W4A16Stats` this-path-ran counter), and bounds fp4-vs-bf16 <= 2e-3. On CPU the dispatcher has no Marlin op so it falls to the bf16 arm's own dequant+matmul (hence a WIRING gate here); the Marlin kernel numerics are CUDA-gated independently by `test_ops_nvfp4_matmul` / `test_linear_method` (2e-3 f32-out / 8e-3 bf16-out vs a bf16 reference). `benchmark_binding=false`. PENDING: GB10 CUDA build + the fp4-vs-bf16 numeric delta and steady per-step timing (disk/build window); real-checkpoint t2va e2e DISK-BLOCKED (~41 GB working set). Comparability: vLLM-Omni serves NO quantized H3 (BF16-only in practice; source-audited `a4ea67a2`, spec §8.3) -> HW/loader-forced-indirect. |
 | 2026-08-06 (startup-latency axis becomes measurable; extends `SERVE-GATE-ONLINE`; no new row; isolated worktree `.claude/worktrees/startup-axis` branch `feat/startup-latency-axis` off `362a3c99`; CPU-only, NO GPU; `benchmark_binding=false`) | **What it does.** Makes cold launch-to-first-`/health` a recordable axis on both arms. `scripts/dgx-online-serving.sh` gains `--startup-only` (3 interleaved ours/vLLM legs under one `/tmp/gpu` lock, page cache dropped per leg, GPU idle proven before/after, no timed client, `server` target only); `wait_ready` moves from a 5 s cadence / `seq 1 360` to a 0.2 s cadence on a deadline preserving the identical 1800 s budget; `start_server` stamps launch and ready immediately around the spawn; new `online_gate.py record-startup` + `summarize-startup`. The 5 s cadence was ~12% of a ~40 s startup, which is why this axis was never reportable despite being in the gate protocol. | **Not a vLLM mirror** — measurement tooling, recorded as such (`porting-inventory.md` §9). vLLM has no startup-latency test to port. The measured definition is deliberately stack-inclusive: whatever each engine really does before `/health` answers (vLLM: torch import, engine init, flashinfer JIT, graph capture; ours: weight load, graph capture). | **PASS (harness correctness; NO throughput number owed or claimed).** RED-first: the new suite failed on `cannot import name 'record_startup'` before the implementation existed. `tests/tools/test_online_gate_startup.py` **19/19**; run together with `test_online_gate_client`, `test_online_gate_summary`, `test_online_gate_trace`, `test_drop_file_cache` -> **66/66**. `shellcheck` + `bash -n` clean. The existing `--execute` purity contract CAUGHT a real regression (a second `for repetition in 1 2 3; do` header broke its split anchor) — fixed by renaming the loop variable. Cold-gate: `record-startup` validates and embeds the leg's cache-drop report, so a warm leg cannot produce an artifact. **This change also REPAIRS main**: squash `b95543c4` (#77) swept the half-written test file onto main without its implementation, leaving the tool suite red. **RESIDUAL (honest): NO ours-vs-vLLM startup number exists.** dgx.casa was at 100% disk with no CUDA `server` build on 2026-08-06; the 27B 3-repetition run is owed. |
 | 2026-08-07 (startup-latency FIRST NUMBERS, provisional; extends `SERVE-GATE-ONLINE`; no new row; `benchmark_binding=false`) | **What it does.** Runs the `--startup-only` series landed the day before: Qwen3.6-27B-NVFP4, GB10, 3 interleaved ours/vLLM repetitions under one `/tmp/gpu` lock, page cache dropped per leg, GPU idle proven before and after each. | Reference = vLLM oracle 0.25.0 in the same production server config the throughput grid launches (`--gpu-memory-utilization 0.6`, matched `--max-num-seqs`/`--max-num-batched-tokens`, prefix caching off). Attribution taken from vLLM's OWN log, not inferred. | **MEASURED, PROVISIONAL, NOT BINDING.** ours 37.94/36.51/35.88 s (median **36.51**), vLLM 460.36/221.51/217.86 s (warm median **221.51**) => **6.07x**. Ours ±3%; vLLM warm legs within 1.7%. vLLM r1's 460 s is one-time FlashInfer autotune+compile (`saved 64 configs`, `init engine 259.42 s`) vs r2 (`loaded 64 configs`, `26.95 s`); even warm, init is only ~27 s of ~221 s, so the gap is process start + imports + weight load. Our cold-autotune start = 69.29 s (+33 s over warm). **Two reasons it is not binding:** (a) a concurrent build session overlapped r2/r3 of both arms, including BOTH warm-cache vLLM legs, biasing vLLM slow and inflating the ratio; (b) the uncontended repeat was destroyed when the box HARD-REBOOTED mid-leg during vLLM's cold-autotune start (previous boot's journal ends with no shutdown sequence) - a NEW trigger for the known GB10 unified-memory reboot hazard, since the warm-cache config ran six times without incident. Owed: one uncontended 3-rep series on a quiet box. |
+| 2026-08-07 (`row/H3-BF16-SHARDED-DIT`; `ROAD-V1-H3`; model `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit`; CPU-only, no GPU and no download; lifecycle unchanged) | **MiniMax-H3 — the ORIGINAL bf16 release (13 safetensors shards, 66.3 GB) is now INDEXABLE.** Every H3 render so far used a QUANTIZED DiT and H3 is unusually quantization-sensitive (Q3_K_M -> Q4_K_M alone turned a murky lattice into a photoreal close-up; ComfyUI PR 15298 blames the partial split-half RoPE's channel-wise magnitude outliers), but the full-precision question was unaskable because every DiT loader took a SINGLE file. Adds (a) `MiniMaxH3ShardedCheckpoint::Open(dir)` (`src/vllm/model_executor/models/minimax_h3_sharded.cpp`), which resolves tensors through the checkpoint's own `model.safetensors.index.json` weight map (never by scanning) with one index over every shard, mirroring the in-tree multi-shard template `LoadMiniMaxH3EncoderWeights(const std::vector<SafetensorsFile>&, ...)`, and throws BY NAME when the index names a tensor its shard does not hold; (b) `EnumerateMiniMaxH3ShardedTensors`, the shapes-only manifest the geometry parser consumes; (c) `LoadMiniMaxH3DitFromShards`, the host-f32 reference loader; (d) `MiniMaxH3IsFp32IslandTensor`, single-sourcing the upstream fp32-ISLAND split the three existing streamers each hand-rolled; (e) `--dit <dir>` in `examples/minimax_h3_gen` for both `--dump-params` and the run path, every existing `--dit` form unchanged. The DEVICE streamer is the stacked follow-up `row/H3-BF16-SHARDED-STREAM`, split out to stay inside the 900-line PR cap. | vLLM-Omni `vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py:85-101` (MINIMAX_H3_FP32_PARAM_NAMES / _BUFFER_NAMES, the island split) and `:906-922` (the parameter set); the shard-index container convention is HF safetensors' own `model.safetensors.index.json` weight_map, already consumed in-tree by `LoadSafetensorsIndex` and the multi-shard encoder/VAE loaders. No vLLM behavior changed; H3 remains BEYOND-PIN (vllm-omni, not the pinned vLLM repo). | **LANDED + CPU-GATED (loader brick; `benchmark_binding=false` — no throughput owed, and NO bf16-vs-quant render or speed number is claimed).** Re-gated AFTER the rebase onto `f34e0d17`: `test_minimax_h3` 72/72 cases / 54497 assertions, clean Release build of `libvllm.a`, `test_minimax_h3` and `minimax-h3-gen`. Two gates: (1) index+name mapping over a synthetic 4-shard set — every tensor resolves to the shard the index named AND to the bytes written there, a tensor missing from its shard throws WITH ITS NAME, and the derived geometry equals the single-file path field for field; (2) a SPARSE 13-shard release declaring the REAL 535 tensors at REAL shapes (66.3 GB declared, 144 KB on disk) derives the SHIPPED geometry 50/5376/56/128/14336/24/32/1x2x2/5120, and `minimax-h3-gen --dit <dir> --dump-params` prints all 20 fields on it. Also FIXES a real latent defect this row's sanitizer lane exposed: `MiniMaxH3ReadSafetensorF32` read 16-bit payloads through `reinterpret_cast<const uint16_t*>`, which is UB on a safetensors file whose JSON header leaves the payload odd-aligned (the format does not require padding); now a byte-wise `memcpy`. RED-first proven: reverting it reproduces UBSan's `load of misaligned address` at the same line and exits 1. Honest residuals: no device load of the real 66.3 GB release, no measured peak RSS, and no bf16-vs-quantized render/speed comparison — the quality question is UNBLOCKED, not answered. |