feat(minimax-h3): stream the ORIGINAL bf16 DiT to the device, one tensor at a time

.agents/NOW.md

@@ -17,7 +17,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | Laguna NVFP4 / DeepSeek-V4 decode | **Both CLOSED, byte-exact, default-ON**: 1.03x vLLM, 1.144x ds4 | Laguna vLLM K-run when convenient |
 | f32-out GEMV audit | Only laguna + ds4 bf16 tower affected; gate models unaffected | Re-verify ds4 tower same-tool |
 | Invocation-parity prevention | CI guard + checklist landing | Merge; build-verify `kGemvHeuristicAlgos` on dgx |
-| MiniMax-H3 lane | **fl2va COHERENT; ref2va grid DIAGNOSED (#95): NO loader bug; bf16 13-shard DiT INDEXES** | residual = community-NVFP4 quant fidelity §8.12; no bf16 render yet |
+| MiniMax-H3 lane | **fl2va COHERENT; ref2va grid DIAGNOSED (#95): NO loader bug; bf16 13-shard DiT STREAMS** | residual = community-NVFP4 quant fidelity §8.12; no bf16 render yet |
 | Kimi-Linear-48B (KDA/NoPE-MLA/MoE) | device-KDA **122/128, 4.24 tok/s** best (§15); MLA device NEG (§16). chunk_kda prefill AOT **SPIKED**: 5 kernels authored+pinned+recipe (§17). Bar = MEET vLLM speed | Phase-2: regen harness, wire `vt::KdaChunkPrefill`, gate STRICT + vLLM 0.82 ladder |
 | 35B fresh grid | **BOUND** @`1ea26427`: 0.93-1.03x, c16 0.93x. INTAKE + Option A both NEGATIVE | Lever left: prefill glue (#61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |

.agents/parity-ledger.md

@@ -917,3 +917,4 @@ Columns:
 | 2026-08-06 (startup-latency axis becomes measurable; extends `SERVE-GATE-ONLINE`; no new row; isolated worktree `.claude/worktrees/startup-axis` branch `feat/startup-latency-axis` off `362a3c99`; CPU-only, NO GPU; `benchmark_binding=false`) | **What it does.** Makes cold launch-to-first-`/health` a recordable axis on both arms. `scripts/dgx-online-serving.sh` gains `--startup-only` (3 interleaved ours/vLLM legs under one `/tmp/gpu` lock, page cache dropped per leg, GPU idle proven before/after, no timed client, `server` target only); `wait_ready` moves from a 5 s cadence / `seq 1 360` to a 0.2 s cadence on a deadline preserving the identical 1800 s budget; `start_server` stamps launch and ready immediately around the spawn; new `online_gate.py record-startup` + `summarize-startup`. The 5 s cadence was ~12% of a ~40 s startup, which is why this axis was never reportable despite being in the gate protocol. | **Not a vLLM mirror** — measurement tooling, recorded as such (`porting-inventory.md` §9). vLLM has no startup-latency test to port. The measured definition is deliberately stack-inclusive: whatever each engine really does before `/health` answers (vLLM: torch import, engine init, flashinfer JIT, graph capture; ours: weight load, graph capture). | **PASS (harness correctness; NO throughput number owed or claimed).** RED-first: the new suite failed on `cannot import name 'record_startup'` before the implementation existed. `tests/tools/test_online_gate_startup.py` **19/19**; run together with `test_online_gate_client`, `test_online_gate_summary`, `test_online_gate_trace`, `test_drop_file_cache` -> **66/66**. `shellcheck` + `bash -n` clean. The existing `--execute` purity contract CAUGHT a real regression (a second `for repetition in 1 2 3; do` header broke its split anchor) — fixed by renaming the loop variable. Cold-gate: `record-startup` validates and embeds the leg's cache-drop report, so a warm leg cannot produce an artifact. **This change also REPAIRS main**: squash `b95543c4` (#77) swept the half-written test file onto main without its implementation, leaving the tool suite red. **RESIDUAL (honest): NO ours-vs-vLLM startup number exists.** dgx.casa was at 100% disk with no CUDA `server` build on 2026-08-06; the 27B 3-repetition run is owed. |
 | 2026-08-07 (startup-latency FIRST NUMBERS, provisional; extends `SERVE-GATE-ONLINE`; no new row; `benchmark_binding=false`) | **What it does.** Runs the `--startup-only` series landed the day before: Qwen3.6-27B-NVFP4, GB10, 3 interleaved ours/vLLM repetitions under one `/tmp/gpu` lock, page cache dropped per leg, GPU idle proven before and after each. | Reference = vLLM oracle 0.25.0 in the same production server config the throughput grid launches (`--gpu-memory-utilization 0.6`, matched `--max-num-seqs`/`--max-num-batched-tokens`, prefix caching off). Attribution taken from vLLM's OWN log, not inferred. | **MEASURED, PROVISIONAL, NOT BINDING.** ours 37.94/36.51/35.88 s (median **36.51**), vLLM 460.36/221.51/217.86 s (warm median **221.51**) => **6.07x**. Ours ±3%; vLLM warm legs within 1.7%. vLLM r1's 460 s is one-time FlashInfer autotune+compile (`saved 64 configs`, `init engine 259.42 s`) vs r2 (`loaded 64 configs`, `26.95 s`); even warm, init is only ~27 s of ~221 s, so the gap is process start + imports + weight load. Our cold-autotune start = 69.29 s (+33 s over warm). **Two reasons it is not binding:** (a) a concurrent build session overlapped r2/r3 of both arms, including BOTH warm-cache vLLM legs, biasing vLLM slow and inflating the ratio; (b) the uncontended repeat was destroyed when the box HARD-REBOOTED mid-leg during vLLM's cold-autotune start (previous boot's journal ends with no shutdown sequence) - a NEW trigger for the known GB10 unified-memory reboot hazard, since the warm-cache config ran six times without incident. Owed: one uncontended 3-rep series on a quiet box. |
 | 2026-08-07 (`row/H3-BF16-SHARDED-DIT`; `ROAD-V1-H3`; model `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit`; CPU-only, no GPU and no download; lifecycle unchanged) | **MiniMax-H3 — the ORIGINAL bf16 release (13 safetensors shards, 66.3 GB) is now INDEXABLE.** Every H3 render so far used a QUANTIZED DiT and H3 is unusually quantization-sensitive (Q3_K_M -> Q4_K_M alone turned a murky lattice into a photoreal close-up; ComfyUI PR 15298 blames the partial split-half RoPE's channel-wise magnitude outliers), but the full-precision question was unaskable because every DiT loader took a SINGLE file. Adds (a) `MiniMaxH3ShardedCheckpoint::Open(dir)` (`src/vllm/model_executor/models/minimax_h3_sharded.cpp`), which resolves tensors through the checkpoint's own `model.safetensors.index.json` weight map (never by scanning) with one index over every shard, mirroring the in-tree multi-shard template `LoadMiniMaxH3EncoderWeights(const std::vector<SafetensorsFile>&, ...)`, and throws BY NAME when the index names a tensor its shard does not hold; (b) `EnumerateMiniMaxH3ShardedTensors`, the shapes-only manifest the geometry parser consumes; (c) `LoadMiniMaxH3DitFromShards`, the host-f32 reference loader; (d) `MiniMaxH3IsFp32IslandTensor`, single-sourcing the upstream fp32-ISLAND split the three existing streamers each hand-rolled; (e) `--dit <dir>` in `examples/minimax_h3_gen` for both `--dump-params` and the run path, every existing `--dit` form unchanged. The DEVICE streamer is the stacked follow-up `row/H3-BF16-SHARDED-STREAM`, split out to stay inside the 900-line PR cap. | vLLM-Omni `vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py:85-101` (MINIMAX_H3_FP32_PARAM_NAMES / _BUFFER_NAMES, the island split) and `:906-922` (the parameter set); the shard-index container convention is HF safetensors' own `model.safetensors.index.json` weight_map, already consumed in-tree by `LoadSafetensorsIndex` and the multi-shard encoder/VAE loaders. No vLLM behavior changed; H3 remains BEYOND-PIN (vllm-omni, not the pinned vLLM repo). | **LANDED + CPU-GATED (loader brick; `benchmark_binding=false` — no throughput owed, and NO bf16-vs-quant render or speed number is claimed).** Re-gated AFTER the rebase onto `f34e0d17`: `test_minimax_h3` 72/72 cases / 54497 assertions, clean Release build of `libvllm.a`, `test_minimax_h3` and `minimax-h3-gen`. Two gates: (1) index+name mapping over a synthetic 4-shard set — every tensor resolves to the shard the index named AND to the bytes written there, a tensor missing from its shard throws WITH ITS NAME, and the derived geometry equals the single-file path field for field; (2) a SPARSE 13-shard release declaring the REAL 535 tensors at REAL shapes (66.3 GB declared, 144 KB on disk) derives the SHIPPED geometry 50/5376/56/128/14336/24/32/1x2x2/5120, and `minimax-h3-gen --dit <dir> --dump-params` prints all 20 fields on it. Also FIXES a real latent defect this row's sanitizer lane exposed: `MiniMaxH3ReadSafetensorF32` read 16-bit payloads through `reinterpret_cast<const uint16_t*>`, which is UB on a safetensors file whose JSON header leaves the payload odd-aligned (the format does not require padding); now a byte-wise `memcpy`. RED-first proven: reverting it reproduces UBSan's `load of misaligned address` at the same line and exits 1. Honest residuals: no device load of the real 66.3 GB release, no measured peak RSS, and no bf16-vs-quantized render/speed comparison — the quality question is UNBLOCKED, not answered. |
+| 2026-08-07 (`row/H3-BF16-SHARDED-STREAM`; `ROAD-V1-H3`; model `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit`; stacked on `row/H3-BF16-SHARDED-DIT`; CPU-only, no GPU and no download; lifecycle unchanged) | **MiniMax-H3 — the ORIGINAL bf16 release (13 shards, 66.3 GB) now STREAMS to the device.** §8.13 made the checkpoint indexable but its only loader was host-f32 (~132 GB on the real release); on a 122 GiB UNIFIED pool that holds the model TWICE, and the non-streaming NVFP4 loader was already OOM-killed at anon-rss 125 GB on HALF this size, so the real release was not loadable at all. Adds `StreamMiniMaxH3ShardedToDeviceBf16` (`minimax_h3_device.cpp`, sharing `BindStreamedDitViews` with the GGUF and NVFP4 streamers): manifest first, then ONE tensor at a time, with a BF16-on-disk tensor bound for a bf16 device slot — essentially the whole 66.3 GB — uploaded DIRECTLY out of the read-only mmap with NO host buffer, each source range released via `MaybeReleaseSourcePages`, and `rope.inv_freq` kept HOST-resident. Adds `MiniMaxH3ShardStreamStats` (mirroring `Nvfp4W4A16Stats`) so the path is observable, and `--dit <dir> --device cuda` in `examples/minimax_h3_gen`. | vLLM-Omni `vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py:85-101` (MINIMAX_H3_FP32_PARAM_NAMES / _BUFFER_NAMES, the fp32-island split the stream honours). The streaming SHAPE is our own in-tree convention (`StreamMiniMaxH3Nvfp4ToDeviceBf16`), which exists because upstream never has to load this checkpoint on one unified-memory device; recorded as a deviation in porting-inventory §9 terms. No vLLM behavior changed; H3 remains BEYOND-PIN. | **LANDED + CPU-GATED (loader brick; `benchmark_binding=false` — no throughput owed, and NO bf16-vs-quant render or speed number is claimed).** `test_minimax_h3` 73/73 cases / 55203 assertions, clean Release build of `libvllm.a`, `test_minimax_h3` and `minimax-h3-gen`. Two gates: (1) streamed == non-streamed — all 46 weight views BIT-EXACT (`memcmp == 0`) vs `StageMiniMaxH3DitWeights(kBF16)`, dtypes included (12 fp32 islands), both device forwards IDENTICAL (video and audio max|diff| == 0.0), `rope.inv_freq` host-resident; (2) the loader RAN — counters ASSERTED, observed `shards=3 tensors=46 direct=37 converted=9 bytes=444504 host_peak=8192`, i.e. BOTH upload paths taken, every view owned by this loader, and `host_peak_bytes` bounded by one tensor (< 1/4 of bytes uploaded) so the peak cannot scale with the model. Honest residuals: the real 66.3 GB load, its measured peak RSS, and CUDA memcpy from a file-backed mmap are all UNVERIFIED (CPU-only row); the bf16-vs-quant A/B is now RUNNABLE and has not been run. |