feat(minimax-h3): stream the ORIGINAL bf16 DiT to the device, one tensor at a time

[localai-bot]

Stacked on #98 (row/H3-BF16-SHARDED-DIT). Base it on main only after #98 merges — the diff below is this row alone (437 non-exempt lines).

What this enables

#98 made the 13-shard 66.3 GB bf16 release indexable, but its only loader was host-f32 — ~132 GB on the real release. This box has 122 GiB of unified memory: host and device draw on one pool, so "load to host, then stage" holds the model twice against that budget, and the non-streaming NVFP4 loader was already OOM-killed at anon-rss 125 GB on half this size. Without a streaming path the original release is not loadable at all.

StreamMiniMaxH3ShardedToDeviceBf16(queue, ckpt, out_params) lands in minimax_h3_device.cpp, beside its GGUF and NVFP4 twins so it reuses BindStreamedDitViews. Manifest first (names+shapes, no payload) so the geometry is known and a wrong name map is caught before a byte is allocated, then one tensor at a time.

Peak host memory, precisely:

• BF16 on disk → bf16 device slot — essentially the whole 66.3 GB — is uploaded directly out of the read-only mmap, with no host buffer at any point. Same for F32 → f32 island.
• Only a dtype mismatch (a BF16 island widened to f32, an F32 body rounded to bf16, an F16 shard) costs one tensor's conversion buffer, freed before the next iteration.
• Each source range goes to MaybeReleaseSourcePages the moment its copy returns, so the page cache does not accumulate against the same pool the weights live in.
• rope.inv_freq stays host-resident: BuildRopeCosSin runs before any kernel, so a device pointer there segfaults on the first forward.

MiniMaxH3ShardStreamStats (mirroring Nvfp4W4A16Stats) makes the path observable. This codebase has shipped a never-executing guarded kernel under a green suite before; counters are how that is prevented. examples/minimax_h3_gen --dit <dir> --device cuda streams and prints the stats line; --device cpu keeps #98's host reference loader.

Gates I ran

• test_minimax_h3 73/73 cases, 55203 assertions, clean Release build of libvllm.a, test_minimax_h3, minimax-h3-gen.
• Streamed == non-streamed, over a 3-shard synthetic set whose dtypes exercise all four (on-disk × device) combinations: all 46 weight views bit-exact (memcmp == 0) vs StageMiniMaxH3DitWeights(kBF16), dtypes included (12 fp32 islands), and both device forwards return identical logits (video and audio max|diff| both exactly 0.0). rope.inv_freq asserted host-resident.
• The loader RAN — counters asserted, not merely printed. Observed: shards=3 tensors=46 direct=37 converted=9 bytes=444504 host_peak=8192. Both upload paths taken, every bound view owned by this loader's staging, and host_peak_bytes (one tensor's buffer) under 1/4 of the bytes uploaded — the peak cannot scale with the model.
• Full scripts/agent-preflight.sh --staged: green.

What is NOT claimed

• No bf16 render exists. No bf16-vs-quantized quality or speed number is claimed. The A/B is now runnable; it has not been run.
• The real 66.3 GB load and its measured peak RSS are unverified, as is CUDA cudaMemcpy straight from a file-backed mmap — valid pageable-source usage, every copy followed by a synchronize, but unexercised on device. This row ran CPU-only: no GPU job, no download.
• The gate's host_peak=8192 is from a reduced synthetic checkpoint. It proves the bound is one tensor, not a number for the real release.

[localai-bot]

Closed automatically by GitHub when its base branch row/H3-BF16-SHARDED-DIT was deleted on #98's merge; GitHub refuses to reopen a PR whose base branch is gone. Continued as #105, same head branch, rebased onto main, same content and same gates.