row/H3-NVFP4-STREAM-DIFF: the #94 NVFP4 ref2va-grid residual DIAGNOSED — no discrete load-path defect - #95
Closed
localai-bot wants to merge 1 commit into
Closed
row/H3-NVFP4-STREAM-DIFF: the #94 NVFP4 ref2va-grid residual DIAGNOSED — no discrete load-path defect#95localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…iscrete load-path defect Ran #94's prescribed identical-weights activation diff (NVFP4-bf16 stream vs the FL2VA-GGUF-bf16 control, byte-identical inputs via an encode-once pe.f32) plus direct WEIGHT fingerprints, through an env-gated per-stage hook in MiniMaxH3DitForwardDevice (VT_H3_ACT_DUMP; byte-inert unset). The result REFUTES the "second NVFP4-render-path defect" hypothesis. Instrument (byte-inert): a reusable H3DumpFingerprint over every embed/scatter/time stage, every block's adaln+post-attn+post-mlp, the final heads, the RoPE cos/sin cache, the block-0 attention internals (qkv/split/qknorm/rope/core/out_proj), and input-independent fingerprints for every island, bias, output head, q/k-norm, and block-0/refiner-0 projection — the classes #94's oracle never sampled. Finding: EVERY weight, bias, fp32 island, output head and q/k-norm loads quant-noise-close to the coherent GGUF (qkv/out/fc/adaln 2-6% rms = Q3K-vs-NVFP4 quantizer variance, sign-correct; all biases <0.1% apart incl adaln_b), and the RoPE cache is BYTE-IDENTICAL — no scramble/transpose/mis-stride/wrong-dtype/ wrong-shape. Both arms run identical code, so the grid is 100% the per-weight NVFP4-vs-Q3K quant delta on the SAME weights. Divergence first appears at the token refiner + the block-0 attention INPUT (not RoPE, not a GEMM/norm weight) and amplifies chaotically through the 50-block stack, driven by the Qwen massive text activation (condition_proj absmax ~7.4e4); final latents are DECORRELATED (sample-rel-L2 >1, not scale). Render A/B re-confirmed in the same byte-inert build: NVFP4 t2va = pale patch grid, FL2VA-GGUF t2va = coherent orange cat. Conclusion: not a loader fix. The residual is the community NVFP4 checkpoint's quantization fidelity (Star Ultimate Model Converter Pro lineage; corr 0.85-0.94 to the coherent Q3_K) times the DiT's massive-activation sensitivity. Clean- reference disambiguation is blocked (bf16 132GiB = OOM on one GB10; a same-finetune REF2VA-GGUF control = 23G-disk-blocked); the path forward is an official modelopt-NVFP4 checkpoint. The #94 nibble fix stands. The fp4-resident Marlin arm's separate grid is untouched (wiring-gated-only residual). Records: spec §8.12 + §8.2 row, STATUS/BENCHMARKS/FEATURES H3 rows, benchmark-record, NOW, state. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
mudler
force-pushed
the
row/H3-NVFP4-STREAM-DIFF
branch
from
August 7, 2026 06:46
c731490 to
37a716e
Compare
localai-bot
changed the title
Collaborator
Author
|
Landed on main as the squash (mudler authorship). The H3 render investigation closes: the engine renders coherently from every well-formed checkpoint; the gridding arm is a defective community quant. 🤖 Generated with Claude Code |
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…quant fidelity (#95) row/H3-NVFP4-STREAM-DIFF squash. Identical-weights activation diff: divergence enters at the token refiner and amplifies chaotically, but the guilty-class audit finds NO loader bug — every tensor class loads quant-noise-close to the coherent GGUF, RoPE byte-identical, no scramble/transpose/stride/dtype defect; both arms run identical code. The grid is the community NVFP4 file's own quantization fidelity (corr 0.85-0.94, same converter lineage as the #94 nibble bug) times the DiT's massive-activation sensitivity. Path forward: an official modelopt-NVFP4 checkpoint, not a loader change. #94 nibble fix stands; the fp4-resident Marlin arm grid stays the named wiring residual. The byte-inert VT_H3_ACT_DUMP fingerprint hook ships as a permanent instrument. Merge also anchors a parallel session's anchorless startup-latency state entry. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
localai-bot
added a commit
that referenced
this pull request
Aug 7, 2026
…rds, 66.3 GB (#98) Branch: row/H3-BF16-SHARDED-DIT (helper; CPU-only, no GPU job, no download). Written 2026-08-06 off 075b9f2, REBASED onto f34e0d1 and re-gated before landing; every number below is the POST-REBASE re-run, not the earlier report. Every H3 render so far used a QUANTIZED DiT, and H3 is unusually quantization-sensitive: Q3_K_M -> Q4_K_M alone turned a murky lattice-covered silhouette into a photoreal close-up (ComfyUI PR 15298 attributes it to the partial split-half RoPE producing channel-wise magnitude outliers that corrupt even INT8). "What does FULL PRECISION look like?" could not be asked, because every DiT loader took a SINGLE file while the bf16 release ships 13 safetensors shards totalling 66.3 GB. This is the CHECKPOINT half. The device streamer the real 66.3 GB load needs is the stacked follow-up row/H3-BF16-SHARDED-STREAM, split out so each PR stays inside the 900-line review cap. - MiniMaxH3ShardedCheckpoint::Open(dir) (new minimax_h3_sharded.cpp) resolves tensors through the checkpoint's own model.safetensors.index.json weight map, never by scanning, with one index over every shard — the shape of the in-tree template LoadMiniMaxH3EncoderWeights(const std::vector<SafetensorsFile>&, ...). A tensor the index NAMES but whose shard does not contain it throws BY NAME; skipping it would read as zeros and render. - EnumerateMiniMaxH3ShardedTensors builds the same names+shapes manifest the GGUF and NVFP4 arms build, so ParseMiniMaxH3DitParamsFromGgufManifest derives the geometry from SHAPES ALONE on a sharded checkpoint too. - LoadMiniMaxH3DitFromShards is the host-f32 REFERENCE loader (comparison baseline and the CPU path for reduced checkpoints; ~132 GB on the real release, so not for real runs), and MiniMaxH3IsFp32IslandTensor single-sources the upstream fp32-ISLAND split the three existing streamers each hand-rolled. - examples/minimax_h3_gen accepts --dit <dir> everywhere it accepted a DiT file, for both --dump-params and the run path; every existing --dit form unchanged. Gates (CPU, re-run post-rebase: test_minimax_h3 72/72 cases / 54497 assertions; clean Release build of libvllm.a, test_minimax_h3 and minimax-h3-gen): (1) index + name mapping over a synthetic 4-shard set — every tensor resolves to the shard the index named AND to the bytes written there, a missing-from-shard tensor throws WITH ITS NAME, and the geometry equals the single-file path field for field; (2) real geometry without the weights — a 13-shard release declaring the REAL 535 tensors at REAL shapes with a SPARSE payload (66.3 GB declared, 144 KB on disk) derives the SHIPPED geometry (50 / 5376 / 56 / 128 / 14336 / 24 / 32 / 1x2x2 / 5120), and `minimax-h3-gen --dit <dir> --dump-params` prints all 20 fields on it. Also repairs two gates that f34e0d1 (#95) left RED on main and that block any PR branched from it: docs/ENVIRONMENT.md now documents VT_H3_ACT_DUMP and VT_H3_ACT_CALL (check-env-doc), and the §9 spec table that ran into the next table without a separator now has one (check-agent-record). Also fixes a real latent defect this row's ASan+UBSan lane exposed: MiniMaxH3ReadSafetensorF32 read 16-bit payloads through reinterpret_cast<const uint16_t*>, which is UB on a safetensors file whose JSON header leaves the payload odd-aligned — the format does not require padding, and LoadMiniMaxH3DitFromShards is the first caller to hand it such a file. Now a byte-wise memcpy load. RED-first proven locally on the same sanitizer build: reverting it reproduces CI's `load of misaligned address ... requires 2 byte alignment` at the same line and exits 1; with it, test_minimax_h3 is 73/73 / 55203 under -fsanitize=address,undefined with ZERO findings. The synthetic shard writer stays deliberately UNPADDED so the case remains covered. Honest residuals: NO device load of the real 66.3 GB release ships here, no measured peak RSS, and no bf16-vs-quantized render or speed number. The bf16-vs-quant quality question is UNBLOCKED, not answered. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Opus 5 (1M context) Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finding: the #94 residual is NOT a discrete load-path defect
Ran #94's prescribed identical-weights layer-by-layer activation diff (NVFP4-bf16 stream vs the FL2VA-GGUF-bf16 control, byte-identical inputs via an encode-once
pe.f32) plus direct weight fingerprints, through an env-gated per-stage hook inMiniMaxH3DitForwardDevice(VT_H3_ACT_DUMP, byte-inert unset). The result refutes the "second NVFP4-render-path defect" hypothesis.No discrete materialization bug. Every weight, bias, fp32 island, output head and q/k-norm loads quant-noise-close to the coherent GGUF (qkv/out/fc/adaln 2-6% rms = Q3K-vs-NVFP4 quantizer variance, sign-correct; all biases <0.1% apart incl
adaln_b), and the RoPE cos/sin cache is byte-identical. No scramble / transpose / mis-stride / wrong-dtype / wrong-shape. Both arms run the identical forward, so the grid is 100% the per-weight NVFP4-vs-Q3K quant delta on the SAME weights (#94: sign-agreement 1.000, dequant byte-verified).Divergence profile. First appears at the token refiner + the block-0 attention INPUT (not RoPE, not a GEMM/norm weight) and amplifies chaotically through the 50-block stack, driven by the Qwen massive text activation (
condition_projabsmax ~7.4e4); final latents are decorrelated (sample-rel-L2 >1, not scale). Render A/B re-confirmed in the same byte-inert build: NVFP4 t2va = pale patch grid, FL2VA-GGUF t2va = coherent orange cat.Conclusion. Not a loader fix. The residual is the community NVFP4 checkpoint's quantization fidelity (
Star Ultimate Model Converter Prolineage; corr 0.85-0.94 to the coherent Q3_K) times the DiT's massive-activation sensitivity. Clean-reference disambiguation is blocked (bf16 132GiB = OOM on one GB10; a same-finetune REF2VA-GGUF control = disk-blocked); the path forward is an official modelopt-NVFP4 checkpoint. The #94 nibble fix stands. The fp4-resident Marlin arm's separate grid is untouched (wiring-gated-only residual).What lands here
VT_H3_ACT_DUMP): a reusable per-stage/weight/RoPE/intra-attention fingerprint in the DiT device forward.Draft: this is a diagnostic + refuting finding, not a code fix (there is no loader bug to fix).
🤖 Generated with Claude Code
https://claude.ai/code/session_014fZAcg1WcU8V629k6HWKys