LTX-2.5: 21B joint video+audio DiT and the generalized video seam - #437
Draft
localai-bot wants to merge 104 commits into
Draft
LTX-2.5: 21B joint video+audio DiT and the generalized video seam#437localai-bot wants to merge 104 commits into
localai-bot wants to merge 104 commits into
Conversation
…d video seam LTX-2.5 released with weights that fit one GB10 comfortably and a text encoder this project has already ported, so the two largest costs of a new video model are already paid. This commits the spec BEFORE any implementation, per the spec-before-code rule, and opens the row. Geometry is MEASURED, not inferred: the FP8 checkpoint's own safetensors header was read by HTTP range request (6124 tensors, 881,048-byte header, no payload downloaded, the same technique used for H3's manifests). 21.00B parameters, 48 blocks at 386.7M, video stream 4096 (32 heads x 128), audio stream 2048 (32 heads x 64). The filename says 22b and the Diffusers card says ~19B; the measured count is what the records use. The structural break from MiniMax-H3 is that H3 packs every modality into ONE sequence with per-row token tags, while LTX runs TWO streams coupled by explicit audio<->video cross-attention. It also adds per-head gated attention on every attention (to_gate_logits = Linear(query_dim, heads, bias=True), attention.py:513-514, applied after the attention output at :577), which H3 has no analogue for. The checkpoint independently confirms the source: `ff` carries no bias while `audio_ff` does, which is exactly ff_bias=false / audio_ff_bias=true from model_configurator.py:78-80. That agreement, not either side alone, is what the verify-against-both rule asks for. The free win: get_ada_values modulates from scale_shift_table USING the timestep (transformer.py:192-197), but the prompt path does not, and at transformer.py:441 kv_modulation reads prompt_scale_shift_table with no timestep term at all. 2.5 ships only that static [2, dim] table and no prompt-side timestep MLP, so the cross-attention K/V for all 48 blocks is computed once per request and reused across every denoise step. L2 gates cached-vs-recomputed bit-identical. Three things are recorded as OWED up front rather than left to be discovered: the speed axis lands PENDING because vLLM-Omni has no native 2.5 path (vllm-omni#6066 open) and its diffusers adapter is a black box (supports_step_execution=False), so it is not vLLM's production denominator; DiffVAE is refused BY NAME rather than silently downgraded to the Conv VAE; and no render-quality claim follows from structural e2e, which H3 taught directly. The MODEL row ratchet moves 362 -> 363 because a row EXISTS, never to make a transition pass. Its semantics are unchanged and test_model_row_ratchet_is_load_bearing still proves it catches drift in both directions; the docstring gains the new transition so the history stays legible. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…Engine (#435) LTX-2.5 phase L1 (.agents/specs/ltx-2-5.md sections 5 and 6). The video capability was concrete-typed on MiniMax-H3: `MiniMaxH3VideoEngine` was the only thing the C ABI, /v1/videos and the example could reach. A second video family (LTX-2.5, a 21B joint video+audio flow-matching DiT) would otherwise have to be a second parallel path, which AGENTS.md "Shared seams" forbids. Introduces `vllm::multimodal::VideoEngine` -- an abstract engine plus a checkpoint-DETECTED family registry -- and moves H3 behind it without moving a single output byte. * `include/vllm/multimodal/video_engine.h`: VideoModelParams / VideoGenParams / VideoResult (the generic superset), the abstract VideoEngine, and the registry (RegisterVideoFamily / RegisteredVideoFamilies / DetectVideoFamilies / LoadVideoEngine) with the self-registration idiom the model registry already uses (REGISTER_VLLM_VIDEO_FAMILY, one line per family, zero shared-array edits). * Family-specific fields ride in a `std::map<std::string,std::string> extras` -- H3's `partition`, LTX's future `pipeline_kind` / `model_version` -- so a new family adds no member to a struct every other family must ignore. * NEVER GUESS A FAMILY. Detection asks every family what the checkpoint HOLDS (tensor names, read header-only through the shared ReadVideoCheckpointTensorNames), never the file extension or path spelling. Zero claimants or several is a refusal naming what was seen and what is registered. "Only one family is registered, so it must be that one" is the silent mis-load this seam exists to prevent: an H3 DiT handed to an LTX loader does not fail, it renders noise. * H3 is now `MiniMaxH3VideoEngine final : public VideoEngine`, registered as family "minimax-h3", detected by its DUAL patch projection (video_patch_proj.weight + audio_patch_proj.weight -- the names all four loader arms bind by, minimax_h3_gguf.cpp:100-103). The H3-typed structs, `Load` and the H3-typed `Generate` are untouched; the generic overrides are ADAPTERS onto them, so there is one code path, not two. * The /v1/videos request mapping moved to the family-agnostic seam (VideoGenParamsFromRequest); MiniMaxH3VideoGenParamsFromRequest delegates to it, so HTTP, FFI and the registry cannot drift. ABI 17 -> 18, purely ADDITIVE: * vllm_video_model_params.family -- NULL/empty (the zero value) means detect, which is exactly what a v12 caller already gets. An unregistered name is refused naming what is registered; it is never a hint. * extra_keys / extra_values / n_extras on vllm_video_model_params and vllm_video_params. `partition` becomes the documented ALIAS for the "partition" extra; supplying both with DIFFERENT values is VLLM_ERR_INVALID_ARGUMENT rather than a silent winner. * vllm_video_engine_family() so detection is visible to a C caller. A v12 caller zero-fills the struct growth and is byte-identical; the existing v12 test_capi section is unchanged and still green at 102 assertions, including the text-checkpoint refusal that names vllm_engine_load (that hint moved into the generic refusal, where it is equally true). NOTE for the operator: the spec says "ABI goes to v13". v13 shipped long ago (vllm_complete_tokens) and VLLM_ABI_VERSION was already 17 -- the spec was reading the VIDEO SLICE's v12 label as the ABI counter. 18 is the only self-consistent next number; the intent (additive, v12 video callers byte-identical) is met exactly. Gates: * RED first: tests/vllm/multimodal/test_video_engine.cpp fails to compile against the missing seam, then 10 cases / 227 assertions SUCCESS. * BYTE-IDENTITY: the abstract path (LoadVideoEngine -> VideoEngine::Generate) reproduces the committed pre-fold goldens frame-for-frame and WAV-for-WAV, and tests/vllm/models/test_minimax_h3_video_fold.cpp is unchanged and green at 6 cases / 137 assertions -- the same count as before the change. * test_capi 55 cases / 502 assertions SUCCESS (v12 section 102, new v18 section 6 cases / 35 assertions). * Clean Release/-Werror rebuild from an empty tree: zero warnings. * Full ctest: 391/391 passed. * Mutation: forcing the H3 detector to claim every checkpoint ("we are the only family") turns test_video_engine RED (2 cases / 3 assertions) and the capi v18 section RED -- the never-guess gate bites. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…duced dims Phase L2 of the LTX-2.5 campaign (.agents/specs/ltx-2-5.md, issue #435): the 21.00B joint video+audio DiT's parameter contract and its full dual-stream forward, ported from Lightricks LTX-2 (packages/ltx-core/src/ltx_core/model/transformer/) and gated against those same modules IMPORTED BY PATH and EXECUTED at reduced dimensions on CPU. The method is MiniMax-H3's, which is what made its bricks trustworthy: both sides rebuild every weight and every input from one deterministic FNV-1a + splitmix64 stream keyed by the parameter's own NAME, so no weight byte is checked in and the weight CONTRACT is itself part of the gate — a name either side invents that the other lacks changes the numbers. scripts/gen-ltx2-goldens.py needs only a sys.path entry pointing at an LTX-2 checkout: no venv, no checkpoint, no gated download. It asserts the resolved ltx_core is the checkout's, not an installed one. Measured max|diff| against the oracle, over the whole forward: split RoPE video 1.19e-07 audio 4.84e-08 interleaved RoPE video 8.94e-08 audio 4.47e-08 float64 frequency ladder video 1.04e-07 audio 7.45e-08 prompt + self masks video 8.94e-08 audio 7.45e-08 audio stream disabled video 8.94e-08 audio 4.47e-08 Each trap the spec names is gated on its own so a failure localizes: the frequency ladder (BIT-equal, see below), the split/interleaved/float64 cos-sin tables, AdaLayerNormSingle, the per-head gated attention, the asymmetric audio->video projection pair, and the ff / audio_ff bias asymmetry. The enumeration is compared name-for-name and shape-for-shape against upstream's own named_parameters(), and ParseLtx2DitParamsFromManifest recovers the geometry from the SHAPES alone. The frequency ladder is asserted BIT-equal, not close, because that is where this port's one real defect lived. Computing rope.py's `linspace(0, 1, n)` in double instead of reproducing ATen's float32 half-forward/half-backward walk moves two of the eight AUDIO samples by a single f32 ulp; the ladder multiplies that up to a 1.5e-4 frequency error and RoPE turns it into 1.2e-4 in cos/sin. The video ladder (n=5, exact step) hides it completely. PROMPT-K/V CACHE. LTX-2.5 sets use_prompt_adaln_single=false and ships only the static prompt_scale_shift_table, so transformer.py:441 builds the K/V modulation with no timestep term and the text cross-attention K/V for every block is computable ONCE PER REQUEST. That ships as the default path, and the gate asserts cached and recomputed are BIT-IDENTICAL — then POISONS the cache and requires the output to move, so the identity cannot pass vacuously by never reading it. NEW SHARED SEAM. vt::Attention refuses Tq != S ("query/key/value token count must match"), which no cross-attention can satisfy, so this adds vt::AttentionCross: dense non-causal attention over differing query and key extents with an optional additive score bias. Purely additive to vt; every existing self-attention call is untouched and LTX's own self-attention still routes through vt::Attention. It ships with a CPU kernel only — unified memory gets it through the reference tier, a discrete CUDA device refuses by name, and the native CUDA kernel is recorded as owed alongside the device-resident forward. Refused rather than silently served, each naming the missing piece: any stream dtype other than f32 (bf16/FP8/NVFP4 are L6); the prompt-K/V cache when the prompt AdaLN MLP is on; the 19B caption-projection form (L3); keyframe absolute-position embeddings; LTXModelType.VideoOnly / AudioOnly, whose weight contract differs. Guidance perturbations run in their no-op configuration, which is upstream's own perturbations=None path, and that is recorded in the header. DOCS, and why this commit touches them. The dispatching operator scoped docs/* out of this task and owns those records. check-doc-checkpoint --staged is RED without them: `src/vllm/model_executor/models/` classifies as feature_surface and `include/vllm/` + CMakeLists.txt as user_usage, so the gate demands docs/FEATURES.md and docs/USAGE.md. Independently of the gate, FEATURES.md's LTX-2.5 row read "SPIKE: spec committed, no forward yet", which this commit makes false, and a public projection that has drifted is worse than a scope deviation. The edits are therefore surgical and LTX-2.5-only: the one existing keyed row in FEATURES.md, and one new USAGE.md section that leads with "there is no LTX-2.5 render path yet" and documents how to reproduce the gate. Flagged to the operator for reconciliation rather than done silently. STILL OWED, so it is not discovered later: no porting-inventory entry was added for vt::AttentionCross. Its CPU kernel mirrors the existing AttentionKernel and torch SDPA's is_causal=False contract, which section 9.1's recorded vt:: deviation already covers in general, but a reviewer may want it named there explicitly. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…pstream Phase L4 of .agents/specs/ltx-2-5.md (issue #435). Ports the DECODE half of both LTX-2.5 VAEs and gates each brick against the upstream `ltx_core` modules EXECUTED at reduced dimensions on CPU — the method that made MiniMax-H3's VAE bricks trustworthy. Both sides rebuild every weight and input from one deterministic stream, so no weight byte is checked in. scripts/gen-ltx2-vae-goldens.py runs Lightricks/LTX-2 as the oracle ltx2_audio_vae.{h,cpp} AudioDecoder + Vocoder (both resblock arms) + VocoderWithBWE ltx2_video_vae.{h,cpp} ConvVideoDecoder, and the DiffVAE refusal test_ltx2_vae.cpp 10 cases / 1120 assertions Measured max|diff| vs upstream, f32 round-off throughout: audio decoder 1.78814e-06 audio decoder (freq pad) 1.78814e-06 BigVGAN v2 vocoder 2.23517e-08 legacy resblock-1 vocoder 2.23517e-08 BWE vocoder chain 1.49012e-07 Conv video decoder 1.40071e-06 kaiser-sinc filter 2.98023e-08 hann-sinc resample filter 1.16415e-08 REFUSED BY NAME, never downgraded (spec section 0 item 2): the diffusion video decoder. `Ltx2VideoDecode(kDiffusion, ...)` throws naming NADiffusionDecoder and its missing neighborhood-attention kernel, and the test asserts the message carries that name and that nothing decoded on the way to the refusal. A silent fall-back to the Conv decoder would return a worse render as if it were the requested one, and no gate this project owns could detect that. `attn_res_x` is refused too: at this upstream revision the block cannot be CONSTRUCTED, because _make_decoder_block passes `attention_head_dim` to UNetMidBlock3D, which does not accept it. TWO FINDINGS, measured rather than assumed, both now gated against upstream: * The audio decoder is NOT end-to-end causal. `causality_axis` governs the CONVOLUTIONS; its AttnBlocks attend over the whole (time, mel) map, so a last-frame perturbation reaches EVERY output frame. Upstream agrees. The convolution-only reach is gated separately with attention off, where the oracle says frames [5, 8] move and the port must agree. * The Conv video decoder is NOT end-to-end causal either, for a different reason: `res_x_y`'s shortcut norm is a one-group GroupNorm over (C,T,H,W) whose statistics span TIME. Its convolution-only reach is gated on a stripped block list, where the oracle says frames [3, 4] move. HONEST GATE LIMIT, found by mutation and recorded rather than hidden: flipping the video decoder's PixelNorm eps from 1e-8 to 1e-6 leaves every golden GREEN, because the normalized activations are O(1) and the two epsilons differ by ~1e-7 relative. The distinction is real upstream (bare `PixelNorm()` vs build_normalization_layer), so it is pinned by a source-anchored constant assertion instead, which does catch that mutation. Mutation-verified RED: zero temporal padding instead of a replicated first frame -> 2.34534; unpatchify q/r swapped -> 1.96412. Tree restored byte-for-byte. OWED, so it is not discovered later: the ENCODER halves (AudioEncoder + its mel front-end, and the video encoder), the tiled decode path, and the DiffVAE row. EXCEPTION ARGUED HERE, per AGENTS.md "Changing the rules or a checker": this commit does not update docs/FEATURES.md or docs/USAGE.md, which check-doc-checkpoint requires for src/vllm/model_executor/models/, include/vllm/ and CMakeLists.txt. The LTX-2.5 campaign runs several implementers in parallel on one branch and the operator holds every record surface, so writing those two files from a helper would collide with the DiT and pipeline phases in exactly the shared-file way AGENTS.md's "no surface that every PR must write" warns about. The obligation is REPORTED to the operator, not dropped; it is owed before the campaign's PR lands, and this commit is visible debt until then. FOLLOWING_AGENTS_PROTOCOL Refs #435 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
L1 found this while implementing. The spec said the seam would land at "ABI v13", which was wrong twice over: VLLM_ABI_VERSION was already 17, and v13 shipped long ago as vllm_complete_tokens. The mistake was reading the VIDEO SLICE's own v12 label as if it were the ABI counter. The requirement never depended on the number. What the spec actually asks for is that the bump be purely ADDITIVE so v12 video callers keep working byte-identically, guarded by the existing test_capi v12 section. That is unchanged and met. Only the number moves. Recorded as a correction in place rather than a silent edit, because the wrong number was committed and a reader of the history should see why it moved. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Both came out of the campaign rather than the reading, and both are the kind of thing that produces a finite, correctly-shaped, WRONG result. Recording them in the spec so they cannot be rediscovered the expensive way. 1.4 — text conditioning is a MULTI-LAYER aggregate, not the last hidden state. feature_extractor.py normalizes hidden states shaped [B, T, D, L] and concatenates ACROSS THE LAYER dimension. The real TE checkpoint confirms it: the two caption projections take 94080 input features, and 94080 = 1920 x 49, the model's per-layer width across its 48 layers plus one. There are two normalization variants (per-batch masked mean/range with an 8x scale, and per-token RMS "for V2 models") and the right one is selected from config, never guessed. Get the variant, the mask handling, the reduction axes or the layer order wrong and the model renders a plausible video for the WRONG PROMPT, which no shape or finiteness check catches. Two loader facts measured alongside it: the tokenizer ships EMBEDDED AS A TENSOR (tokenizer_json, ~32 MB) rather than as a sibling file, so a loader that looks for tokenizer.json fails on this checkpoint; and the TE is quantized with torchao NVFP4, not compressed-tensors as H3's arm is, so L6 must verify the layout rather than assume the two are the same. 1.5 — the audio VAE is NOT end-to-end causal. L4 measured this instead of assuming it: its first causality probes failed and upstream agreed with the failure. causality_axis governs the convolutions, but the AttnBlocks attend over the whole (time, mel) map, so a last-frame perturbation reaches every output frame. The Conv video decoder is not causal either, for an unrelated reason: a one-group GroupNorm whose statistics span time. "Causal" is precisely the property a port assumes and never checks, so both are now gated in two parts, with upstream supplying the expected windows. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…d prompt L2's fresh review disproved a claim this spec made before it was true. 1.2 said the bit-identity gate meant the cache "cannot silently diverge". The distinction it missed is the whole point of the feature: Against a changed TIMESTEP the cache cannot diverge. That is the property the checkpoint gives us, it is why use_prompt_adaln_single=false is a free win, and it is real. Against a changed PROMPT it silently could. The gate ran the forward twice with IDENTICAL inputs, so it could only ever prove "same in, same out". The cache carried no prompt identity and its only validity check was on SIZE. A probe that swapped in a different prompt of equal token count found the cache did not notice, while the ground truth moved. What that implies is not academic. A pipeline or server reusing one cache across two requests whose prompts differ but tokenize to the same length renders the SECOND REQUEST WITH THE FIRST REQUEST'S PROMPT. No error, no shape mismatch, no finiteness failure, and no gate. That is precisely the class of defect this campaign keeps naming: finite, correctly shaped, and wrong. The repair carries a content fingerprint and refuses by name; it is with a fresh implementer. Also recorded, from the same review: binding-oracle parity is PENDING for every brick landed so far. L1-L5 gate against the ltx-core CROSS-CHECK, not against vLLM-Omni, because -Diffusers access is still awaiting manual approval. That is legitimate under 3 and 6, and it is what a cross-check is for, but it was implicit and is now stated. BENCHMARKS.md recorded only the SPEED axis as pending, which understated it; both axes are now named there. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ated (#435) Repairs the adversarial review findings on LTX-2.5 phase L1 (3db9233). All four are the same defect class: a hole in the never-guess guarantee is not an error-handling nicety here, it is a silent-wrong-render risk, because an H3 GGUF handed to an LTX loader does not fail, it renders noise. F1 (MEDIUM) -- RegisterVideoFamily refused an empty name and a missing detector or loader, but NOT a name already registered. The reviewer registered a second "minimax-h3" whose detector claimed every checkpoint and whose loader was an impostor, and got `registered listing size=1, claimants=1`: two families under one name, the SEVERAL-claimants refusal unreachable because both claimants carry the SAME name, and which loader runs decided by static-init and link order. The real family won only because stable_sort preserved registration order and its TU linked first. RegisterVideoFamily now refuses a collision, naming it. Registrars run at static init, so that throw ends the process -- the intended outcome, and the same one the empty-name refusal beside it already produces: a name collision is a BUILD defect, and dying while naming it is strictly better than rendering noise. Once the registration refuses, the two std::unique calls (RegisteredVideoFamilies, DetectVideoFamilies) were masking nothing, so they are gone rather than left to look like protection. They never were: adjacent-only, they collapsed the printed listing while leaving two entries in the registry for resolution to choose between, and in DetectVideoFamilies a std::unique is actively harmful, since merging a duplicate-name pair back into one claimant is exactly what lets the SEVERAL refusal fall through into a load. Assertions state the invariants in their place. F3 (LOW) -- the canonical sort ran inside a function-local `static const bool`, so it happened once, on first query. Anything registered afterwards was appended unsorted, and std::unique then also stopped de-duplicating. No REGISTER_VLLM_VIDEO_FAMILY registrar can hit that (registrars all run before main), but RegisterVideoFamily is a public header function and a caller may register at any time. Sortedness and uniqueness are now invariants of the TABLE, established by sorted insertion at every registration, so they hold whenever anyone registers rather than whenever someone first asks. F2 (LOW) -- the SEVERAL-claimants branch is unreachable with one family registered, so nothing gated it and it could rot before LTX-2.5 arrives. A registry test does not have to wait for a real second family: the new test case registers a throwaway one, whose detector claims ONLY when the caller sets its extra (so its presence cannot change what any other test detects) and whose loader is an impostor that counts its own calls. That single case gates F1, F2 and F3 together, and asserts the refusal happened INSTEAD of a load rather than after one. F4 (LOW) -- test_video_engine never asserted VideoResult::audio_path. CheckAgainstGoldens read `out_dir + "/audio.wav"` off disk, so the WAV half of the byte-identity claim rested on a filename convention: mutating `out.audio_path = "/dev/null"` in MiniMaxH3VideoResultToGeneric left the seam test GREEN at 10 cases / 227 assertions. It now reads the path the RESULT reports, and pins that path, the way the frames were already pinned through result.frame_dir. F7 (nit) -- the empty-dit_path refusal advised "Declare the family explicitly rather than letting the loader guess", which is a dead end for a caller who supplied no checkpoint: the declared loader would still have nothing to open, so following the advice produces a second, more confusing refusal. That case now names the missing input and what would satisfy it. Not changed: F5. The struct growth is source-compatible but not binary-compatible, which is a property of the project's established ABI-growth model (v2/v4/v7/v8/v9/v10/v14 all use the same zero-filling language) and not a regression this work introduced. No ABI or behaviour change was needed for any of the above; nothing under ltx2*, no H3 numerics, and no spec or record surface was touched. docs/USAGE.md: `check-doc-checkpoint --staged` classifies `include/vllm/multimodal/video_engine.h` as `user_usage`, so the C++ consumption section gains the video seam's resolution contract -- the refusals a caller now meets, including the two this commit changes. That is the minimal keyed edit that discharges the obligation; the ABI table is untouched because the ABI is. Gates (CPU-only Release, -Werror, GCC 13.3.0): * RED first, verbatim, before the fix: test_video_engine 11 cases / 247 assertions, Status: FAILURE -- `std::is_sorted(all)` false (F3), `a family name that is already registered must be refused` (F1), and `msg.find("Declare the family explicitly") == npos` reading `128 == 18446744073709551615` against the live message (F7). * GREEN after: test_video_engine 11 cases / 254 assertions, Status: SUCCESS (was 10 / 227). * MUTATION for F2 (no product defect to go red on): forcing LoadVideoEngine to take the first claimant (`claimants.size() == 1` -> `!empty()`) turns the new case RED at 5 failed assertions, and the impostor loader RUNS (g_impostor_loads 1, expected 0). Restored byte-for-byte. * MUTATION for F4: `out.audio_path = "/dev/null"` turns 3 cases RED. Note the doctest trap in the raw output: the total assertion COUNT DROPS 254 -> 237 as cases throw, so the count alone reads like a smaller run, not a failure. Status is the signal. Restored byte-for-byte; minimax_h3_video.cpp is identical to 3db9233. * Baselines held: test_minimax_h3_video_fold 6 cases / 137 assertions, test_capi -tc='capi v12*' 4 cases / 102 assertions (the byte-identity evidence, unmoved), test_capi 55 cases / 502 assertions. * Clean Release rebuild from an empty tree: 1185/1185, exit 0, zero warnings. * Full ctest: 391/391 passed, 0 failed. * Debug (assert-enabled, NDEBUG off) build of the three gate binaries: test_video_engine 11 / 254, fold 6 / 137, capi v12 4 / 102, all SUCCESS -- the new assertions hold. SCOPED, not full: a full Debug tree of all 1185 targets hit ENOSPC on this box (447G volume, 622M left at the failure), and filling the disk breaks every other agent's gate here, so the Debug lane covers exactly the binaries that exercise the added assertions. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… upstream Phase L3 of .agents/specs/ltx-2-5.md (issue #435). LTX-2.5 does NOT condition on a text encoder's last hidden state: it takes EVERY Gemma-4 hidden state, stacks them on a new last axis, normalizes, concatenates ACROSS THE LAYER AXIS and projects the flattened result twice. This ports that path and gates it against the upstream `ltx_core` modules IMPORTED BY PATH and EXECUTED at reduced dimensions on CPU, the method L2 and L4 used on this campaign and MiniMax-H3 before them. Both sides rebuild every weight and input from one deterministic FNV-1a + splitmix64 stream keyed by the parameter's own NAME, so no weight byte is checked in and the weight CONTRACT is part of the gate. scripts/gen-ltx2-text-goldens.py needs only a sys.path entry pointing at an LTX-2 checkout, and asserts the resolved ltx_core is the checkout's rather than an installed one. scripts/gen-ltx2-text-goldens.py runs Lightricks/LTX-2 as the oracle ltx2_text_encoder.{h,cpp} both norm variants, both caption projections, the conditioning hand-off, the asset pack gemma4.{h,cpp} ForwardHiddenStates: the per-layer capture test_ltx2_text_encoder.cpp 15 cases / 2473 assertions Measured max|diff| vs upstream, f32 round-off throughout: hidden-state stack (a permutation) 0 BIT-equal _norm_and_concat_padded_batch left / right 4.77e-07 / 4.77e-07 norm_and_concat_per_token_rms left / right 2.38e-07 / 2.38e-07 FeatureExtractorV1 (is_av) left / right 6.52e-08 / 5.96e-08 FeatureExtractorV2 video left / right 4.47e-08 / 5.96e-08 FeatureExtractorV2 audio left / right 2.98e-08 / 2.98e-08 conditioning video / audio left / right 4.47e-08, 5.96e-08 / 2.98e-08 additive mask, sort index, reordered mask EXACT A CORRECTION TO THE DISPATCH BRIEF, measured from the checkpoint header rather than assumed. The brief read the projections' 94080 as the input feature count and concluded hidden = 1920. NVFP4 packs TWO values per byte, so the U8 widths are half the real ones: `q_proj.weight U8 [4096, 1920]` is [4096, 3840], `o_proj U8 [3840, 2048]` is [3840, 4096], and `model.norm.weight BF16 [3840]` -- stored BF16, therefore unpacked -- confirms it independently. The Gemma hidden size is 3840 and the caption projections take 188160 = 3840 x 49 inputs. The layer count (48 + 1) was right; only the width moves. THE CONCATENATION IS HIDDEN-MAJOR, LAYER-MINOR. `stack(..., dim=-1)` then `.reshape(B, T, D*L)` places layer `l` of channel `d` at flat index `d * L + l`. Layer-major is a correctly shaped, finite, PERMUTED conditioning vector, so the stack is emitted as a golden of its own and the test also proves the layer-major alternative is a different tensor. VARIANT SELECTION IS GATED, NOT GUESSED. `Ltx2SelectTextFeatureVariant` mirrors encoder_configurator.py:163-209: none of the four V2 marker keys means V1, all four with their exact values means V2, and a partial set or a drifted value is refused by name. Shapes cannot distinguish the two (3840 x 49 is the flat width under both), which is exactly why the selection has to come from config. The test asserts against upstream's OWN resolved classes and shapes, plus both refusals. PADDING IS GATED ON BOTH SIDES. The tokenizer LEFT-pads (base_encoder.py:235), the connector wants RIGHT-padding (embeddings_processor.py:82-84) and the extractor sits between them, so every brick runs on a left-padded and a right-padded mask. Both norms are asserted to zero the padded positions EXACTLY. The projected pads are asserted NOT to be zero under V2: the norm zeroes them, so their projected value is exactly the Linear's bias, and a port that force-zeroes projected pads diverges from upstream on every padded row while still looking masked. V1 has no bias, so its padded rows ARE zero, and that asymmetry is gated too. TWO MEASURED FINDINGS, gated as measured rather than repaired: * `_to_binary_mask`'s predicate is `< 1e-6`, and BOTH masks upstream can hand it satisfy it everywhere -- `zeros_like(additive)` with learnable registers on (which 2.5 has, embeddings_connector.py:152) and the additive mask itself with them off, since -FLT_MAX < 1e-6 too. The mask EmbeddingsProcessor hands the DiT is therefore ALL ONES. Gated on both inputs so a port cannot "fix" it. * The shipped vonkaiser/LTX-2.5-FP8-NVFP4 text encoder has 1688 tensors and NO safetensors `__metadata__` block at all, so upstream's GemmaAssets.from_single_file raises on it before reading a tensor (gemma_assets.py:110-114). `Ltx2LoadGemmaAssets` refuses identically by default; a caller holding the Gemma config out of band passes require_config=false. THE GEMMA-4 EXTENSION, kept as small as the seam allows. Our Gemma-4 returned logits and nothing else, so `Gemma4Model::ForwardHiddenStates` was added: one optional out-parameter on the existing ForwardBody, nullptr on every other call site, so no shipped path changes shape or cost (the test asserts the plain forward's logits are bit-identical with the capture on). It mirrors transformers' `output_hidden_states=True` ORDER exactly, which is the part that fails silently: [0] is the sqrt(hidden)-scaled embeddings, [i] is the output of decoder layer i-1, and [L] is model.norm(output of the LAST decoder layer) -- the RAW last layer output never appears. That last entry is proven by the invariant only the final-normed state satisfies: the logits are exactly that state through the tied lm_head (max|diff| 2.05e-08 over max|value| 0.167). MUTATION-VERIFIED RED, tree restored byte-for-byte (md5 checked): layer order reversed in the stack -> 6 cases / 17 assertions fail; stack 0.899, norm V1 7.96, norm V2 2.87, V2 extractor 0.543/0.463 variant selection ignored (always per-token RMS) -> V1 extractor 0.431 / 0.508 gemma4 capture stores the RAW last layer -> the lm_head invariant moves from 2.05e-08 to 0.172 over max|value| 0.0167 DTYPE. Everything is f32, exactly as L2 records for the DiT: the PARITY dtype of this gate, not a widening of a bf16 path. Upstream resolves ONE model dtype (base_encoder.py:41) and FeatureExtractorV2 casts the normalized tensor straight back to `encoded.dtype`, so the bf16/FP8/NVFP4 arms are a single stream-dtype choice, phase L6, and are OWED. Every entry point REFUSES a non-f32 compute_dtype with a message naming the missing phase, and that refusal is gated. Reduction accumulators are double; that is an accumulator width, not a memory format, and no buffer this file produces is wider than f32. OWED, so it is not discovered later: * `Embeddings1DConnector` (embeddings_connector.py:74-191) is NOT ported. It is built out of the DiT's own Attention, FeedForward and RoPE, which phase L2 owns, so it belongs to the change that can link against them. This stops at the connector's INPUT contract, which is what it gates. * The Gemma-4 TOWER itself is not gated against LTX's oracle here. transformers 5.3.0 on this box carries no `gemma4_unified` in CONFIG_MAPPING, so `AutoModelForImageTextToText.from_config` cannot build the tower at reduced dimensions and there is no oracle to run. The LTX-specific delta -- which states, in which order, and everything downstream -- IS gated; the tower is gated against vLLM by its own row. * The quantized arm. The checkpoint is torchao NVFP4 (`weight` U8 packed, `weight_scale` F8_E4M3 grouped, `weight_scale_2` F32 scalar, plus a `torchao_nvfp4` U8 [240] marker per module), which is NOT the compressed-tensors layout H3's NVFP4 arm uses. Phase L6. * No porting-inventory entry was added for the ltx2_text_encoder anchors. DOCS, and why this commit touches them. The dispatching operator scoped docs/* out of this task and owns those records, but check-doc-checkpoint --staged is RED without them: src/vllm/model_executor/models/ classifies as feature_surface and include/vllm/ + CMakeLists.txt as user_usage. The edits are minimal and LTX-2.5 only: ONE NEW keyed row in FEATURES.md (additive, so it does not collide with L2's edit of the existing DiT row) and one USAGE.md section that leads with "there is no LTX-2.5 render path" and documents how to reproduce the gate. Flagged to the operator for reconciliation rather than done silently. Evidence. RED before: 13 cases, 3 passed / 10 failed, "assertions: 25 | 25 passed | 0 failed", Status: FAILURE -- the documented trap where a failing gate prints zero failed assertions. GREEN after: 15 cases / 2473 assertions, Status: SUCCESS. Clean Release -Werror CPU-only rebuild, 0 warnings; full ctest 391/391. FOLLOWING_AGENTS_PROTOCOL Refs #435 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…is 3840 wide L3 caught an error I put in this spec. 1.4 recorded the caption projections as taking 94080 = 1920 x 49 inputs, reading the checkpoint's U8 shapes as if they were logical. NVFP4 packs TWO VALUES PER BYTE along the last dimension, so every U8 width in that file is half the real one. The projections take 188160, and 188160 = 3840 x 49. The tell was in the same header the whole time, and I printed it without reconciling it: model.norm.weight is BF16 [3840], and BF16 is unpacked, so 3840 is authoritative. Three further confirmations agree. embed_tokens U8 [262144, 1920] -> [262144, 3840]. o_proj U8 [3840, 2048] -> [3840, 4096], which is 16 heads x 256. And the projections' own biases are BF16 [4096] and [2048], unpacked, matching the DiT's two stream widths exactly. The layer count (48 + 1) was right; only the width moved, and nothing downstream of the width is affected. The general lesson, which applies again at L6: read a quantized checkpoint's UNPACKED tensors to establish a width, never its packed ones. A packed width is plausible, self-consistent, and wrong by exactly a factor of the packing. Also settled by execution rather than inference: 2.5 uses FeatureExtractorV2 (per-token RMS), selected by four marker keys, with _rescale_norm applied separately per projection over that projection's own out_features. create_caption_projection is not on this path. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Repairs the adversarial review of phase L2 (issue #435, PR #437, spec .agents/specs/ltx-2-5.md §1.2/§7). The math the review reproduced is unchanged; what was missing is that two shipped paths had nothing holding them. F2 (MEDIUM) — the prompt-K/V cache served a STALE PROMPT. Its only validity check was a SIZE check, so two requests whose prompts differ but whose token counts agree — the ordinary case for anything that reuses one cache across requests — rendered request 1's prompt for request 2 with no shape mismatch, no non-finite value and no error. The cache now carries an FNV-1a fingerprint over both streams' context tensors, their geometry and their prompt masks (Ltx2PromptIdentityOf), and a filled cache that meets a different prompt refuses BY NAME, saying which part moved and that Ltx2PromptKvCache::Reset() rebinds the allocation to a new request. The digest hashes bytes, so it is an identity test and never a similarity one: a false refusal costs one recompute, a false accept renders the wrong prompt. Against a changed TIMESTEP the cache still cannot diverge — that property is real and untouched. F1 (MEDIUM) — the dense [Tq, S] attention bias in vt::AttentionCross had ZERO coverage. Upstream documents (B, T, T) as the dense form (transformer_args.py:212-215 @ fd4ded7f), the port accepts it and vt::AttentionCross validates it, but every existing test used the key-only broadcast, where there is exactly one bias row and a kernel reading row 0 for every query is indistinguishable. scripts/gen-ltx2-goldens.py now emits a DenseMask forward case with (BATCH, T, T) masks on both streams, and a focused vt case pins two identical queries to different bias rows. The reviewer's mutation (every query reads bias row 0) now fails both. F5 (LOW) — the GQA broadcast (g = h / (Hq/Hkv)) was unreachable: every LTX attention has Hq == Hkv. Gated with an Hq=4/Hkv=2 case plus the non-multiple refusal. F3 (LOW) — "refuses by name" refused by NUMBER. vt::OpName gives every OpId its canonical spelling; GetOp's refusal and the reference-tier warning now name the op and the device (the integers are kept for grep). The defining switch is exhaustive and default-free, so appending an OpId without naming it fails the -Werror build — proved by deleting one case (error: enumeration value 'kAttentionCross' not handled in switch). The refusal is gated in tests/vt/test_op_provider.cpp (message + OpName totality and uniqueness) and in test_ltx2.cpp for kAttentionCross specifically. The same finding's second half: Ltx2Attention routed to vt::Attention on `s == tq && bias == nullptr`, so the DISPATCHED OP — and on a device carrying kAttention but not kAttentionCross, the success or failure of the call — depended on the prompt length. It now routes on `context == nullptr`, upstream's own self-attention marker (attention.py:556). The two ops are asserted BIT-for-BIT equal on the square unbiased problem, so this is a dispatch decision and not an arithmetic one, and all five pre-existing max|diff| values are unmoved. F4 (LOW) — vt::AttentionCross now has a porting-inventory entry (§9.17): upstream semantics (torch SDPA as LTX's PytorchAttention calls it, attention.py:97-102), what was written from scratch and why the pinned vLLM has no equivalent, local anchor, backends, the OWED CUDA arm, tests and spec. EVIDENCE. Every finding was RED first. The reviewer's two mutations now fail: `brow = bias_data` gives 27/29 cases, 3 failed assertions, Status: FAILURE (the literal form needs [[maybe_unused]] on bias_rows to compile under -Werror); dropping the identity guard gives 27/29 cases, 7 failed assertions, Status: FAILURE. Restored byte-for-byte after each. Clean Release -Werror rebuild, zero warnings. ctest 391/391. test_ltx2 29 cases / 1615 assertions / Status: SUCCESS! (was 21 / 1350). test_op_provider 12 / 385 / SUCCESS!. kLtx2ParamCount = 214. The goldens regenerate byte-identical apart from the --out path the generator records in its own header, and the five forward max|diff| values are exactly as before: split video 1.19209e-07 audio 4.84288e-08 interleaved video 8.9407e-08 audio 4.47035e-08 float64 frequency ladder video 1.04308e-07 audio 7.45058e-08 prompt + self masks video 8.9407e-08 audio 7.45058e-08 audio stream disabled video 8.9407e-08 audio 4.47035e-08 dense self-attn mask video 5.96046e-08 audio 4.47035e-08 (new) DOCS. docs/FEATURES.md and docs/USAGE.md are touched only because check-doc-checkpoint --staged refuses a change under src/vllm/model_executor/models/ and include/vllm/ without them. Both edits are minimal and keyed: the FEATURES row gains "key-only and dense self-attention masks" and "prompt-bound" (trimmed twice to stay inside check-public-doc-tables' 220-char cell cap), and USAGE gains the paragraph a caller needs to not hit the new refusal by surprise. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Phase L1 of the LTX-2.5 campaign, reviewed twice and repaired between. vllm::multimodal::VideoEngine with a family registry that identifies a checkpoint by its TENSOR NAMES, never by path or extension, and refuses when zero or several families claim one. MiniMax-H3 moves behind it unchanged. The C ABI grows to v18 by APPENDING only. The seam exists for one reason, stated in its own header: a checkpoint handed to the wrong family does not fail, it renders noise. Both review rounds attacked exactly that. The first found that RegisterVideoFamily accepted a DUPLICATE NAME, and that the two std::unique calls then hid the collision, so the several-claimants refusal never fired and which loader ran was decided by link order. The repair makes sortedness and uniqueness invariants of the table and removes both std::unique calls rather than leave them looking like protection. Evidence carried through both rounds unmoved, which is what makes the H3 move believable: test_minimax_h3_video_fold 6 cases / 137 assertions, and the test_capi v12 section 4 cases / 102 assertions. That second number is the ABI additivity evidence. test_video_engine went 10/227 to 11/254. Reviewed by two fresh agents, neither of which wrote the code. The second confirmed the impostor loader never runs, that nothing depended on the removed dedup, and that the new assertions compile out under NDEBUG while the throw that actually enforces the invariant stays live in Release. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…pinned epsilons Repairs the adversarial review of phase L4 (.agents/specs/ltx-2-5.md, issue #435, PR #437). The review verdict was PASS conditional on three repairs: it reproduced all 8 max|diff| values, confirmed the goldens rebuild byte-identically from upstream, and survived 13 of its own mutations. Nothing was wrong with the math. The defects were on the EVIDENCE and RECORD surfaces, plus one open hole class. All 8 originally measured values are UNCHANGED by this commit: audio decoder 1.78814e-06 BWE vocoder chain 1.49012e-07 audio decoder (freq pad) 1.78814e-06 Conv video decoder 1.40071e-06 BigVGAN v2 vocoder 2.23517e-08 kaiser-sinc filter 2.98023e-08 legacy resblock-1 vocoder 2.23517e-08 hann-sinc resample 1.16415e-08 R1 THE GOLDENS HAD NO UPSTREAM REVISION ANCHOR, and the generator never proved which `ltx_core` it imported. AGENTS.md requires a ported test to preserve the upstream revision anchor; grepping for one across the generator, the goldens, both .cpp and both .h returned nothing. The generator now resolves `git -C <--ltx2> rev-parse HEAD` and emits it as `kLtx2VaeUpstreamRevision`, which the suite asserts equals the SHA it pins, so regenerating against a different checkout FAILS instead of silently replacing the oracle. The oracle-identity hole was real, not theoretical, and is demonstrated rather than asserted. With a decoy `ltx_core` (upstream plus one drifted constant) on PYTHONPATH and `sys.path.insert` refactored to `sys.path.append`, the OLD generator imported the decoy, reported success, and emitted goldens with the IDENTICAL md5 3af38d1d9f8e741ae5c2e2f6723d2779 -- the wrong oracle, undetectable. The new `load_upstream` aborts with the resolved path named. R2 THE VIDEO VAE'S f32 WAS UNANNOTATED, AND UPSTREAM DOES THE OPPOSITE. The audio tower's f32 is upstream-grounded (vocoder.py:585-595 forces float32 for the BWE chain). The video decoder has no such justification: it runs in the CHECKPOINT dtype, `sample.to(weights_dtype)` in and `sample.to(output_dtype)` out (conv_video_decoder.py:283-286, 355-356). This is AGENTS.md's "a token gate cannot catch a dtype that is too WIDE" exactly, and the golden structurally cannot catch it because `fill_from_stream` casts every weight to f32, so the ORACLE runs f32 too. Annotated, with phase L6 named as owing the checkpoint-dtype arm. Numerics unchanged. R3 THE DUPLICATED 1-D PRIMITIVES ARE NOW ONE IMPLEMENTATION. ltx2_audio_vae.cpp carried its own Conv1d, ConvTranspose1d, pad, Snake and alias-free Activation1d because MiniMax-H3's were TU-private -- a parallel path, argued only in a source comment. H3's are promoted to minimax_h3.h (generalized: the pad takes a `replicate` flag, Snake takes an optional beta) and LTX now calls them; 172 lines of duplicate deleted. BEHAVIOUR-PRESERVING, measured against the base SHA rather than assumed: test_minimax_h3 79 cases / 57395 assertions before AND after test_minimax_h3_video_fold 6 cases / 137 assertions before AND after And the seam is proven to be one seam: mutating the SHARED upsample gain (`value *= ratio` -> `* 1.01f`) with ONE edit turns BOTH suites red (ltx2 4 cases, h3 1 case). Before this commit that same edit could not have touched LTX at all. R4 THE INVISIBLE-CONSTANT CLASS IS CLOSED, NOT ONE INSTANCE. The author pinned `pixel_norm_eps`; the review found four more that survive mutation with every golden green. All are correct against upstream, and nothing held them. Each is now a named constant with a source-anchored assertion, and the header note says the limit is a CLASS. Mutations, all RED after this commit: norm_eps 1e-6 -> 1e-4 (resnet.py:31) RED mel log clamp 1e-5 -> 1e-8 (vocoder.py:515) RED, 2 cases Snake eps 1e-9 -> 0.0 (vocoder.py:198, :221) RED _RMSNorm2D floor 1e-12 -> 0 (attention.py:11-30) RED The mel clamp gets more than a constant assertion, because it is the member of the class that BINDS IN PRODUCTION: it floors the log-mel fed to the BWE generator, and real silence reaches it. The reduced-dimension stream could not, measured: the raw mel minimum is ~4.4e-3 and STAYS there even for a zero input, because the vocoder's conv biases keep the waveform off silence. A new arm attenuates mel_basis by 1e-4 so all 384 bins saturate. It catches the mutation NUMERICALLY, not just by constant: 4.07919e-07 -> 0.144965. R5 act_post WAS CONDITIONAL, UPSTREAM MAKES IT UNCONDITIONAL. `self.act_post = Activation1d(SnakeBeta(final_channels))` (vocoder.py:388) sits inside `if self.is_amp` and takes no `activation=` argument, unlike the resblocks one line earlier (vocoder.py:376). Reading `.beta` only when `snakebeta` made that one activation silently reuse ALPHA as its reciprocal scale on the `activation="snake"` arm. Fixed and gated by a new arm; upstream's own state_dict is the second half of the proof, carrying `act_post.act.beta` while every `acts1/acts2` entry has only `.alpha`. Reverting the fix goes RED. R6 the header claimed the vocoder defaults mirror `Vocoder.__init__`, whose defaults are `resblock="1"` / `activation="snake"` while ours are AMP1/snakebeta. The DEFAULTS are right and the comment was wrong: VocoderConfigurator's BWE branch REQUIRES both via check_config_value (model_configurator.py:59-64), so a default mirroring `Vocoder.__init__` would be one no shipping checkpoint can use. Comment corrected. R7 a comment said conv_in is "ALWAYS causal upstream ... independently of config.causal" directly above a line passing `config.causal`. The CODE is right: `causal=True` at conv_video_decoder.py:216 selects the CausalConv3d MODULE, while runtime one-sidedness comes from `self.conv_in(sample, causal=self.causal)` at :307. Corrected, because as written it invited a future "fix" that breaks it. R8 FOUR ARMS WERE UNGATED, INCLUDING UPSTREAM'S OWN DEFAULT. `causal=False` is the declared default (conv_video_decoder.py:184) yet every video arm ran causal=True, and the audio kNone / kWidth / kWidthCompatibility axes never ran. Added, and each independently reproduces the reviewer's measurement: non-causal Conv video decoder 1.81794e-06 audio causality_axis NONE 1.56462e-06 audio causality_axis WIDTH 9.14559e-07 audio WIDTH_COMPATIBILITY 7.00355e-07 The non-causal arm shares the causal arm's weights, input and noise stream, so the padding rule is the only difference and the test asserts the two DIVERGE. Collapsing the non-causal pad into the causal rule goes RED; so does collapsing WIDTH_COMPATIBILITY into WIDTH, which is the subtle one -- upsample.py:44-48 does NOT drop the first element for that axis. R9 the tolerances sat 6-447x above what the port produces, which makes them decoration. Replaced with two derived constants: kLtx2GoldenTol 5e-6 (~2.7x over the worst arm, 1.81794e-06) and kLtx2FilterTol 1e-7 (~3.4x over 2.98023e-08). The margin is for libm, not for us: our reductions accumulate in double against fixed golden constants, so reduction order cannot reach them, but sin/exp/tanh/sqrt differ by ~1 ulp between implementations and the vocoder composes them deeply. A structural porting error moves these by orders of magnitude, not ulps. GATE. Clean Release build, -Werror, ZERO warnings. Full ctest 391/391 passed. Debug build of the affected targets green with identical counts (a full Debug build of all 391 targets exhausted disk on this box and was scoped down rather than worked around). test_ltx2_vae 10 cases / 1120 assertions -> 16 cases / 1816 assertions goldens md5 3af38d1d... -> e96b7a582ce7548d10ed3c40de725e3d The new md5 is expected: the goldens gained the revision anchor and six arms. Every PRE-EXISTING golden line is byte-identical (diff shows zero removed or changed lines), and the file still rebuilds byte-for-byte from the generator. DOC CHECKPOINT. b0aa475 argued an exception for docs/FEATURES.md and docs/USAGE.md, on the grounds that the campaign runs several implementers in parallel and the operator holds the record surfaces. This commit PAYS that debt instead of inheriting it, with one keyed FEATURES row and one scoped USAGE subsection stating that LTX-2.5 has no user-facing entrypoint yet -- so a reader cannot infer a render is available -- and naming the DiffVAE refusal. The range checker still reports b0aa475 itself, which cannot be repaired without rewriting history; the campaign lands as one squashed PR, at which point the combined change carries both documents. Every mutation above restored the tree byte-for-byte, verified by md5. FOLLOWING_AGENTS_PROTOCOL Refs #435 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Two findings about the METHOD, from the L2/L3/L4 review rounds. They compound, and together they change what this campaign's evidence is evidence OF. Recording them in 7.0 because they apply to every future brick, not just LTX-2.5. (a) There is a CLASS of constants a reduced-dimension golden cannot see. Not one constant, a class, confirmed independently on two phases. An epsilon, a clamp bound or a normalize floor only becomes load-bearing in a regime the synthetic fixture never enters. Seven are now tabulated with the mutation that stayed green, including a norm_eps that tolerates a 100x change and a BWE mel log clamp that the fixture cannot saturate because mel_basis is built non-negative and well-scaled, while REAL SILENCE saturates it in production. The repair is not a tighter tolerance, because the tolerance was never the problem. It is a source-anchored constant assertion pinning the value to the upstream line it came from, plus, where feasible, a golden arm whose input actually enters the regime. (b) Byte-identical goldens are NOT evidence the oracle was right. L4's repair tested this rather than assuming it: with a decoy ltx_core differing by ONE drifted constant and path precedence defeated, the generator imported the decoy, exited 0, and emitted goldens whose md5 matched the real ones exactly. That is (a) and (b) compounding. A drifted constant from the invisible class produces identical goldens, so "the goldens reproduce byte-for-byte" cannot distinguish the right oracle from a wrong one. Reproducibility proves determinism. It does not prove provenance. So every generator here owes TWO separate things: assert the resolved ltx_core.__file__ lives under the --ltx2 checkout (identity), and record the upstream revision SHA in the emitted goldens (provenance). AGENTS.md already required the anchor; the identity assertion is what makes the anchor mean anything. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Phase L2, reviewed and repaired. The 21.00B dual-stream DiT forward, gated against upstream ltx_core executed at reduced dimensions on CPU. Parity, independently reproduced by the reviewer rather than taken on the implementer's word: 1.19209e-07 video / 4.84288e-08 audio on split RoPE, with interleaved RoPE, the float64 frequency ladder, the mask cases and audio-disabled all in the same band, plus a new dense-mask arm at 5.96046e-08. RED-first found a real defect on the first run: computing rope.py's linspace in double instead of reproducing ATen's f32 half-forward/half-backward walk moved two of eight AUDIO samples by one ulp, which the 5-sample video ladder hid completely. The reviewer went further and showed the FULL FORWARD also misses it, because the drift stays under the round-off bar; only the bit-equality ladder assert catches it at all. Review returned FAIL on two ungated shipped paths, both now closed. The prompt K/V cache carried no prompt identity and its only validity check was on SIZE, so two requests whose prompts differ but tokenize to the same length would have rendered the second with the FIRST request's prompt, silently. It now carries an FNV-1a identity over both streams' contents, geometry and masks, refuses naming what moved, and offers an additive Reset(). The dense [Tq, S] attention bias had ZERO coverage. One methodological note worth keeping: the reviewer's original F1 mutation did not compile under -Werror, so its 'green' run was a STALE BINARY. The repair verified the coverage gap with a semantically identical mutation that does compile, rather than dismissing the finding. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] # Conflicts: # tests/CMakeLists.txt
…, pin the epsilons Repairs the three findings from the adversarial review of phase L3 of .agents/specs/ltx-2-5.md (issue #435, PR #437). The verdict was PASS with 3 findings, none blocking: the math was right and every claimed number reproduced. This closes the contract hole and the two recorded gaps. No numerics change: all 8 measured max|diff| values are byte-for-byte what L3 recorded, and scripts/gen-ltx2-text-goldens.py still regenerates the existing goldens BYTE-IDENTICALLY (md5 1a803faa... before, and the new section is purely appended). F1, MEDIUM. THE DECLARED CONTRACT WAS NEVER CHECKED AGAINST THE WEIGHTS. `Ltx2TextFeatureConfig::aggregate_bias` was written by the selector (ltx2_text_encoder.cpp:130, :166), asserted in the selection test and NEVER READ by the forward, which keyed entirely on `w.bias.empty()`. `video_out_features` fed only `_rescale_norm`; the width that actually ran came from `w.out_features` and nothing compared the two. `const int64_t flat = config.FlatDim()` was dead. The reviewer demonstrated both, and they ran silently: PROBE(a): bias-less weights under aggregate_bias=true ran SILENTLY, video drifts 0.0194063 from upstream PROBE(b): out_features mismatch ran SILENTLY, video size = 80 while config claimed 40 [doctest] test cases: 16 | 16 passed | 0 failed Status: SUCCESS! That is phase L6's most likely mistake, not a hypothetical: the loader reads `video_aggregate_embed.weight` (U8/NVFP4) and misses `.bias` (BF16, a DIFFERENT dtype on a different unpack path) while the config still says bias=True. Every conditioning row is then shifted by the missing bias and every padded row projects to 0 instead of to the bias. Finite, correctly shaped, wrong prompt. ltx2_text_encoder.h:57-60 already listed exactly this under "THE FOUR THINGS THAT FAIL SILENTLY" — the header named it and the code did not enforce it. `RequireDeclaredProjection` now refuses, BY NAME and before any work, on all three axes: `aggregate_bias` vs `w.bias.empty()`, `*_out_features` vs `w.out_features`, and `FlatDim()` vs `w.in_features`. The third is what the dead `flat` becomes, so the "+1 is the embedding layer" trap is caught from the weight side too. Only the projections that actually RUN are checked: V1 uses `video` alone and returns it twice under `is_av` (feature_extractor.py:95-96), and V2's audio arm exists only when the config gave it a width. Upstream cannot hit any of these because it builds both Linears from the one config object (encoder_configurator.py:187, 206-208); a port that resolves the config and the tensors on separate paths can. The same two probes, re-run verbatim against the fixed library: PROBE(a): REFUSED -> ltx2 text encoder: video projection: the config declares aggregate_bias=true but the supplied weights carry NO bias. ... PROBE(b): REFUSED -> ltx2 text encoder: video projection: the config declares out_features=4 but the supplied weights are 8 wide. ... CONTROL : accepted, video size = 80 audio size = 40 probes refused = 2 of 2 No caller breaks: `Ltx2TextFeatureExtractorForward` and `Ltx2TextEncoderConditioning` have no production call site (grep over src, include, tests, examples), and the CONTROL line above is the honest half of the gate — the correct weights still go through the same door, asserted for V1, V2 and both entry points. F2, LOW. TWO INVISIBLE EPSILONS, THE SAME CLASS L4'S REVIEW FOUND. Both constants were present and faithful to upstream and nothing held them. Measured on this suite BEFORE the pins: `range + kEps` -> `range + 0.0f` moved norm V1's max|diff| from 4.77e-07 to 5.24521e-06 and the gate still said SUCCESS, because 5.2e-06 is under the suite's 1e-5 bound. `denom + kEps` -> `denom + 0.0` moved nothing. The reason is structural, and it is why the fix is not "one more golden": an epsilon that only changes the answer on a DEGENERATE input is invisible to any golden built from random values. So both are now held TWO ways. * The VALUE, against upstream MEASURED rather than restated. The generator recovers each epsilon numerically from upstream by a probe whose algebra inverts it exactly: for V1, two valid tokens differing by exactly r make range_ == r and the DIFFERENCE of the two outputs cancels the mean, giving eps = 8r/(y0-y1) - r; for V2, a single element v gives eps = (v/y)^2 - v^2. No source parsing and no restating our own constant back to ourselves. Recovered: 9.9998279390984754e-07 and 9.9999993911447867e-07. * The DEGENERATE INPUT on which each is the only thing between the port and a division by zero, with upstream's own behaviour on that input emitted as a golden so the property asserted is measured and not invented. `kLtx2TextNormV1Eps` and `kLtx2TextNormV2Eps` are now named in the header, used by the .cpp, and carry their reachability conditions next to them, plus a header note that the epsilon limit is a CLASS: when a fourth epsilon arrives it owes the same pair, not a comment saying it matches upstream. MUTATION-VERIFIED, tree restored byte-for-byte (md5 checked both ways): M1 range + kEps -> range + 0.0f FAILURE, 168 assertions, the new case (7 valid tokens x 24 features, all NaN) M3 kLtx2TextNormV1Eps -> 0.0 FAILURE, 169 (168 + the value pin) M4 kLtx2TextNormV2Eps -> 0.0f FAILURE, 27 across 2 cases M5b variance + kEps -> + 0.0f*kEps FAILURE, 26 across 2 cases M6 encoded_mask < 1e-6f -> < 0.0f FAILURE, 34 M2 denom + kEps -> denom + 0.0 SUCCESS, still invisible. REPORTED, not papered over: see below. M2 is stated in the test and the header as a LIMIT rather than a pass. That epsilon is unobservable at the output FOR EVERY POSSIBLE INPUT, not merely for this fixture: `denom == 0` requires a batch row with no valid token, and every position of such a row is a pad that feature_extractor.py:44-45 zeroes, so the NaN it prevents in the intermediate `mean` can never escape. Upstream behaves identically, which the generator now emits as a golden (kLtxTeNormV1ZeroLenRowIsZero). It is mirrored because upstream has it and held by the constant assertion, which M3 proves is load-bearing. Claiming a behavioural gate on it would have been false. M5 in its natural form (`variance + kEps` -> `variance`) does not COMPILE: -Werror=unused-variable. That is the exact trap this campaign was warned about, and the harness reported BUILD_EXIT=1 and refused to run the stale binary rather than printing a green gate from a build that never happened. M5b is the compile-clean equivalent. F3, LOW. `f32` ON A MODEL PATH WITHOUT A LOCAL REASON. `Gemma4HiddenStatesResult` returns `std::vector<std::vector<float>>` downloaded from BF16 device buffers (gemma4.cpp:571-578) while upstream runs this path in bf16 (base_encoder.py:41, `dtype: torch.dtype = torch.bfloat16`) and its `hidden_states` tuple is bf16. The reason existed in ltx2_text_encoder.h:62-72 and in L3's commit body, and every LTX entry point refuses a non-f32 compute dtype rather than widening silently, which is gated. But gemma4.h stated "host f32" as bare fact and AGENTS.md's dtype-polarity rule wants the line next to the buffer. It is there now, naming the CPU-reference-arm reason, upstream's bf16 dtype, the host cost (~771 MB at 49 x 1024 x 3840 x 4B where upstream holds ~385 MB) and phase L6 as the owner of the bf16 arm. Comment only; no numerics touched. DOCS, and why this commit touches them. The operator scoped docs/* out of this task, but check-doc-checkpoint --staged is RED without them: src/vllm/model_executor/models/ classifies as feature_surface and include/vllm/ as user_usage, both by blanket prefix. The edits are minimal, keyed and LTX-2.5 only: the existing FEATURES.md row for this feature moves 15/15 -> 17/17 and gains the refusal (no new row, no collision with L2's or L4's rows, and the cell stays under check-public-doc-tables' 220-char entry cap), and USAGE.md's existing LTX-2.5 section gains one paragraph on the contract refusal, which is user-visible behaviour for anyone wiring a loader. BENCHMARKS.md is untouched: this is not a lifecycle change. Flagged for operator reconciliation rather than done silently. EVIDENCE. RED before F1: 17 cases, 16 passed / 1 failed; assertions 3340 | 3330 passed | 10 failed; Status: FAILURE! — every failure "did NOT throw at all!", which is the intended reason. GREEN after: 17 cases / 3340 assertions, Status: SUCCESS! (was 15 / 2473) All 8 max|diff| values UNCHANGED: stack 0 BIT-equal; norm V1 4.76837e-07 both sides; norm V2 2.38419e-07 both sides; V2 extractor 4.47035e-08 / 5.96046e-08 video, 2.98023e-08 audio; V1 extractor 6.51926e-08 / 5.96046e-08; conditioning 4.47035e-08 / 5.96046e-08 video, 2.98023e-08 audio; gemma4 lm_head invariant 2.04891e-08 over max|value| 0.166625. Goldens regenerate BYTE-IDENTICALLY; the revision anchor fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca is still present. Gemma-4 gate counts UNCHANGED: test_gemma4_honesty 2/6, test_gemma4_rocm_fp8_seams 2/10, test_gemma3_forward 3/503, test_tool_parser_gemma4 72/161, and test_gemma4_{vision_tower,registry_e2e, audio_tower,paged_engine} 1 case / 0 assertions each. All SUCCESS! Clean Release -Werror CPU-only rebuild from an EMPTY tree, 0 warning lines; full ctest 391/391, 0 failed. Environmental, reported as such: the first clean rebuild died with "No space left on device" writing /tmp/cc*.s at 100% disk. That is the box, not the change; it cleared and the rebuild was redone from empty. FOLLOWING_AGENTS_PROTOCOL Refs #435 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…435) Phase L4, reviewed twice and repaired between. Both VAE decoders gated against upstream ltx_core executed at reduced dimensions, max abs diff 1.16e-08 to 1.79e-06 across eight bricks, every value independently reproduced by two reviewers. Three things this phase established that are worth more than the numbers. The audio VAE is NOT end-to-end causal, and it was MEASURED rather than assumed: the first causality probes failed and upstream agreed with the failure. causality_axis governs the convolutions, but the AttnBlocks attend over the whole (time, mel) map. The Conv video decoder is not causal either, for an unrelated reason: a one-group GroupNorm whose statistics span time. Both are now gated in two parts, with upstream supplying the expected windows. Byte-identical goldens do NOT prove the oracle was right. The repair tested this instead of asserting it: a decoy ltx_core differing by ONE drifted constant, with path precedence defeated, produced goldens with an IDENTICAL md5. The generator now asserts the resolved ltx_core.__file__ is the checkout's and records the upstream revision. Identity and provenance are separate obligations and this phase needed both. The shared-seam finding took the harder route. Rather than documenting the duplication as an exception, H3's 1-D primitives were promoted to the shared header and 172 lines of LTX duplicate deleted. It is genuinely one seam now, proven by mutating a shared primitive with ONE edit and watching BOTH suites go red, where at the base SHA that same edit could not reach LTX at all. H3's own gates measured identical at both SHAs by the reviewer: 79 cases / 57395 assertions, and 6 / 137. docs/FEATURES.md conflicted on the shared LTX-2.5 row, which is the collision the L4 reviewer predicted. Resolved per AGENTS.md by taking the target row wholesale and reapplying the scoped edit, so both phases' claims survive in one row rather than either eviction winning. Both CMake lists resolved as unions. Two MEDIUM findings from the re-review stay OPEN and are owed on this PR: a DIRTY upstream tree still defeats the revision anchor (rev-parse reports the committed SHA regardless of uncommitted edits), and the audio decoder's own norm_eps is an unpinned member of the invisible-constant class its sibling pins. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…put that proves it Closes the one MEDIUM finding the scoped re-review of L3's repair returned FAIL on (issue #435, phase L3 of .agents/specs/ltx-2-5.md). The rest of that repair is untouched and re-verified below. THE DEFECT. `Ltx2NormAndConcatPaddedBatch` computed `denom + eps` in double. Upstream computes it in float32, and not by choice: `sequence_lengths` is an int64 tensor (feature_extractor.py:30), `* d` keeps it int64 (:34), and torch promotes an int64 tensor plus a python float to the DEFAULT dtype, so the add at :35 happens in f32. Measured against the pinned oracle at fd4ded7f: torch: (int64 18) + 1e-6 -> dtype=torch.float32, 18.000001907348633 our double: 18 + 1e-6 = 18.000001000000001 upstream mean = f32(9.0/18.000001907348633) = 0.49999994039535522 our mean = f32(9.0/18.000001000000001) = 0.49999997019767761 One f32 ulp. This is AGENTS.md's dtype-polarity rule in its exact stated form: too WIDE is still numerically finer, so tokens match, every random golden passes and nothing goes red. `range_ + eps` at :41 then multiplies that ulp by 8/eps = 8e6 whenever a (batch, layer) slice is constant, which turns it into 23842x the suite's own kTol. THE GATE WAS DEFINED DOWN. scripts/gen-ltx2-text-goldens.py ran upstream on exactly the falsifying input, had the whole output tensor in `const_out`, and reduced it to `kLtxTeNormV1ConstantSliceFinite = 1`. Both a float32 and a float64 denominator are finite, so the assertion could not see the defect. Two further degenerate cases were reduced the same way. That is the reason this commit is mostly test and generator: the port defect is two lines. RED FIRST. Emitting the arrays with the dtype still unfixed: test_ltx2_text_encoder.cpp:938: MESSAGE: ltx2 text norm V1 constant slice max|diff| = 0.238419 test_ltx2_text_encoder.cpp:939: ERROR: CHECK( worst < kTol ) is NOT correct! values: CHECK( 0.238419 < 1e-05 ) test_ltx2_text_encoder.cpp:1002: MESSAGE: ltx2 text norm V1 near-constant max|diff| = 0.238419 test_ltx2_text_encoder.cpp:1003: ERROR: CHECK( worst < kTol ) is NOT correct! values: CHECK( 0.238419 < 1e-05 ) [doctest] test cases: 1 | 0 passed | 1 failed | 16 skipped [doctest] assertions: 861 | 859 passed | 2 failed | [doctest] Status: FAILURE! GREEN after mirroring the denominator: those two read 0, and so does the one-valid-token case. 17/17 cases, 3350/3350 assertions, Status: SUCCESS! WHAT CHANGED IN THE PORT. `denom` becomes int64, as upstream's is, and the add and divide happen in f32. The min/max become f32 too: they ARE f32 values, so `hi - lo` is then upstream's single f32 rounding instead of an f64 subtraction rounded twice. The f64 SUM accumulator STAYS, and is now annotated with the measurement rather than left bare. Upstream's `masked.sum` is an f32 reduction but a blocked one, so no straight loop reproduces its order. Measured on this fixture against upstream: f64 accumulate 2.38e-07 (left) 4.77e-07 (right) f32 accumulate 4.77e-07 (left) 2.38e-07 (right) A naive f32 accumulate wins on one mask and loses on the other, which is noise, not a mirror. Accumulating exactly and rounding once is the closest single-rounding approximation to any order, so it is the deliberate escape. THE THREE FALSE CLAIMS. ltx2_text_encoder.h:143-148, the test's section 3 and the generator's docstring each asserted that `denom + eps` is "UNOBSERVABLE at the output for every possible input". It is not. It is unobservable on an all-pad row or an all-zero mask, where :44-45 zeroes every position first. All three now say that, name the falsifying input and its two numbers, and record that on a realistic Gemma workload `range_` is O(1) so the same perturbation lands around 2e-12. A public header that overstates an invariant is worse than one that says nothing, because the next reader trusts it. Also fixes the off-by-one at ltx2_text_encoder.h:139: upstream defines `denom` at feature_extractor.py:34 and adds `eps` at :35. MUTATION. `denom + kEps` -> `denom + 0.0` was Status: SUCCESS before this change, which is what the old comment's "moved nothing at all" recorded. Against the arrays it now fails three gates, at 0.476837 (constant slice), 0.715256 (one valid token) and 0.476837 (near-constant). Tree restored byte-for-byte afterwards, md5 6ccebc95657cb91613e486152bc1925f. NOT REGRESSED. The generator reproduced every pre-existing golden BYTE-IDENTICALLY (219 insertions, 0 deletions), so the upstream anchor is intact and no prior value drifted. The epsilon value pins are unchanged at 9.9998279390984754e-07 and 9.9999993911447867e-07. F1's `RequireDeclaredProjection` refuses by name on all seven out-of-tree probes and the control is still accepted; A/B'd against a library built from the pre-change source, the probe output is byte-identical. gemma4.h is not in the diff, and its eight gates read 2/6, 2/10, 3/503, 72/161 and 1 case / 0 assertions x4, all SUCCESS. Of the eight max|diff| values, six are unchanged and the two the reviewer predicted improve: norm V1 (left) 4.76837e-07 -> 2.38419e-07, V1 extractor (left) 6.51926e-08 -> 5.96046e-08. Nothing got worse. docs/FEATURES.md carries the extractor number, so it moves 6.5e-08 -> 6.0e-08. Full gate: 391 tests, 389 passed under -j 4; test_serve_low_tools and test_engine_core_proc failed on contention and both pass serially. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Phase L3, three review rounds. The multi-layer feature aggregation, both normalization variants, the two caption projections, mask and pad handling, and the embedded tokenizer/asset pack. The architectural finding first: LTX does NOT condition on the encoder's last hidden state. feature_extractor.py normalizes [B, T, D, L] and concatenates ACROSS LAYERS, so the projections take 3840 x 49 = 188160 inputs. The operator's brief said 94080 = 1920 x 49, reading the checkpoint's NVFP4-packed U8 widths as logical; L3 caught it. Two values per byte, so every packed width is half. The BF16 tensors in the same file settle it. Round 1 PASSED with three findings: the declared weight contract was never enforced, two epsilons were invisible, and an f32 lacked its reason. Round 2 fixed those but FAILED re-review, and that failure is the one worth recording. The port computed denom + eps in DOUBLE where upstream promotes int64 + python float to FLOAT32 (feature_extractor.py:34-35). One f32 ulp in the mean, amplified by 8/eps = 8e6 wherever the range collapses. It survived because the generator RAN upstream on the falsifying input, had the full output array, and reduced it to a single boolean, so the test asserted only isfinite. Emitting the array shows 0.238418579 against upstream's 0.476837158 -- 23,842x the suite's own tolerance, on a case the test already constructed. That is exactly the too-WIDE dtype AGENTS.md says a token gate cannot catch, hiding behind an oracle narrowed to a bool. Round 3 mirrors upstream's arithmetic width and emits arrays. The re-reviewer checked BITS rather than tolerance: 0 differing f32 bits against upstream on all three degenerate inputs, where the old code differed by 72. Two pinned values improved as predicted and none regressed. One judgement kept rather than swept: the f64 sum accumulator stays, because a naive f32 loop wins on one mask and loses on the other -- noise, not a mirror, since upstream's masked.sum is a blocked reduction no flat loop reproduces. The measurement is annotated beside it. The re-reviewer independently confirmed that torch.sum over the same gathered values also fails to reproduce upstream's order, which is the stronger datum. docs/FEATURES.md conflicted again on the shared row. L3 branched before L2 and L4 landed, so its side carried a STALE DiT row plus a genuinely new text row. Resolved per AGENTS.md by keeping the target row wholesale and reapplying only the scoped addition, so no phase's claims were evicted by an older snapshot. Open and owed, not dropped: the V2 per-token-RMS path carries the same too-wide class (it squares in f64 where upstream squares in f32) and wants measuring at shipped widths rather than a blind narrowing, since the fixture runs D=6 against a shipped 3840. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Phase L5 of .agents/specs/ltx-2-5.md, issue #435: the flow-matching schedule, the noiser, four diffusion steps, guidance, the patchifiers, the recipe table, the latent spatial x2 upsampler, the duration head, and the Embeddings1DConnector that was orphaned between L2 and L3. Every brick is gated against the UPSTREAM module executed at reduced dimensions on CPU, with both sides rebuilding weights and inputs from one deterministic FNV-1a + splitmix64 stream, so no weight byte is checked in. Measured max abs diff per brick, worst arms: schedules 2.80e-06, upsampler 1.19e-06, connector 7.75e-07, diffusion steps 5.96e-08, duration head 0 (pooler 8.94e-08), patchifiers and perturbations bit-exact. 33 cases / 1512 assertions. WHERE THE VALUES COME FROM, because the two references disagree. vLLM-Omni is the binding oracle and its recipe table stops at 2.3, so the three rows this adds -- one_stage 2.4, one_stage 2.5, distilled_two_stage 2.5 -- take their values from Lightricks ltx-pipelines, which is the spec's designated cross-check. The SHAPE of the recipe model stays vLLM-Omni's, and its refusal on an unknown (kind, version) pair is mirrored exactly rather than relaxed: a plausible but wrong sigma schedule renders a video instead of failing. The two references also ship DIFFERENT default negative prompts; both strings are kept, each row takes its own source's, and the disagreement is a gated value rather than a preference. Recorded in .agents/porting-inventory.md §9.18. THREE PROJECTION GUIDERS ARE REFUSED, and the refusal is a measurement. CFGStarRescalingGuider, LtxAPGGuider and LegacyStatefulAPGGuider all multiply projection_coef's rank-2 (B, 1) result straight into the latent, which torch right-aligns onto the latent's last two axes: the expression composes at rank 2, raises at every rectangular rank >= 3 -- i.e. at every real (B, C, F, H, W) video latent -- and on a coincidentally square shape silently indexes an axis that is not the batch. No ltx-pipelines entry point constructs any of the three. The shape matrix that measurement produced is itself a golden, so upstream repairing the shapes fails this gate rather than going unnoticed. THE GENERATOR REFUSES A DIRTY UPSTREAM TREE. Spec §7.0(b) established that byte-identical goldens are not evidence the oracle was right; the revision anchor is what makes them interpretable. An anchor read from a tree with uncommitted edits stamps a clean SHA on goldens that SHA does not produce, which survives a bisect and misdirects it, so both --ltx2 and --vllm-omni must be clean. The oracle-identity assertion (resolved ltx_core.__file__ under --ltx2) is kept from L2/L4. TWO FINDINGS RECORDED RATHER THAN PAPERED OVER. (a) The duration head's fixture had to be WIDENED, and that is a gate property. Its output is exp(...) through an attenuating chain, so at the shared 0.05 parameter scale its both / video-only / audio-only arms collapsed to within 2.98e-06 of one another -- BELOW this suite's round-off bound. A gate that cannot separate its arms would accept an implementation that ignored one of the two streams. Measured spreads: 0.05 -> 2.98e-06, 0.2 -> 2.3e-03, 0.35 -> 4.9e-02. The fixture uses 0.35 and the separation is now asserted, so a future change that collapses it fails loudly. (b) The concat ORDER of the two streams is NOT observable, and that is upstream's property, not a hole. A mutation reversing it left every golden green; AttentionPooler is cross-attention with no mask and no positional encoding over the token axis, so it is permutation invariant. Measured on upstream: a reversed concat and a random permutation each move the pooled output by 2.98e-08, while giving the audio stream the VIDEO modality embedding moves it by 4.80e-03. What tags a stream is the embedding, and both facts are now gated. INVISIBLE-CONSTANT CLASS (spec §7.0(a)). Each constant this phase introduces carries a source-anchored assertion, and the two whose regime is reachable also carry a golden that ENTERS it: kLtx2CfgPpAlphaEps binds at step 0 where sigma is exactly 1.0, and kLtx2ProjectionCoefEps binds on an all-zero project_onto. The unreachable ones -- kLtx2Res2sSigmaUpClamp, kLtx2UpsamplerNormEps, kLtx2ConnectorRmsNormEps -- are pinned against their upstream line and said to be unreachable rather than claimed as covered. DTYPE. f32 throughout, which is upstream's OWN width on these paths (the noiser lerps in .float(), every diffusion step casts to torch.float32, the guider opens with cond.float()). The two schedulers do NOT agree on width and both are mirrored: LTX2Scheduler is float32 tensors with a Python-double shift term, LinearQuadraticScheduler is double throughout and narrows once at the end. The one deliberate NARROWING is Embeddings1DConnector.learnable_registers, which upstream stores in bfloat16; keeping it at f32 would be wider than upstream and only the padded positions would move, so the rounding is explicit and gated on its own golden. DOCS. docs/FEATURES.md and docs/USAGE.md are edited because check-doc-checkpoint --staged refuses otherwise: the checker classifies any src/vllm/model_executor/models/ and include/vllm/ change as feature_surface and user_usage. The edits are minimal and keyed -- the existing LTX-2.5 row and the existing LTX-2.5 sections -- and they say what is now gated and what is still refused, not that a render exists. Refused by name and recorded as owed: the temporal x2 upsampler, LoRA fusion, multishot, int8-convrot, CFG/multi-GPU parallelism, VideoEngine wiring (L7), the Beta scheduler (it inverts a Beta CDF through scipy and no pipeline constructs it), the upsampler's dims=2 arm, and the three projection guiders above. Unchanged baselines: test_ltx2 29/1615, test_ltx2_vae 16/1816, test_minimax_h3 79/57395, test_minimax_h3_video_fold 6/137, test_video_engine 11/254, test_capi 'capi v12*' 4/102. Full ctest 394/394 on a clean Release -Werror CPU-only build with zero warnings. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
L3 landed after this branch's base (0cb654a), so the two share three keyed surfaces: CMakeLists.txt, tests/CMakeLists.txt and docs/FEATURES.md. Each is resolved by taking the TARGET branch version wholesale and reapplying this branch's scoped edit, per AGENTS.md §Records, rather than accepting an automatic three-way merge of a keyed record. Concretely: CMakeLists.txt / tests/CMakeLists.txt L3's text-encoder entries kept, L5's four pipeline entries appended after them. docs/FEATURES.md L3's new text-conditioning row kept; the LTX-2.5 DiT row takes L5's text. docs/USAGE.md auto-merged, both sections verified present and intact. ONE KEYED CELL CORRECTED, because the merge makes it false. L3's text row said "`Embeddings1DConnector` and the quantized arm are owed". L5 ships the connector, so the cell now reads "`Embeddings1DConnector` landed in L5; the quantized arm is owed". Leaving it would have recorded an owed item that is not owed, which is exactly the drift a keyed table exists to prevent. Re-gated on the MERGED tree, not on either parent: clean Release -Werror CPU-only build with zero warnings, full ctest 395/395. Unchanged baselines hold: test_ltx2 29/1615, test_ltx2_vae 16/1816, test_minimax_h3 79/57395, test_minimax_h3_video_fold 6/137, test_video_engine 11/254, test_capi 'capi v12*' 4/102, plus L3's own test_ltx2_text_encoder 17/3350 and L5's test_ltx2_pipeline 33/1512. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… same defect
7.0 gains a third finding, from L5's review, and it is the sharpest instance
this campaign has produced.
L5's linspace mirror -- the two-sided walk that lands the last sigma on EXACTLY
0 -- was declared load-bearing in a comment and was NOT GATED AT ALL. Swapping
it for a naive forward walk left 33 cases and 1512 assertions green. That is not
cosmetic: for 23 of the first 198 step counts the naive walk misses exact zero,
and at steps=41 a 5.96e-08 terminal survives the sigma==0 guard, takes the shift
transform, and displaces last_non_zero, moving the WHOLE schedule by 0.1 so the
denoise loop never reaches zero noise. A plain --steps 41 renders confidently and
wrongly.
It hid because the fixture exercises steps in {8,6,5,1,4,7}, every one of which
the broken walk happens to get right.
Two more from the same phase, both caught before landing. The duration head's
three arms collapsed to within 2.98e-06 at the fixture's scale, BELOW the
round-off bound, so an implementation ignoring one input stream would have
passed. And a guider-refusal probe matrix omitted the B=1 row, so a claim that is
false at batch 1 -- the ordinary single-request shape -- was recorded as a
golden.
The generalization is what makes this worth a spec section. (a) is a CONSTANT the
fixture never drives into its active regime; (c) is an INPUT the fixture never
drives into the regime that discriminates. Same failure, different axis. A golden
proves only what its inputs can distinguish, so for every brick the question is:
what input would tell a correct implementation from a plausible wrong one, and
does the fixture contain it? Sweeping a parameter costs almost nothing and is
what turns a golden from a witness into a gate.
Issue: #435
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ctually carries
Materializes the SHIPPED LTX-2.5 checkpoints onto the contracts phases L2 and
L3 committed: the FP8 DiT, an NVFP4 DiT, and the torchao-NVFP4 Gemma-4 text
encoder with its embedded tokenizer pack.
NO NEW QUANT SCHEME. The text encoder is torchao, not compressed-tensors, and
the difference was verified rather than assumed: every quantized module carries
a `torchao_nvfp4` U8[240] marker reading `{"format": "torchao_nvfp4",
"block_size": 16, "is_swizzled_scales": true, ...}`. The encoding is the modelopt
W4A16 one we already have, and `weight_scale_2` is a MULTIPLIER — confirmed
numerically on the real file, not inferred: q_proj's scale_2 is 1.35e-4 with a
group-scale maximum of exactly 448.0, so 6*448*1.35e-4 = 0.363 is a plausible
amax, where the compressed-tensors DIVISOR convention would imply 1.99e7. The
one delta is that the group scales are stored in the cuBLAS block-scaling-factors
layout, so `Ltx2UnswizzleNvfp4BlockScale` inverts the permutation vLLM's own two
producers apply and hands the result to an UNCHANGED `DequantNvfp4ToBf16`.
MEASURED, AND IT CONTRADICTS THE SPEC. The shipped FP8 DiT carries four module
families phase L2 does not port, and two of them falsify committed assumptions:
`prompt_adaln_single` / `audio_prompt_adaln_single` exist only when
`use_prompt_adaln_single` is TRUE (model.py:222-226), which is the opposite of
spec section 1.2 and ltx2.h:115-117 — so the prompt-K/V cache's premise, that
the prompt modulation carries no timestep term, does not hold for this
checkpoint; and `keyframes_abs_pos_embedding` contradicts ltx2.h:47-49. The two
`*_embeddings_connector` towers were already recorded as owed. None is dropped
silently: the load is REFUSED by name and an explicit opt-in still reports every
one of them. Reconciling the spec is the operator's; this change reports.
DTYPE. bf16 is the default, which is the checkpoint's own model dtype; the F32
scale_shift tables stay F32 because the FILE stores them F32. `widen_to_f32` is
opt-in and exists for one caller, the f32 parity forward L2 declared.
Also fixes `BindLtx2DitWeights`, which ltx2.h:228-232 promised would throw BY
NAME and did not — "a required tensor is missing from the weight map" cannot
tell a caller which of 4078 parameters it forgot. Red first, at
tests/vllm/models/test_ltx2_loader.cpp.
Gated three ways, none of them substitutable:
* the SHIPPED manifests, captured from the files' own headers with no payload
read (6124 and 1688 tensors). Every non-scale name is accounted for:
4078 contract + 258 connectors + 12 prompt-AdaLN + 1 keyframes = 4349.
* a few hundred of their own bytes, with the expected values decoded by TORCH
— an fp8-e4m3 implementation that is not ours. The DiT's FP8 head matches
bit for bit; the torchao path's per-row group-scale probe is within half a
bf16 ulp.
* synthetic files for whole-model materialization, the by-name refusals, and
the load-time device staging.
Two mutations, tree restored byte for byte: perturbing the unswizzle index
failed 4 cases / 9 assertions including the real tile; deleting the unswizzle
from the dequant path failed 3 cases including the real-bytes probe (which was
strengthened to reach every row after the first mutation showed row 0 alone is
invariant under the permutation).
OWED, recorded in .agents/porting-inventory.md section 9.18: the swizzle oracle
is a pinned TRANSCRIPTION, not a running one — vLLM is not importable on this
host — though the generator diffs its transcription against the checkout and
refuses on drift; Lightricks' first-party NVFP4 DiT is behind an un-accepted HF
gate (403) and was not downloaded, so that arm is synthetic-only; and the
torchao arm of the Gemma TOWER is not wired, because ltx2_text_encoder.h
declares no tower contract.
Full ctest 393/393. Baselines unmoved: test_ltx2 29/1615, test_ltx2_vae 16/1816,
test_ltx2_text_encoder 17/3350, test_minimax_h3 79/57395,
test_minimax_h3_video_fold 6/137, test_video_engine 11/254, test_capi v12 4/102.
Issue: #435
Spec: .agents/specs/ltx-2-5.md (phase L6)
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
… here L6 loaded the real checkpoint and disproved the headline claim of this spec, of issue #435 and of the PR. I verified it against the checkpoint myself before correcting anything. The claim was that 2.5 sets use_prompt_adaln_single=false, so the cross-attention K/V carry no timestep term and can be computed once per request and reused across all denoise steps. The shipped ltx-2.5-22b-distilled-transformer-fp8 checkpoint carries 12 tensors that only exist when that flag is TRUE, including prompt_adaln_single.emb.timestep_embedder.linear_1 at [4096, 256] -- and 256 is the sinusoidal timestep width. model.py:223-227 builds the module only when cross_attention_adaln and use_prompt_adaln_single. So the K/V DO carry a timestep term here and caching them would be wrong. The mistake is precise and worth naming: I quoted transformer.py:441 as proof of "no timestep term at all" and stopped two lines early. :442-443 add one whenever prompt_timestep is not None, and the comment immediately above them spells out both branches. What made it convincing was that three separate true facts pointed the same way. model_configurator.py:74-76 really does document KV-cacheable checkpoints as setting the flag false. The checkpoint really does carry the static prompt_scale_shift_table, 96 of them. Line 441 really has no timestep. None of the three says the flag is false for THIS checkpoint, and the tensor that settles it was never looked for -- the header dump that would have shown it was filtered with 'adaln_single' not in k. NO SHIPPED DEFECT, and the reason is worth recording. L2's implementer read upstream's conditional correctly even though my brief handed it the wrong conclusion, and wrote a refusal at ltx2_dit.cpp:672 that fires by name before any block runs. The cache is correct-and-inapplicable rather than silently wrong, and stays gated bit-identical and prompt-bound for any checkpoint that does set the flag false. The lesson is the one 7.0 keeps teaching from a new angle: a claim assembled from three true facts is not thereby true. The decisive test here was one grep against the checkpoint, and it went unrun because the conclusion already looked supported. Corrected in the spec, the roadmap portfolio row, the model-matrix row and docs/FEATURES.md. Retracted in place rather than quietly edited, because the wrong claim was committed, published in an issue and a PR, and reported. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…on a TRUE premise Closes the two MAJOR findings from the adversarial review of phase L5, plus its two MINOR ones. Issue #435. Phase L5 of .agents/specs/ltx-2-5.md. F1 -- THE linspace MIRROR WAS UNGATED, AND `--steps 41` RENDERED WRONG. `LinspaceF32` walks the second half backwards from `end`, which is the only reason the terminal sigma is exactly 0. Nothing tested that: replacing it with a naive `start + step * i` forward walk left the whole suite green, because the fixture's step counts (8, 6, 5, 1, 4, 7) are all counts the naive walk happens to get right. 24 of the first 198 counts are not. At steps=41 the residual 5.96e-08 survives the `sigma != 0` guard, takes the shift transform, and becomes the stretch anchor, so the WHOLE schedule moves: measured max|diff| 0.146356 and a terminal of 0.1 instead of 0. Adds golden arms at steps=41 and 47 (the first two affected counts above 1) and a sweep asserting exact-0 termination over steps 1..200, so the property is gated by the arithmetic rather than by which counts someone thought of. RED first: the naive walk now fails 2 cases / 27 assertions. `LinspaceF32` is the only construction of this kind in the phase; `Ltx2LinearQuadraticSchedule` terminates on a literal 1.0 and is structurally exact. F2 -- THE GUIDER REFUSAL WAS RECORDED ON A FALSE PREMISE, IN SIX PLACES. The claim was that `projection_coef`'s rank-2 (B, 1) result "raises at every rectangular rank >= 3, i.e. at every real (B, C, F, H, W) video latent". Measured against upstream, that is false. torch right-aligns (B, 1) onto the last two axes, so the real predicate is `B > 1 && shape[-2] not in {1, B}`. At B = 1, the ordinary single-request latent, it composes AND is numerically correct because (1, 1) is a scalar; (2, 128, 8, 2, 16) composes too. Where it composes with B > 1 it is silently wrong, applying the per-batch coefficient along axis -2. The `norm(dim=[-1,-2,-3])` in the threshold arms is a SEPARATE rank >= 3 constraint. The probe matrix now carries B=1 and shape[-2]==B rows, the `square` predicate is gone (it was a mis-generalization of those two axes), and the test asserts the real predicate in BOTH directions. The REFUSAL ITSELF STANDS, on the premise that is true and was verified independently: nothing upstream constructs these three. CFGStarRescalingGuider, LtxAPGGuider and LegacyStatefulAPGGuider appear in the whole LTX-2 tree only at their own class statements (guiders.py:31, 78, 129); every pipeline builds MultiModalGuider from MultiModalGuiderParams (utils/constants.py:49-68). That is an unported arm refused by name and recorded as owed, which AGENTS.md permits. Corrected at all six sites: the generator, the test, ltx2_pipeline.cpp, ltx2_pipeline.h (which the review did not list but carried the same text), and .agents/porting-inventory.md 9.18(b), which keeps the correction visible rather than quietly rewriting it. Recording a wrong finding as a golden is exactly what spec 7.0(b) exists to prevent. F3 -- TWO CONSTANTS DOCUMENTED AS PINNED HAD NO ASSERTION. `grep -rn` over tests/ returned zero references for both kLtx2ConnectorRmsNormEps and kLtx2BlurKernelSize. Both are now compared against values the generator READS OFF upstream's own signatures (utils.py:7, blur_downsample.py:14) rather than retyped literals, so upstream moving either fails the gate. Mutating them 1e-6 -> 1e-4 and 5 -> 7 now fails 3 cases / 8 assertions. Both header comments corrected to say what actually gates them. F4 -- DTYPE ANNOTATIONS, AND A CORRECTION TO THE L5 COMMIT BODY. That commit said the upsampler and duration head are "f32 throughout, which is upstream's OWN width on these paths". That is not accurate. Five sites compute in double as REDUCTIONS, which the suite's f64-reduction convention covers (L3 precedent, ltx2_text_encoder.cpp:259-269) but which were unannotated; each now carries a site note. Two more, `Silu` and `GeluTanh`, are POINTWISE and are NOT covered by that convention at all: they are computed WIDER than upstream's f32, the polarity AGENTS.md warns a value gate cannot catch. Annotated as visible debt rather than narrowed, because narrowing moves goldens and owes its own red-first change. Same class as #445. A NEW MAJOR DEFECT FOUND WHILE GATING F1, REPORTED AND DELIBERATELY NOT REPAIRED. The steps 1..200 sweep exposed `Ltx2SigmaSchedule(1, ...)` returning {-nan, 0} where upstream returns {0.10000002, 0}. Root cause measured: upstream's shift is a PYTHON SCALAR divided by a tensor (schedulers.py:43-45) and torch evaluates scalar/tensor as `scalar * reciprocal(tensor)`, yielding 0.99999994 at sigma == 1 rather than 1.0. At steps == 1 that one-ulp residue is the only non-zero sigma and so is the entire stretch anchor; this port computes the same expression in f32, gets exactly 1.0, and divides 0/0. The `OneStep` golden already carries upstream's correct value and did not catch it because `MaxAbsDiff` drops NaN (`d > worst` is false for NaN). Both are left untouched on purpose: mirroring torch's reciprocal-multiply changes every sigma on every arm, and fixing the helper turns the suite red on that defect, so the pair owes its own spec, issue and fresh review rather than a drive-by edit in a review-repair branch. This is the stop condition the task named. The sweep gates finiteness for steps >= 2 and carves out steps == 1 explicitly, so the defect is recorded in the tree instead of only in a report, and the file header no longer claims it mirrors width everywhere. Evidence: CPU-only Release build, exit checked on every build, no output discarded. test_ltx2_pipeline 35 cases / 2358 assertions / Status: SUCCESS (was 33 / 1512). Unchanged: test_ltx2 29/1615, test_ltx2_vae 16/1816, test_ltx2_text_encoder 17/3350, test_minimax_h3 79/57395, test_minimax_h3_video_fold 6/137, test_video_engine 11/254, test_capi 'capi v12*' 4/102. Full ctest -j 4: 395/395, no flakes. Goldens regenerated against LTX-2 fd4ded7f and vllm-omni a4ea67a2, both trees verified clean before and after; zero drift in any pre-existing golden value. The generator's provenance refusals re-exercised: wrong path, dirty --ltx2 and dirty --vllm-omni all still exit 1. docs/FEATURES.md and docs/USAGE.md are touched only because doc-checkpoint classifies any edit under src/vllm/model_executor/models/ and include/vllm/ as feature and usage surface; the entries are minimal and keyed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
#449) Every LTX-2.5 and MiniMax-H3 golden comparison, and both DeepSeek-V4 forward gates, reduced to `max|got - want|` written one of two ways, and BOTH are NaN-blind: form A if (d > worst) worst = d; // NaN is never > anything form B worst = std::max(worst, std::abs(...)); // std::max(a,b) is // `a < b ? b : a`, and // `a < NaN` is false So an all-NaN brick compared against entirely correct goldens reported `max|diff| = 0.0` and passed every bound. The instrument could not see the one defect class it exists to catch. On LTX-2.5 phase L5 it concealed a real `Ltx2SigmaSchedule(1, ...)` returning `{-nan, 0}`, reported green. This replaces the six local copies with ONE shared helper, tests/support/max_abs_diff.h, where a non-finite operand on either side is a FAILURE: `MaxAbsDiffScan` reports the offending index and returns +infinity, which fails every `< tol` bound, and `MaxAbsDiff` additionally raises a doctest failure naming the index and both values, so the `> tol` "these must differ" callers in the DeepSeek-V4 suites cannot read a NaN as a difference either. `vllm_cpp_add_test` now puts tests/ on the include path so any suite can reach tests/support/; no header name under tests/ collides with one under include/ or src/. RED first, twice. 1. The helper's own gate. Built against the OLD reduction body, tests/support/ test_max_abs_diff.cpp reported `test cases: 7 | 3 passed | 4 failed`, `assertions: 33 | 16 passed | 17 failed`, `Status: FAILURE!`. With the guard: `7 passed | 0 failed`, `33 | 33 passed | 0 failed`, `Status: SUCCESS!`. The wrapper's raise is gated too, by the test_max_abs_diff_nan_raises CTest entry: the case is skipped in the normal run (a raised failure would count as one) and forced on there under WILL_FAIL, so it must exit non-zero. 2. End-to-end on a real suite. Forcing Ltx2AdaLayerNormSingle's modulation to all-NaN against its correct golden: with the old reduction test_ltx2 printed `29 passed | 0 failed`, `Status: SUCCESS!`; with the guard it prints `28 passed | 1 failed`, `Status: FAILURE!`, naming index 0, got = nan, want = 0.0129515. The mutation was reverted byte-for-byte (md5 checked). WHAT THE HARDENING UNCOVERED: nothing on this branch. Every affected suite is byte-for-byte green before and after, at identical case and assertion counts: test_ltx2 29/1615, test_ltx2_vae 16/1816, test_ltx2_text_encoder 17/3350, test_minimax_h3 79/57395, test_minimax_h3_video_fold 6/137, all `Status: SUCCESS!`. The shipped MiniMax-H3 gate is unaffected. The two DeepSeek-V4 suites gain 8 assertions each (26 -> 34, 29 -> 37): the shared wrapper REQUIREs equal sizes where the local copy silently compared only the shorter prefix, so each of their four call sites now runs two extra passing REQUIREs. Full `ctest -j 4`: 396/396, 0 failed. The `Ltx2SigmaSchedule(1, ...)` arm the issue predicts would go red is NOT on this branch: test_ltx2_pipeline.cpp and Ltx2SigmaSchedule arrived with L5 (742e38a) on row/LTX25-L5-PIPELINE, which row/MODEL-DIFFUSION-LTX25 does not contain. That file carries the same NaN-blind form and needs the same edit when L5 lands; its OneStep arm should then go red, and that is the point. The reduction is pervasive beyond the six files fixed here: an audit of tests/ finds the same unguarded `std::max`/`>` shape at roughly 112 further sites in 43 other files, including 26 in-body comparisons inside test_minimax_h3.cpp itself and the whole tests/vt/ paged-attention, NVFP4, MoE and Tenstorrent surface. tests/vt/test_ops_mla_attn.cpp, test_ops_mla_prefill.cpp and test_ops_mla_chunked_context.cpp already REQUIRE non-NaN inline and are the in-tree precedent this helper generalizes. Those remaining sites are deliberately NOT touched here: this change is scoped to the helpers issue #449 names, and sweeping them belongs in its own row. No golden reached through this helper legitimately holds Inf or NaN. LTX-2.5's `additive_mask` carries -FLT_MAX, which is finite and compared exactly, not through here; the gate pins that a saturating-but-finite magnitude still compares rather than being refused. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…d the sweep proving nothing (#435, #598) Six review findings on the L9a nibble-order change. The implementation is sound and the correctness argument holds; every one of these is an accuracy or coverage defect in what the change CLAIMS. F1 -- a safety guarantee that is false, stated to users. Three places said any other marker/shape combination is "refused by name". It is not. A marker-less NVFP4 checkpoint whose weight_scale is stored LINEAR [N, K/16] has, for every geometry with N % 128 == 0 and G % 4 == 0, a shape numerically IDENTICAL to the cuBLAS-padded framing. Ltx2ResolveNvfp4Producer returns kNvfp4Prequant for it, the loader then unswizzles scales that were never swizzled AND reads high-first, and the result is finite, correctly shaped, correctly scaled, WRONG. The refusal branch is unreachable for that entire class. That is not an exotic file. LINEAR [N, K/16] is what ModelOpt, llm-compressor and compressed-tensors all write -- vLLM's own readers allocate exactly that shape at modelopt.py:1335-1345 and compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py:73-76 (pin 555967922) -- and none of the three emits a .torchao_nvfp4 sidecar. So the marker's absence excludes torchao and NOTHING ELSE. The decision is still right, because there is no better evidence in the file, and that was checked rather than assumed: the shipped DiT's __metadata__ carries exactly config, gemma_source_checkpoint, model_version and license -- no quantization_config, no producer key, no nvfp4/torchao/quant substring anywhere in the config, and no tensor name mentioning the quantizer over all 7876 tensors. Unlike MiniMax-H3, whose community checkpoint DID name its converter, there is nothing here to key on. So the gate is NOT weakened; the sentence is corrected, in ltx2_loader.h, docs/USAGE.md and the spec's risk table, and the residual hazard becomes a tracked condition at spec section 3.1.1 beside the existing H3 reversal condition. F2 -- the ten-gate "byte-identical" evidence is half vacuous, and two live gates are blind. Instrumented DequantNvfp4ToBf16 with a call counter and re-ran mutation M3 (flip the shared default to kHighFirst) across all ten: RED test_nvfp4_dequant 6 calls, test_gguf_nvfp4 6, test_ltx2_loader 294 GREEN test_qwen3_forward 30 calls, test_minimax_h3 44 calls / 57,395 asserts GREEN the other five make ZERO calls on a default run Three of ten can see a nibble-order change. Five never execute the function, so their identical counts are evidence of nothing about it. Two execute it live and still pass -- and one of those is the project's OTHER high-first family, gated by exactly the statistic this spec proves is blind. If H3 is ever routed through the new kHighFirst while MiniMaxH3Nvfp4HighNibbleFirst() stays default-ON, the two compose into a double flip and nothing in the tree fires. Opened #598 for a nibble-sensitive H3 gate, linked from the roadmap issue table and the spec. F3 -- a recorded number the gate does not produce. Section 7 recorded the wrong-order arm at 0.00514, which matches no row of its own table and no line of the gate. The committed gate reports corr -0.00239115 / rel 1.41856. The control on the same sentence, 0.00362, was exact. Section 5.1 also named "the loader refuses the shape" as THE red-before for a change with two independent halves; a throw can only ever demonstrate the layout half, so both reds are now stated separately with the value each produces. F4 -- a comment contradicted by four call sites. nvfp4_dequant.h said the order is "never inferred and never defaulted per call site". It IS defaulted, and deliberately: minimax_h3_nvfp4.cpp:112, minimax_h3_device.cpp:1311, qwen3_5.cpp:1298 and dense_nvfp4_gemm.h all rely on it. The seam that genuinely has no default is Ltx2DequantNvfp4ToBf16. Comment corrected to say which is which and why. F5 -- the correlation tolerance was loose in one direction and unbounded in the other, RE-BOUNDED strictly stronger. rel_rms is a PREDICTED QUANTITY (the disagreement two different quantizations of the same base weights must show, measured 0.100672), not an error budget. `rel <= 0.15` alone admitted a +10% uniform group-scale error, and Pearson correlation is scale-INVARIANT -- corr reads 0.994968 to every printed digit for every multiplier -- so nothing else in the gate could catch it. It also had no floor, so an arm that reproduced the FP8 oracle exactly scored rel 0.0 / corr 1.0 and passed every correlation assertion. Replaced by a band: 0.085 <= rel <= 0.115, plus corr <= 0.998. Proven strictly stronger by A/B on the SAME binary path, only the bound differing: x1.09 group-scale error inside DequantNvfp4ToBf16, rel 0.1411 old `rel <= 0.15` GREEN 45/45 Status: SUCCESS! band RED CHECK( 0.1411 <= 0.115 ) is NOT correct! the FP8 oracle handed to the gate as its own answer old bounds every CORRELATION assertion passes; only the incidental absmax equality fires band RED on the result itself: rel >= 0.085 AND corr <= 0.998 Both mutations are now permanent arms of the gate rather than one-off logs, in the same style as the wrong-nibble arm: the gate asserts that a x1.10 scale error moves corr by less than 1e-6 (measured 7.35e-11) while pushing rel out of the band, and that the oracle-against-itself lands outside it too. F6 -- record the limit honestly. Section 1.1 claimed "three mutually consistent and independent witnesses". torchao is neither installed nor vendored on this box (import fails; no mx_formats/kernels.py on the filesystem), so its two lines are upstream source only, not executable here. And vLLM's break_fp4_bytes is reached only from dequantize_to_dtype, which serves the ModelOpt and compressed-tensors paths -- vLLM's torchao path (torchao.py:290-318) delegates to torchao.quantization.quantize_ and never calls it. So vLLM witnesses ModelOpt, not torchao's writer. Section 1.1 now tabulates what each witness proves and how it was verified. The default is unchanged; nothing moves either way. Gates (Release, CUDA=OFF, BUILD_EXIT=0, no ENOSPC/BFD in any build log, df 84% throughout). ctest -N 409, full ctest 409/409 passed, 0 failed, 1 skipped (test_voxtral_e2e). The ten NVFP4 gates, nine unchanged and one moved: test_nvfp4_dequant 5 / 69 test_ops_nvfp4_matmul 4 / 1 test_gguf_nvfp4 14 / 2352 test_ops_moe_grouped 6 / 3 test_qwen36_weights 7 / 45 test_ops_nvfp4_fp4 22 / 919 test_qwen3_forward 7 / 1557 test_ltx2_device 13 / 498 test_minimax_h3 79 / 57395 test_ltx2_loader 24 / 4809 test_ltx2_loader 4793 -> 4809 is the +16 from the re-banding and its two built-in mutation arms; it is the suite this change adds assertions to. Tree restored byte-for-byte after every mutation. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…icts (#560) This PR exists to stop constants carrying a wrong reachability verdict, and it shipped three of its own. The deliverable is an accurate record, so an inaccurate record here IS the defect. Every claim below was re-measured on this tree before the text it falsifies was touched; nothing about the norm_eps arm or the upstream revision pin changes. TWO CONSTANTS LISTED AS INVISIBLE ARE NUMERICALLY GATED. The pin case opened with "Each of these was mutated with every golden staying green", which is false for both encoder entries. Mutating the FIELD DEFAULTS, which is what every arm runs because no arm overrides them: Ltx2ConvVideoEncoderConfig::norm_eps 1e-6 -> 1e-4 RED 4.38839e-05 vs the 5e-6 band, on 2 goldens ("the video ENCODER (*_res family)", "the video encoder CROPS a frame count that is not 1 + k*factor") Ltx2ConvVideoEncoderConfig::pixel_norm_eps 1e-8 -> 1e-6 RED on 4 goldens: 1.02744e-05, 1.02744e-05, 8.10623e-06, and 0.000175595 on "(strided convs, per_channel, reflect)" The cause is the one this PR already established for the decoder, reaching the other half through the SAME LINE. The encoder shares ResnetBlock3d -- ltx2_video_vae.cpp:1051,1056 call what the decoder calls at :693,700 -- so it reads norm3 at :405 for exactly the reason F-1 gave. Forcing :405 to 1.0 reds those two encoder goldens at 0.150858, which is what IDENTIFIES norm3 as the reader rather than inferring it: norm_layer is kPixelNorm on both encoder arms, so neither ApplyNorm nor conv_norm_out (:1081-1087) enters a GroupNorm branch. Arm B has no res_x_y and so no norm3, and stays green -- the coverage is real but partial, which is why the pins stay. Coverage was BETTER than recorded, not worse, and that is still a defect: a list whose membership claim is wrong in the safe direction is a list nobody can trust in the unsafe one. The class statement is now per-entry, quantified, and says the mutation was run, because this case has carried a wrong verdict twice. THE NEW ARM FALSIFIED A LINE THIS PR LEFT STANDING. ltx2_video_vae.h recorded `Ltx2ConvVideoDecoderConfig::pixel_norm_eps 1e-8 -> 1e-6 green` under "EVERY golden staying green". With section 5d present that mutation REDS at 1.69305e-04: the low-scale latent built to make norm_eps a first-order term made its neighbour one too. The fixture that closed one hole closed another, and the line claiming otherwise survived the change that refuted it. THE ENCODER HALF GETS THE SAME NOTE AS THE DECODER. ltx2_video_vae.h gained a full "norm3 is the reason this is LIVE on a PixelNorm checkpoint too" note; ltx2_video_vae_encoder.h still said only "a field here only so the gate can pin it". True, and incomplete for the identical code path. One line cannot be live for one caller and dead for the other. THE QUALIFIER F-8 ADDED IS NOW GATED. The refusal test checked only VAE_ENCODER_COMFY_KEYS_FILTER and default_image_crf, so deleting the `crf == 0` qualifier again would not go red. Upstream `preprocess` returns the image untouched at crf == 0 (media_io/decode.py:427, in :413-435), so "re-compresses before encoding" is only true of a nonzero resolved CRF; naming the round trip without its exception overstates what is unported. Proven RED by removing the qualifier from the message: 16 passed / 1 failed, the one being the new CHECK. ONE BAND, ONE DEFINITION. gen-ltx2-vae-goldens.py hardcoded `10 * 5e-6` while the suite applies kLtx2GoldenTol. Two definitions of one number in two languages: a widened C++ band would leave the generator certifying arms against a band nobody uses. It now PARSES kLtx2GoldenTol from the suite and dies unless it finds exactly one definition. And docs/USAGE.md:335 had grown to 119 chars inside a 79-column paragraph; re-wrapped, no wording changed. Gate on this tree, CPU-only Release, clean rebuild: BUILD_EXIT=0, no ENOSPC or BFD line, ctest -N 401, ctest -j4 400/401 with test_engine_core_proc the known -j flake, PASS on a serial re-run. test_ltx2_vae 36 cases / 3039 assertions (comment -only, unchanged from before). test_ltx2_video 17 cases / 173 assertions, up from 172 by the added CHECK. Issue: #560 (campaign #435) FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… can see scale Reviewed PASS at dbf8a52 after one FAIL round: all six findings closed, with the reviewer reproducing every load-bearing number rather than accepting the report. The state change in the docs row is real: the first-party NVFP4 file now LOADS, so the row's "first-party NVFP4 does not" clause is retired. Fifth hand reconciliation of this one keyed row (#595). What the round actually bought: The loader's "refuses by name" guarantee was FALSE for the class that matters -- a marker-less checkpoint with LINEAR [N, K/16] scales, which is what ModelOpt, llm-compressor and compressed-tensors all write, is shape-identical to the cuBLAS-padded framing whenever N%128==0 and G%4==0. It would have unswizzled and read high-first: finite, correctly shaped, correctly scaled, wrong, and it renders. There is no better discriminator -- the file's __metadata__ carries no producer key -- so the DECISION stands and the three false sentences, one of them user-facing, were corrected instead. And the correlation gate could not see a scale error at all. Pearson is scale-invariant by construction: a uniform x1.09 defect moves corr by 7.35e-11, so the corr>=0.99 floor had no power over the defect class it looked most authoritative about, and rel_rms carried the load one-sided at <=0.15, where x1.10 measures 0.150094 and failed by 0.06% -- by luck. Now a measured BAND, 0.085 <= rel <= 0.115 plus corr <= 0.998, verified strictly stronger: at x1.05 the band reds where the old ceiling passed green. Carrying one non-blocking finding forward rather than silently: the "too-good" arm at test_ltx2_loader.cpp:1200-1208 is TRUE BY CONSTRUCTION -- `perfect` is a copy of the oracle, so it prints CHECK( 0 < 0.085 ) whatever the code does. A tautology wearing a gate's label, in the fix for exactly that class. The real guard at :1175 is proven live; the arm is owed removal or relabelling. Issue: #435, #598 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ird stale refusal goes away
The last hop of the LTX-2.5 campaign. Two branches had each built half of it and
neither could see the other:
* L10 made the Gemma-4 tower RUN — the embedded tokenizer reaches a prompt
string, the torchao-NVFP4 tower materializes onto `Gemma4Weights`, all 49
hidden states come out within the oracle's own bf16 noise floor, and both
caption projections turn them into the 4096/2048 conditioning streams.
* L9c put the `Embeddings1DConnector` on the render path with the checkpoint's
OWN weights.
L10's `encoder_path` refusal said the connector weights were "still among the
modules `Ltx2LoadDitFromSafetensors` refuses (ltx2_loader.h:96-99)". On the
merged tree that was FALSE — `ltx2_loader.cpp:417` already recorded them as
loaded elsewhere by L9c. That is the THIRD refusal in this campaign whose stated
reason went stale, so the header comment it cited is corrected in the same
change rather than left to mislead a fourth reader.
WHAT NOW HAPPENS. `encoder_path` loads the tower, the tokenizer and both caption
projections; `Ltx2SelectTextFeatureVariant` resolves the V1/V2 shape from the
DiT's own transformer config; and a request's `prompt` is tokenized, encoded,
projected, run through the connector and handed to cross-attention, per request.
`has_encoder()` is true. `ltx2-gen` grows `--encoder`, `--encoder-config` and
`--prompt`, so the capability is reachable through the ABI rather than only from
a test.
ONE GENUINE GAP, refused rather than defaulted: the only shipped LTX-2.5 text
encoder carries NO `__metadata__` at all, so the Gemma config is an INPUT. It
comes from the checkpoint's `__metadata__["gemma_config"]` when present and from
the new `encoder_config_path` extra when not; neither source is a refusal, both
sources is a refusal. `layer_types`, `global_head_dim`,
`num_global_key_value_heads` and `attention_k_eq_v` each resolve a DIFFERENT
tower out of a byte-identical tensor set, which is the same polarity
`dit_config_path` already has.
TWO THINGS RECORDED AT THE CODE rather than left for a reader to rediscover.
`Ltx2TextEncoderConditioning` and `Ltx2ConnectorCreateEmbeddings` are two ports
of overlapping halves of `embeddings_processor.py:70-117` and BOTH carry the
right-pad sort, so composing them sorts an already-sorted stream; that is the
identity because a stable argsort of a 0/1 key is idempotent, and the engine now
ASSERTS the precondition instead of relying on the claim. And the tower runs on a
CPU queue even on the device arm, because everything in `ltx2_text_encoder.h` is
f32 by declaration — stated as owed, not hidden.
THE INSTRUMENT, and why it can see the difference. `Ltx2VideoEngine::last_conditioning()`
reports an FNV-1a digest and an absmax over the exact f32 buffers
`Ltx2ModalityInput::context` pointed at, after the connector and immediately
before the denoise loop. A frame statistic was deliberately NOT used: L9c's
reviewer found a scene and a colour field INDISTINGUISHABLE to the existing frame
analyzer (neighbour |dx|/sd 0.093 vs 0.033) and needed contact sheets. A digest
is a function of the bytes rather than a summary of them, so it has no such blind
spot; the absmax is what separates "the tower ran" from "the tower returned
zeros", which would otherwise satisfy a difference check for the wrong reason.
GATE. tests/vllm/multimodal/test_ltx2_video.cpp 24 -> 29 cases, 365 -> 485
assertions, on a reduced-dimension text encoder written in the SHIPPED format:
bf16 tower with the mixed sliding/full geometry and no `v_proj` on the full
layer, torchao-NVFP4 caption projections with a PADDED swizzled scale, the
tokenizer and HF sidecars stored as tensors, and no `__metadata__`. The fixture's
DiT config also gains the three V2 marker keys it was missing — it carried only
`caption_proj_before_connector`, which is the PARTIAL set upstream refuses, so it
gated no variant selection at all. The three added values are read from the
first-party NVFP4 DiT's own metadata.
RED before GREEN: against the pre-L13 engine the new cases do not compile —
`kLtx2EncoderConfigPathExtra`, `Ltx2ConditioningTrace` and `last_conditioning`
do not exist — which is the honest statement that the surface could not be
expressed. Mutation evidence, each applied alone and the tree restored
byte-identically (sha256 verified):
M1 tower encodes a CONSTANT instead of `gen.prompt`
-> 3 RED: both digests equal across two prompts, and the frames equal
M2 prompt path skips `RunConnector`
-> 3 RED in "goes through the CONNECTOR": same prompt, same tower, same
DiT weights, only the connector differs, and it stopped mattering
M3 drop the missing-gemma-config refusal
-> 2 RED. Worth recording: an EARLIER version of that subcase asserted
only that the message named `encoder_config_path`, and M3 left it GREEN
— the load still failed, but with "cannot open " from a fallthrough
reading the empty path. A refusal firing for the wrong reason is
exactly what this phase exists to stop shipping, so the assertion was
strengthened to the discriminating clauses before being called a gate.
NOT gated, and named rather than claimed unreachable: the additive-mask
right-pad guard. I did not construct a probe that reaches it, and per this
campaign's own rule a mutation that moves nothing is not evidence of
unreachability.
Full gate on this worktree, CPU Release build, x86_64: BUILD_EXIT=0, no
"No space left"/"BFD assertion" in the build log, 61G free on /. ctest -N 408,
`ctest -j8` 408/408 passed 0 failed (1 skipped: test_voxtral_e2e). Focused:
test_ltx2_video 29/485, test_ltx2_loader 20/2371, test_ltx2_text_encoder
23/3583, test_ltx2_pipeline 37/2382, test_ltx2 29/1615, test_video_engine
11/254, test_capi 55/505 — all SUCCESS.
`scripts/agent-preflight.sh --staged` passes `doc-checkpoint --staged` and
`now-current --staged`. Its `doc-checkpoint range` failure is INHERITED: all
three commits it names (b0aa475, d67f812, aa6aa0e) are ancestors of the
base SHA 67a7b1c and the same failure reproduces on the untouched base.
Issue #435.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
… list (#560) Round 3 closed every repair from round 2 and failed on one new instance of the same defect: a record calling a constant unreachable when a golden reds on it. One entry over, the list still said green. THE BWE MEL LOG CLAMP IS NOT INVISIBLE. ltx2_video_vae.h:108-110 carried kLtx2BweMelLogClamp 1e-5 -> 1e-8 green under "these three left EVERY golden green". Reproduced on this tree before touching the text, by mutating ltx2_audio_vae.h:212: test_ltx2_vae.cpp:1400: CHECK( err <= kLtx2GoldenTol ) is NOT correct! values: CHECK( 0.144965 <= 5e-06 ) logged: saturated-clamp BWE max|diff| = 0.144965 36 cases: 34 passed, 2 failed | 3039 assertions: 3037 passed, 2 failed exit code 1 The second failure is the constant assertion at :1279. Restored byte-for-byte afterwards, md5 fe861901ef8f18fbbc9e6caedf513d1b before and after. The saturating arm that reds it, "ltx2 vae: the BWE mel log clamp is gated where it actually binds", landed at 93329b1 -- and `git log -S "left EVERY golden green"` returns only d45bcb5, so the phrase was written over an arm that already existed. Two places in the tree already said so: this same header 26 lines later, and test_ltx2_vae.cpp:1274-1278 ("until the saturating arm below"). The list is the surface a reader trusts, so the list is what was wrong. The entry is moved out with the number that proves it, exactly as round 2 moved pixel_norm_eps. What made it look invisible is SCALE, not the constant's nature: the ordinary arm's raw mel minimum is ~4.4e-3 and never approaches the floor, so the reachable arm attenuates mel_basis by 1e-4 until every bin lands under it and asserts the saturated-bin count rather than assuming it. THE SAME FALSE CLAIM SAT AT THE CONSTANT'S OWN DEFINITION. ltx2_audio_vae.h:206 declared it "the member of the invisible-constant class" and stated the 1e-5 -> 1e-8 mutation "leaves every tensor golden green". Both sentences shipped in 93329b1, the very commit that added the golden refuting them. Fixing the list while leaving the declaration would have left round 4 the identical finding one file over, so it is corrected here. Its upstream citation was also off by one -- the clamp is vocoder.py:515, not :516; :516 is the `return`. Verified at the pinned checkout, which is what test_ltx2_vae.cpp:1273 already cited correctly. THE CRF CITATION POINTED AT THE WRONG EARLY RETURN. test_ltx2_video.cpp:668 read "media_io/decode.py:413-435, the early return at :427". At LTX-2 fd4ded7f, `if crf == 0:` is :425 and `return image` is :426; :427 is `if min(image.shape[0], image.shape[1]) < 2:`, a different early return for degenerate image size. The range and the behavioural claim were both right, so only the pinpoint moves -- but this is the worst place to be off by two, because a reader following it lands on the size guard and concludes the crf == 0 claim is unsupported. Now cited as :425-426 and naming :427-428 as the guard it is not. That size guard is also a second case where upstream does NOT re-compress, and docs/USAGE.md:334 claimed the round trip happens "whenever the resolved CRF is not 0" with no other exception. Qualified there too. A SHARED LINE IS NOT AN ARGUMENT FOR LIVENESS. ltx2_video_vae_encoder.h:130-132 argued the encoder's norm_eps is live because "It is ONE line in the port for both halves ... so it cannot be live for one and dead for the other". The conclusion holds -- both halves were measured live -- but the inference does not, and round 3's :405 -> 1.0 probe is the counterexample: :405 sits behind the `input.channels != out_channels` guard at :400, so even entering ResnetBlock3d is not reaching it. `res_x` passes `x.channels` as `out_channels` at :1051 and the guard is false; encoder arm B never enters ResnetBlock3d at all, since all four of its blocks are plain strided CausalConv3d (:1060-1068). Rewritten to rest on the per-arm measurement, and to say what test_ltx2_vae.cpp:1327 already says -- the coverage is real but PARTIAL, which is what the pin is still for. Comment-only in the headers plus one doc qualifier; no constant, golden, band or executable line changes, and the assertion counts are unchanged by construction. Gate on this tree, CPU-only Release: BUILD_EXIT=0, no ENOSPC or BFD line, ctest -N 401, ctest -j8 401/401 CTEST_EXIT=0. test_ltx2_vae 36 cases / 3039 assertions exit 0; test_ltx2_video 17 cases / 173 assertions exit 0 -- both identical to the pre-edit baseline measured on the same build dir. `doc-checkpoint range` still fails on b0aa475 and d67f812. Both are pre-existing L4 and L7+L8 branch commits, both flagged identically before any edit in this change, and neither is touched here; it is a campaign-integration item, not this repair. Issue: #560 (campaign #435) FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…survive measurement (#435) Eight findings from the L10 fresh review. Six close as written; two close differently, because the thing they prescribed does not work and the oracle says so. Both are recorded here rather than quietly worked around, since an implementer who accepts a prescription that fails is the failure this protocol exists to catch. ──────────────────────────────────────────────────────────────────────────────── F3 — THE PRESCRIBED FIX IS REFUTED; THE ROPE TABLE IS THE INSTRUMENT The finding is real: `g.rope_partial_full = 1.0` (full rotary on the full- attention layers instead of the config's 0.25) left the suite green at 23 cases / 3583 assertions. Reproduced here before touching anything. The prescription was "a fixture whose global_head_dim/seq keep the rope contribution above the floor". MEASURED against the oracle, per state, as signal = max|bf16 @0.25 - bf16 @1.00| over floor = max|f32 - bf16|: committed head_dim 8/16 seq 8 worst signal/floor 1.095 wider full head head_dim 16/32 seq 8 0.258 longer seq head_dim 8/16 seq 24 0.764 wider + longer head_dim 16/32 seq 24 0.523 wider + longer head_dim 16/32 seq 32 0.650 Enlarging the fixture makes it WORSE. bf16 accumulation noise grows at least as fast as the rope contribution, so no reachable fixture size separates them: the hidden states are the wrong instrument, not a badly sized one. The right instrument is the table. `BuildProportionalRopeCache`'s host computation is split out and exported as `Gemma4ProportionalRopeCosSin`, the generator emits the oracle's own `Gemma4UnifiedTextRotaryEmbedding` cos|sin for the full-attention layer type (section 6), and the new case compares them in f32 with nothing accumulating. Measured agreement 1.267e-07 against a 1e-6 bound, plus an EXACT structural check that pairs at and beyond `int(partial*head_dim//2)` are cos=1, sin=0 and that the pairs below it really rotate. RED under the mutation: 202 assertions fail, worst 0.56533 vs 1e-6, exit 1 GREEN restored: 26 cases / 4115 assertions, exit 0 ──────────────────────────────────────────────────────────────────────────────── F2 — THE VALUE GATE AND THE LOADER, IN CI; AND M6 IS NOT WHAT IT LOOKED LIKE `Ltx2EncodePromptToConditioning`, `Ltx2LoadGemmaTowerFromSafetensors` and all four documented refusals were reachable from exactly ONE place — the opt-in 24 GB case — so none had CI coverage. They do now. The reduced tower fixture, which is already held to a running oracle, is written out as a real .safetensors under the CHECKPOINT's tensor names, loaded back, and compared to the fixture byte for byte; then a tokenizer whose added tokens spell the oracle's own token ids drives the whole prompt path and its conditioning is held to the committed LEFT-PADDED oracle run at a floor propagated through the identical projection. concat order q,k,v -> q,v,k RED 14 assertions, exit 1 PLE refusal disabled RED 1 assertion, exit 1 missing-v_proj refusal disabled RED 1 assertion, exit 1 module dtype F16 read as bf16 RED 1 assertion, exit 1 norm vector F16 read as bf16 RED 1 assertion, exit 1 conditioning at 0.565x / 0.688x of its propagated floor Now the part that did not survive. The review called `positions[i] = i` "the exact defect the surrounding comment warns about". It is not a defect at all. Rotary embedding is RELATIVE and the pads are masked out of attention, so shifting every position by the pad count cancels. MEASURED in the oracle, same 8 tokens told 12..19 and told 0..7: f32 max|diff| 5.11e-05 over max|value| 14.35 3.6e-06 relative bf16 max|diff| 1.09 0.65-1.70x the dtype floor f32 round-off, and at bf16 the rounding of different absolute angles. So the comment at ltx2_text_encoder.cpp — "a port that renumbers from zero rotates every query by the wrong angle" — was an overclaim, and so was its twin in the left-pad case. Both now carry the measurement. The numbering is still mirrored, because transformers derives positions from `cache_position` and fidelity is the reason; arithmetic is not. The new case does red under the renumbering, at 1.10x the audio floor, and says in its own comment that the narrow margin is a property of the defect rather than of the gate. ──────────────────────────────────────────────────────────────────────────────── F4 — THE GENERATOR'S LEGS WERE COUPLED, AND NOT WHERE IT LOOKED `run_tower` did `inner.to(dtype)`, which converts IN PLACE, so every leg after the bf16 one ran downstream of it. Each leg now runs on a `copy.deepcopy`. The mechanism is narrower than "bf16-rounded weights": MEASURED on this transformers build, the parameters round-trip unchanged, and it is the rotary embedding's non-persistent `inv_freq` BUFFERS that do not — 9.36e-05 on the full-attention table, 9.77e-05 on the sliding one. Regenerating moved 12 of the 13 padded f32 states (state 12 by 1.03e-02 on values of 2.705) and every entry of the pad-equivalence vector, while sections 2, 3 and 3b are byte-identical — which is exactly the partition the in-place conversion predicts. The header's claims about sections 4 and 5 are corrected to match. ──────────────────────────────────────────────────────────────────────────────── F5, F7, F8, F9 F5 337 -> 329, and pinned exactly rather than `> 300`. The count is arithmetic and now says so: 40*7 + 8*6 + 1. A lower bound is satisfied by a loader that quietly took a third of the layers from a bf16 fallback. F7 The rationale was inverted. Upstream calls `self.tokenizer(text, ...)` — `__call__` with its default `add_special_tokens=True` — so upstream DOES run the post-processor and we do not. Identical on this checkpoint because its `special_tokens` map is empty, measured. "THE TWO REFERENCES DISAGREE" also overstated it: both let the post-processor run, and only the explicit BOS prepend differs. Both corrected. F8 `kv.dtype` on the production prompt path was f32 where upstream resolves bf16 and where the attention itself runs bf16. Narrowed, not annotated. It was never wrong — it was WIDER, which is why nothing could see it: with the cache back at f32 the new conditioning gate reproduces 0.0617859 / 0.0394362 byte for byte. Half the cache bytes, and it stops taking `Gemma4AttnBlock`'s two-cast-buffers-plus-CastF32 arm on every layer. F9 The dropped `docs/FEATURES.md` edit is reapplied onto the campaign's rewritten row, truthfully: "Text tower RUNS" alongside the `encoder_path` refusal, which another change owns and this one does not touch. Row 533 chars, cells 60/28/217/215. `docs/USAGE.md` is owed too, and the debt turned out to be real rather than procedural: line 372 still said "the Gemma-4 12B text tower is not ported", which is false and which the paragraph 85 lines below it already contradicted. One document, two answers about one model. Corrected to state that the tower runs and to defer to that paragraph for what still blocks a prompt. The `encoder_path` passage itself is untouched — it belongs to the change that owns F1. ──────────────────────────────────────────────────────────────────────────────── NOT CLOSED HERE, AND OWED F1 and F6 are out of scope by instruction — another implementer owns whether the `encoder_path` refusal can be lifted now that L9c landed the connector weights. The spec is operator-owned, so this reports rather than edits it. Two things it needs. First, §4.2's open question stands and should not be narrowed by anything above: the shipped NVFP4 tower's output is still never compared to an independent oracle, so "all 49 hidden states within the oracle's bf16 noise floor" is proven for the SYNTHETIC reduced tower with bf16 weights and not for the quantized one that ships. PR #571 built the independent-oracle correlation technique that would close it. Second, the position-numbering claim appears in the spec in the same overclaiming form corrected here. ──────────────────────────────────────────────────────────────────────────────── GATE cmake -S . -B bld -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_BUILD_EXAMPLES=OFF cmake --build bld -j 12 BUILD_EXIT=0, no ENOSPC/BFD, df 88% ctest -N 409 tests ctest -j 8 --output-on-failure 409/409 passed, exit 0 ./bld/tests/test_ltx2_text_encoder 26 cases / 4115 assertions, exit 0 (baseline was 23 / 3583) `scripts/agent-preflight.sh` fails `doc-checkpoint range` on commits b0aa475, d67f812 and aa6aa0e. All three are ancestors of the branch head this work started from and none is touched here. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…that was only a witness Review findings S1, S2, S4-S7 on `43aa5837`. The reviewer's verdict on the code was that it is the strongest of this campaign and that the docs cell over-reads it; nothing here redesigns anything. S1 (BLOCKING) — A PUBLIC DOC ASSERTED A RENDER NO RUN SUPPORTS. `docs/FEATURES.md` read "e2e at 320x192/25f from a TYPED PROMPT via Gemma-4: coherent scene, valid MP4+WAV". Both halves were attached to a run that did not happen. The 320x192/25f arm is L9c's, and this record's own L9c section says it plainly: the conditioning was `--prompt-valid-rows 24` over SYNTHETIC N(0, 0.2), 104 of 128 rows were the connector's trained `learnable_registers`, and "It is not a depiction of a prompt." L10's real-checkpoint run produced conditioning only, no frames. L13's own gate is fixture-only, CPU Release. OPTION (b) TAKEN — state what happened, do not manufacture the run. The arithmetic is the reason, not squeamishness: L9c's 320x192/25f arm bottomed out at 68.2 GiB MemAvailable on a 119 GiB box that REBOOTS rather than OOM-kills, the tower is a further ~24 GB of host bf16 (ltx2_video.cpp:846), and at the time of writing dgx.casa's `$HOME/gpu.lock` was held by another session's `ctest` with root at 99% (62 GiB free). Risking the box for one number, with other agents live, is not a trade this finding asks for. The cells now say the render was register-conditioned and that a prompted one is OWED. S2 — `docs/STATUS.md` still said "the Gemma-4 tower is owed, so no prompt encodes yet", which this PR made false. Corrected without growing the page: the new cell is 216 chars, under the 220 that `oversized_cells` counts, so the ratchet is untouched. S4 — THE COMPOSITION'S VALUE ORACLE IS OWED, and is now recorded where a reader meets the instrument. `last_conditioning()` is a WITNESS, not a gate: the reviewer's R6 (video conditioning x1.5 after the connector) and R7 (conditioning rows REVERSED) each passed all 485 assertions with exit 0. A digest detects CHANGE; nothing at this level pins VALUES. The per-brick oracles are real and untouched; the two JOINS - `Ltx2ConnectorCreateEmbeddings` and the `Generate` composition - have none, and both mutations live in exactly that gap. Recorded rather than closed, with the closure specified and its one prerequisite VERIFIED so the next implementer does not have to: upstream's counterpart is `EmbeddingsProcessor.process_hidden_states` (embeddings_processor.py:97-117), in the same package `gen-ltx2-pipeline-goldens.py` already executes at section 10, and that generator reproduces its committed output BYTE-FOR-BYTE on the pinned upstream (md5 53e2a6aba8885d7d58302ad0b7b09eb4 both sides, LTX-2 clean at fd4ded7f, vllm-omni at a4ea67a2), so a section can be added without disturbing anything already gated. S5 — A DATA RACE ON THE API THE HEADER ADVERTISES, fixed rather than documented. `last_conditioning()` read `impl_->trace` without the mutex `Generate` holds. It now returns BY VALUE under that mutex; a reference could not be made safe, since the lock is released before the caller reads. The second half is real too: the trace is filled before the denoise loop, so a `Generate` that throws later left a healthy-looking trace for a render that never completed. New `completed` flag, set at the single successful return. Gated in BOTH directions rather than asserted, on a REAL refusal - keyframe / reference conditioning is refused after the trace is written, so a prompted request carrying a reference image fills the trace and then fails: * drop `completed = true` -> 3 cases RED, exit 1 * set it at trace-fill time (pre-fix) -> the new case RED, exit 1 Tree restored byte-identically after each (sha256 22342f84...c546 before and after). S6 — TRUE VALUES, NOW RECORDED. The four V2 markers were correct but nothing in the repo carried the header they came from, so the next reader needed the 18.72 GB checkpoint mounted. The observed `__metadata__["config"]["transformer"]` is now recorded with its path, byte size and revision, at the fixture and in the record. All four present, none drifted; `text_encoder_norm_type: PER_TOKEN_RMS` corroborates independently and the selector deliberately does not read it. And a CAUSE CORRECTED. The fixture said an earlier partial marker set "gated no variant selection at all". `Ltx2SelectTextFeatureVariant` DOES refuse a partial set (ltx2_text_encoder.cpp:184-192). It never fired because no production path CALLED the selector before L13 - the keys and the first caller landed in the same commit. S7 — ONE GUARD PRESENTED AS A POSITIVE IS UNGATED. The right-pad precondition guard gets the same sentence the mask-value guard already had. No reachability is claimed: no probe was built that fails to reach it, and a mutation that moves nothing is not evidence of unreachability. ONE SELF-CORRECTION worth naming, since it is this campaign's own failure mode. I first replaced "+~24 GB text tower" with an on-disk figure, believing the number ungrounded. It is grounded - 12B params at bf16, stated at ltx2_video.cpp:846 and in docs/USAGE.md - and the resident figure is the useful one for a memory cell. Reverted. GATE. CPU Release, clean configure, BUILD_EXIT=0, build logs clean of `No space left` / `BFD assertion`, df 53-67G free throughout. `ctest -N` 414 registered; full `ctest -j4` 414/414 passed, 0 failed, exit 0 (test_voxtral_e2e skipped). test_ltx2_video 29/485 -> 30/499 (the one new case), test_ltx2_loader 20/2371, test_ltx2_text_encoder 23/3583, test_ltx2_pipeline 37/2382, test_capi 55/505 - all unchanged and all SUCCESS with exit 0 read separately from the summary. `scripts/agent-preflight.sh --staged` passes `doc-checkpoint --staged`, `now-current --staged` and `check-public-doc-tables`. Its `doc-checkpoint range` failure is INHERITED and re-verified here: b0aa475, d67f812 and aa6aa0e are each `--is-ancestor` of the base SHA 67a7b1c. Issue #435. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
, #604) An exhaustive sweep of every reachability claim in the tree measured 25 and found six false, at sites no earlier round on this branch enumerated. The pin list in test_ltx2_vae.cpp that rounds 1-4 kept repairing is correct in every entry and is untouched. Nothing here weakens a bound, adds tolerance or deletes a gate: the suites keep their exact case and assertion counts. Every number below was reproduced on this tree before the text it falsifies was edited, one mutation per leg, each leg preceded by a FORCED rebuild of the pristine tree proved green and followed by a restore verified back to the pre-mutation md5. Binary md5s were compared pre vs mutant on every leg and differed on all ten, which is the discriminator for the ninja-mtime leak that would otherwise let one leg's mutation survive into the next. 1. Ltx2ConvVideoDecoderConfig::pixel_norm_eps. test_ltx2_vae.cpp said 1e-8 -> 1e-6 "leaves every golden green". It REDS "the video decoder's norm_eps is gated where it BINDS" at CHECK( 0.000169305 <= 5e-06 ). ltx2_video_vae.h has recorded this correctly since d45bcb5; the test file contradicted its own header, so the test file is what moves. 2. "Unlike the decoder pair". Both halves of that pair are reachable. The audio decoder's pixel_norm_eps 1e-6 -> 1e-4 REDS 5 goldens across three arms -- 0.0120053, 0.00461239, 0.00302449, 0.00245912, 0.0120053 -- and the video half reds per (1). All four PixelNorm epsilons are numerically gated. 3. kLtx2UpsamplerNormEps, claimed invisible at BOTH ltx2_upsampler.h and test_ltx2_pipeline.cpp. At the class's OWN 100x bar, 1e-5 -> 1e-3 REDS all three arms of "the latent spatial upsampler reproduces upstream": PixelShuffle 0.0289409, Rational2 0.0347079, Rational1p5 0.0649014. 4. kLtx2UpsamplerNormGroups, called a member of the same class. 32 -> 16 REDS the same three arms at 0.63738, 0.633718, 0.874346. 5. kLtx2ConnectorRmsNormEps, claimed inert at BOTH ltx2_connector.h and test_ltx2_pipeline.cpp on the reasoning "the fixture's rows are never near-zero". That is not what rms_norm does with it: the epsilon is added to the MEAN SQUARE, so it perturbs every row. 1e-6 -> 1e-4 REDS 5 connector arms -- Split 0.0558581, Interleaved 0.104284, Float64 0.140343, NoRegisters 0.000542641, GatedNoBias 0.0892045. 6. kLtx2BlurKernelSize, called reachable "only through a default upstream never passes explicitly". A default upstream never overrides IS the shipped width, which ltx2_upsampler.h:98 already said -- the two records disagreed and the test file held the wrong one. 5 -> 3 REDS the upsampler's Rational1p5 arm at 0.689782, and only that arm, because BlurDownsample runs on the rational `den` (ltx2_upsampler.cpp:439) and 1.5 -> {3, 2} is the one covered scale with den != 1. 7. kLtx2Res2sSigmaUpClamp -- a REASONING error, not a fixture gap, and the one worth reading. Both ltx2_pipeline.h and test_ltx2_pipeline.cpp argued "eta <= 1 keeps sigma_up <= sigma_next, so the clamp never binds". <= includes ==. `step` forms sigma_up = sigma_next * eta (ltx2_pipeline.cpp:339), so at eta = 1 the two are equal and `min` takes sigma_next * 0.9999 on EVERY step -- the clamp is not a fallback, it is the only thing keeping the residual off zero exactly there. A 1% move, 0.9999 -> 0.99, REDS the Eta1 arm the suite already runs, at 0.086 (index 0) and 0.130563 (index 1); EtaHalf stays green because 0.5 * sigma_next is below the clamp, and Eta1 index 2 stays green because sigma_next == 0 returns the denoised prediction unchanged (:181-182). The corrected note states the boundary, not just the verdict. 8. kLtx2TextNormV1Eps / kLtx2TextNormV2Eps. Two blurbs in ltx2_text_encoder.h ("invisible to any golden built from random values", "Reachable only when...") contradicted the CORRECT detailed note 14 lines above one of them, which already warns of exactly this. Both epsilons are additive on an O(1) denominator against a 1e-5 band. V1 1e-6 -> 1e-4 REDS "`_norm_and_concat_padded_batch`, both padding sides" at 0.000524044 and "FeatureExtractorV1" at 7.53999e-05 / 6.61612e-05. V2 likewise REDS "`norm_and_concat_per_token_rms`" at 0.00232971, carrying into "FeatureExtractorV2" and the hand-off at 0.000344872 / 0.000259042 / 0.00039053. The blurbs now agree with the note. 9. The class prose itself. ltx2_video_vae.h:106-107 said the tensor comparison "accepts any value at all -- including 0.0, and including one 100x off". False even of its own remaining members: kLtx2RmsNorm2dEps at 1e-12 -> 1.0 REDS "the video ENCODER (*_res family)" and "the video encoder CROPS a frame count that is not 1 + k*factor", both at 0.000525832. The per-entry verdict (1e-12 -> 0.0 green) stands and is unchanged. "Never BINDS at the shipped value" and "is not read" are different statements, and only the first was ever true. Per #604 the standard is now explicit in the prose: only a probe that FAILS TO REACH proves unreachable, a mutation that happens not to move anything proves nothing, and every corrected claim therefore names its magnitude and its arm. Magnitude escalation to the class's own 100x bar is what exposed (3), (5), (6) and (9); a mere 1% exposed (7). Every constant stays PINNED. Reachability by a golden and a source-anchored pin are not substitutes: a regeneration that moves a constant and its expected tensors together passes every value comparison, and only the pin compares against upstream's own signature. The corrected text says that instead of claiming the pin is the only gate. docs/USAGE.md gains the operator-facing half of the same finding, in the pipeline golden regeneration recipe: which constants a regenerated ltx2_pipeline_goldens.inc now carries, and why the pin cases are still not redundant. GATE, CPU-only Release, VLLM_CPP_CUDA=OFF, 20 cores: BUILD_EXIT=0, no "No space left" or "BFD assertion" in any build log, df 87%. ctest -N 399 tests ctest -j 8 398/399 passed; test_engine_core_proc failed and PASSES serially (known -j flake), so the gate is green. test_ltx2_vae 36 cases / 3039 assertions, exit 0 test_ltx2_pipeline 35 cases / 2358 assertions, exit 0 test_ltx2_text_encoder 17 cases / 3350 assertions, exit 0 Counts are byte-identical to the pre-edit baseline, which is the correct result for a comments-and-records change: nothing was added, removed or relaxed. The pre-existing `doc-checkpoint range` failure on b0aa475 and d67f812 is NOT repaired here. It reproduces identically at the untouched head ef83d94 (`check-doc-checkpoint.py --base origin/main --head ef83d94`), predates this work by 56 commits, and belongs to the L4 / L7+L8 phases rather than to this scoped repair. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
S8, the one blocking finding of the re-review. `docs/USAGE.md` still described the pre-L13 world in two places, and this PR is what falsified them. `:487` claimed "the Gemma-4 text tower is still not ported"; `:499-510` restated that a prompt is refused because the connector weights sit among the modules the DiT loader refuses, so "`encoder_path` is still refused" and "`has_encoder()` is still false". At this head `im.has_encoder = true` on the `encoder_path` branch (`ltx2_video.cpp:893`), and the connector families load under their own contract (`ltx2_loader.cpp:416-427`). The file also contradicted itself: `:372`, added by this PR, opens "A typed prompt works." Provenance, since a later reader should be able to re-check the cause rather than trust the repair: `:487` arrived at `e48c86253` (L9c) and `:499` at `ab6671394` (L10), BOTH TRUE when written; `43aa58377` in this PR made them false and left them standing. The "renders a scene, not YOUR scene" material is RE-ATTRIBUTED, not deleted. It is still true of the L9c render and of the embeds path, so it now says which run it describes: `--prompt-embeds` with `--prompt-valid-rows 24` over synthetic N(0, 0.2) rows, no tower on the path, 104 of 128 connector rows the connector's own trained `learnable_registers`. What it must stop being is a property of the `--encoder` + `--prompt` command L13 rewrote the example into, because that is the S1 defect in the other direction: NO prompted render has been run in either direction, so the page now claims nothing about what that command renders -- not that it puts a fox on the screen, and not that it fails to. Two accuracy repairs in the same sweep: - The `Ltx2ConditioningTrace` header said both composition mutations "passed all 485 assertions ... on this exact head". 485 was `43aa58377`'s count; this head is 499, so a comment whose whole purpose is precision named a number no run of it could produce. RE-MEASURED here rather than transcribed, because a reviewer report is an input and not a gate result: video conditioning scaled x1.5 after the connector, and the conditioning rows reversed, each applied alone, each recompiled and relinked (verified in the build log), the tree restored byte-for-byte and re-verified green between legs. BOTH still pass, at 30 cases / 499 assertions, exit 0. The verdict did not move; the COUNT had drifted. - `last_conditioning()` was documented as safe to call from a server thread while another renders. True as to safety, but `Generate` holds that same mutex for the whole render, so the caller blocks for minutes rather than getting a stale-but-immediate answer. Stated. One sibling found by sweeping the whole file rather than fixing the instance in front of me, which is what #604 says has failed four rounds running. The AUTO duration comment said the head "needs the encoded prompt this engine cannot produce" -- a fifth refusal-reason that went stale while its refusal stayed correct, since L13 produces exactly that. The real reason is now recorded: no duration head is ever constructed here, and `duration_head_path` is accepted in `kKnownLoadExtras` while no code reads it, so the extra is inert. Named in `docs/USAGE.md` too, where a reader would otherwise supply it and silently get the recipe default. NOT PAID, and named rather than left to be found: `.agents/benchmark-record.md`'s L13 section carries the same superseded 485/485 pair. Correcting it is not free -- `check-doc-checkpoint` classes any benchmark-record edit as a measurement and demands `docs/STATUS.md` + `docs/BENCHMARKS.md`, and those are the surfaces the re-review passed and told this repair not to disturb, with STATUS at its `oversized_cells` ratchet. Manufacturing an edit to a blessed surface to turn a gate green is the thing this protocol forbids, so it is reported instead. Gate (CPU Release, no CUDA): BUILD_EXIT=0, 0 warnings, no `No space left` / `BFD assertion`, disk 87%. `ctest -N` 414 registered; full `ctest -j8` 413 pass / 1 skip, the single failure `test_engine_core_proc` a known `-j` flake that passes serially. `test_ltx2_video` 30 cases / 499 assertions, exit 0. `agent-preflight.sh --staged` doc-checkpoint and now-current both ok; the `doc-checkpoint range` failure is pre-existing on this branch (`b0aa475a3`, `d67f8125e`, `aa6aa0ecd`) and untouched here. Refs #435, #604. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…, #604) A precision nit in the note 4f1ec6e added, in the one place it matters: the deliverable of this branch is an accurate record, so a loose sentence in it is the same defect one size down. The blur-width entry read "1.5 -> {3, 2} is the one supported scale of the three with den != 1". Read as written that says the SUPPORTED-SCALE MAP has one entry with a non-unit denominator, which is false -- 0.75 -> {3, 4} (ltx2_upsampler.cpp:296) has one too and would reach kLtx2BlurKernelSize just as well. What is actually true is narrower: of the three arms the suite runs (PixelShuffle at 2.0 non-rational, Rational2 at 2.0 -> den 1, Rational1p5 at 1.5 -> den 2, per ltx2_pipeline_goldens.inc:1335, :1568, :1801), only Rational1p5 has den != 1, so only it reaches the constant. That is ARM COVERAGE, and 0.75 is an uncovered scale rather than a nonexistent one. Stating it the loose way would have left a reader believing the map cannot grow another reachable arm, which is exactly the kind of inference #604 asks these notes to stop making. The corrected text names 0.75 explicitly so the gap is visible rather than argued away. Comment text only, in one test file. GATE, CPU-only Release, VLLM_CPP_CUDA=OFF: BUILD_EXIT=0, no "No space left" or "BFD assertion", df 85%. ctest -N 401 tests ctest -j 8 100% tests passed, 0 failed out of 401 CTEST_EXIT=0 test_ltx2_pipeline 35 cases / 2358 assertions, exit 0 Counts unchanged from 4f1ec6e and from the pre-edit baseline. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…easured Reviewed across four rounds plus one exhaustive sweep. The sweep is what finally worked: rounds 1-4 each repaired the pin list in test_ltx2_vae.cpp and stopped, so the same defect class survived at five sites nobody had enumerated. Enumerating all 25 claims and mutating each found six false in one pass. All six are now corrected, each carrying the magnitude and arm it was measured at. The best of them is a reasoning error rather than a fixture gap. kLtx2Res2sSigmaUpClamp was documented as "never binds on a well-formed schedule, so no value comparison can see it". Ltx2Res2sStep forms sigma_up = sigma_next * eta, so at eta = 1 the min takes sigma_next * 0.9999 on EVERY step -- the clamp is the only thing keeping residual = sqrt(next^2 - up^2) off exactly zero, and a 1% change reds two goldens on an arm the suite already ran. Docs conflict resolved to the eps side, which is a strict superset: it adds the CRF-and-2-pixel qualifiers its own upstream verification licensed, and the sentence distinguishing "the VAE encoder landed in L11" from "the engine can reach it". I verified that second claim against the tree rather than taking it, because a sentence of exactly that shape has gone stale five times in this campaign; the refusal at ltx2_video.cpp:836 makes the same distinction, so the doc matches the code. Issue: #435, #560, #604 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ck came from `vllm::Pool()` is a process-wide free list keyed by BYTE SIZE CLASS ONLY. The device is not in the key, so a block allocated through one backend is handed to a `DBuf` running on another. One fault, two symptoms, selected by direction: a `cudaMalloc` block reaching a CPU-backend forward SIGSEGVs host-side (and `compute-sanitizer` is clean, because the fault is not on the device), while a host block reaching a CUDA forward returns a UNIFORM `0x7fff0000` quiet NaN -- computed and propagated, not garbage read. Three arms already separate the cause from its neighbours: `VT_POOL_BYPASS=1` (free list removed) is 13/13 green, a per-case `DevicePool` is 13/13 green, and `VT_POOL_EXACT=1` (reuse kept, size-class rounding removed) is STILL RED. So it is cross-device reuse, not over-allocation, and not the pool's existence. Spec only; no implementation in this commit, which is the point of committing it first. It carries scope, the upstream anchors read at the pin (vLLM's allocation handle carries the device as field 0; torch's cache is per-device by construction), the design, what was rejected and why, the RED-first tests, the gates and the baselines that must not move, the risks and the stop conditions. Refs #516 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…rough DBuf The gate for #516, and it is RED at this commit deliberately: the fix lands in the next one, so git itself records that the test was seen failing rather than written to fit a change that had already been made. test cases: 4 | 2 passed | 2 failed | 0 skipped assertions: 15 | 10 passed | 5 failed Status: FAILURE! exit=1 The decisive assertions are not the pointer comparison but the ownership ones: `b.Owns(on_b)` is FALSE and `a.Owns(on_b)` is TRUE -- device 1 did not merely receive an equal pointer, it received a block that device 0's allocator made. It allocates through `dense_attn::DBuf`, the seam every production forward draws scratch from, so it holds the path the LTX-2.5 device suite crashes on rather than a paraphrase of it. No GPU, no checkpoint, no NAS, milliseconds: two fake backends stand in for two devices, the technique test_backend_multidevice and test_reference_tier already use. Two of the four cases pass now and must KEEP passing. Reuse on one device returns the identical block, and two byte sizes in one class still share a block. Without them a "fix" would be indistinguishable from VT_POOL_BYPASS=1, which also separates the devices -- by reinstating the per-op cudaMalloc/cudaFree sync storm the pool exists to remove. Every case uses its OWN size class, so no case can be decided by what another left in a free list, including under --order-by=rand. Two test-side properties are load-bearing: a fake backend never returns a block to the C allocator (these cases compare pointer identity ACROSS a free), and no fake backend is destroyed before exit (the pool is keyed on backend identity, and a reused address would let one case's pool answer another case's question). Refs #516 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…l "the pool"
Turns the RED-first gate green: 8 cases / 26 assertions SUCCESS, from 2 of 4
cases failing at f4be8a4e2.
The device is now STRUCTURAL, not a field a caller has to remember. A
`DevicePool` is bound to one backend at construction, `Pool(b)` resolves the pool
for a device, and the no-argument `Pool()`/`AuxPool()` are GONE -- "the pool"
without a device was the defect, so it is no longer expressible. `vt::Backend*`
is the device identity, since the registry hands out exactly one Backend* per
Device{type,index}; that puts the device in the key with NO new virtual on
vt::Backend, so not one backend implementation is touched.
Lookup sits on the hottest allocation path in the tree -- a DBuf resolves its
pool on every construction -- so a thread-local last-(backend,pool) memo makes
the steady state a single pointer compare. No hash, no lock, on the hit path.
Every pool operation VERIFIES its backend and throws. Deliberately not an
`assert`: the gate builds are Release/NDEBUG, where an assert compiles out and
the silent cross-device hand-off returns. The only way to reach the throw is an
`ActivePoolScope` aimed at another device's pool, which is exactly the mistake
this row makes impossible to make quietly.
`DBuf::ReleaseShared()` replaces 31 copy-pasted shared_ptr deleters that closed
over a byte count ALONE. Those named neither the device nor the pool, so they
returned another device's block -- and, separately, an AUX-STREAM block -- to the
main pool. That second bug was live on every path that used the idiom. Each site
goes from three lines to one, and the backend-less `Put` overload is removed with
its last caller.
Two more instances of the same ambient-device assumption, found while fixing it
and repaired here rather than left to be rediscovered: the decode-graph
`PersistentDecodeInputPool` was a process-wide static, and `ResolveDevicePool
Policy` memoized whichever device asked FIRST and applied its residency cap to
every later one. Both are now per device. Byte-neutral today (every platform's
`device_pool_cap_bytes` is 0), which is why it is safe to do here.
The two per-caller workarounds for this bug are REMOVED, not kept: the
`ActivePoolScope` around the LTX-2.5 bf16 CPU arm (the only test in the tree that
reaches the SILENT direction) and the per-arm pools in the DeepSeek-V2 CUDA-vs-CPU
case. Both are again detectors instead of callers that were scoped away from the
hazard, and each carries a comment saying not to re-add the scope: it would pass
whether or not the pool is correct.
Mutation-proven, each restored byte-for-byte afterwards. Drop the device check ->
only the refusal case fails (7/8). Also collapse the pool table to one pool ->
4 of 8 fail, exactly the two direction cases plus Drain plus the refusal. Make
ReleaseShared use the device's main pool instead of the buffer's own -> only the
scoped-pool case fails. The two "still a pool" cases (reuse returns the identical
block; two sizes in one class share a block) pass throughout, so this is not
VT_POOL_BYPASS wearing a fix's clothes.
NO docs/FEATURES.md OR docs/USAGE.md UPDATE, AND HERE IS THE ARGUMENT FOR IT,
attached to the diff it excuses because this protocol has no waiver registry.
`check-doc-checkpoint.py` classifies any edit under `src/vllm/model_executor/
models/` as `feature_surface` and any edit under `include/vllm/` as
`user_usage`, so it asks this commit for both. Nothing here is either. No
feature, model, backend or quantization surface changes; no command, C API,
config key, install step or workflow changes. The 24 model TUs are touched by a
mechanical three-lines-to-one call-site rewrite, and the header change is an
internal allocator seam that `include/vllm.h` does not expose. Writing filler
into two public projections to satisfy a path prefix would make them less true,
not more. AGENTS.md states the same rule in prose the other way -- "Editing
src/, include/, or tests/ on its own owes none of these" -- and says the checker
and the prose are deliberately not kept in sync; this is one of the places they
disagree. The gate also already fails on this branch's base for the same reason
at b0aa475 and d67f812, so this is not a new red. A reviewer who does not
accept the argument should not merge it.
Refs #516
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…, and what is still unattributed What neither the code nor git records: what was measured, what was rejected, and why each default is set the way it is. Both symptoms were reproduced at the RED commit on GB10 and both are gone at the fix. The SIGSEGV direction: `--rand-seed=7` exit 139 with the trap it carries -- 44 assertions, 0 failed, beside a crash. The SILENT direction: the shipped 21B FP8 DiT returning `REQUIRE(std::isfinite(v))` FAILED, which needed the opt-in fixture actually satisfied, so the checkpoint was proven READABLE first -- with the NAS down that case SKIPS and the suite reports SUCCESS, an environmental failure wearing the shape of a repair. After: 13/13 and 6176 assertions in every ordering, with no per-case pool scoping left anywhere in the file. #486 was a hypothesis and is now a measurement, in both directions: at the RED commit `test_minimax_h3` SIGSEGVs with the pool on and is 79/79 under `VT_POOL_BYPASS=1`; at the fix it is 79/79 with the pool ON. The nine dgx full-suite failures are recorded as UNATTRIBUTED rather than explained away. The dgx BEFORE arm could not be run -- the box was at 99-100% disk, so two 31 GB trees would not fit -- and two bounded `flock -w 2700` waits for the shared GPU lock expired without acquiring. One of the nine IS resolved: `test_capi` reproduced on the CPU host, where this change cannot cause it, and is 8-of-8 green standalone with per-run times spanning 0.78 to 339.88 s. The other eight get a named next step and the script to run it, not an adjective. Also records the ENOSPC re-verification the operator asked for (no build log contains `No space left`; local gates re-run chained to their build, ninja "no work to do" first, so nothing here is a stale binary), the measured blast radius (42 of 402 binaries instantiate a DevicePool; exactly one instantiates more than one on a single-backend host), the rejected designs, and two things left open on purpose: why a host block yields a uniform quiet NaN on GB10 rather than running correct-but-slow, and `MoeAuxStreamFor` keying on device INDEX alone -- the same family, unreachable today because its only call site is gated on `SupportsAuxStream()`. Refs #516 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ew findings (#516) A fresh review returned FAIL on the LANDING STATE while endorsing the fix itself. This closes all six findings. The code the reviewer endorsed -- the device key, `ReleaseShared`, both workaround removals and `test_device_pool` -- is unchanged in behaviour. F1 (HIGH). The branch was based on `row/MODEL-DIFFUSION-LTX25` @ `aac24761`, which is not an ancestor of `origin/main`, and `git merge-tree` CONFLICTED in `device_pool.h`. Since that merge base, main changed the same file in `49539559d` / `8fa2ecdbb` (Windows contracts, #117) with three changes the reviewed header lacked: `__builtin_clzll` -> `std::bit_width`, an `std::overflow_error` guard in `ClassOf`, and `static SizeClassForTest`. A resolver taking the row's side -- the heavily rewritten side -- would have silently reverted the portability fix and the overflow guard, and `origin/main:tests/vt/test_cpu_isa_x86.cpp` calls `SizeClassForTest` NINE times including `CHECK_THROWS_AS(..., std::overflow_error)`, so it would not have compiled. The four commits are rebased onto `row/MODEL-DIFFUSION-LTX25` @ `2d437d5a9`, which now contains main, and `device_pool.h` was resolved BY HAND: all three of main's changes sit ON TOP of the device-keyed rewrite, not instead of it. `git diff row/POOL-DEVICE-KEY..HEAD -- device_pool.h` is exactly those three hunks and nothing else. `test_cpu_isa_x86` is 6 cases / 8242 assertions / SUCCESS. The row keeps its base. `tests/vllm/models/test_ltx2_device.cpp` does not exist on `main` at all and it is the only test exposing the SILENT-NaN direction, so deleting its workaround -- which the row requires, because a list of remembered callers is what this fault was -- needs LTX-2.5 to land first. F2 (MEDIUM). The row had no record surface, and merging would have made an existing record FALSE. `#516` is now in the issue table of `.agents/roadmap_v1.md`; the spec carries a `## Now`; `docs/STATUS.md` carries the paragraph under "Backend detail"; and `porting-inventory.md` §L8, which still said the shared `DevicePool` "is DEVICE-BLIND ... repairing it is owed as its own row", now says what actually happened. `docs/BENCHMARKS.md` is deliberately not written: this row claims no measurement on any axis. F3 (MEDIUM). Recorded in the spec's §11 with the dgx BEFORE/AFTER pair, the disk and lock state at both ends, and what remains unattributed. F4 (LOW) is CORRECT and the assertions are reworded. Enumerated at the base commit: none of the nine `Release()` sites (`gemma4_moe.cpp:1197,1541`, `qwen3_5.cpp:6324,6520,6820,7100,7134,8034,8065`) is inside or under any of the four `ActivePoolScope` regions (`laguna.cpp:2574`, `qwen3_5.cpp:5468,8644,8964`), which are leaf-ward of all of them. The aux-pool half was LATENT, not live, and `device_pool.h`, `dense_device_glue.h` and spec §4 D4 now say so. `ReleaseShared` stands on its own merits and is untouched. F5 (LOW). Both debug lanes are GREEN, which matters because §10 hands `VT_POOL_BYPASS=1` to the next reader as the cheap discriminator: a suite that reds under the lane it recommends costs that reader an hour deciding whose red it is. A case whose subject is REUSE now states what the ACTIVE lane does, and the size-class case states sharing by default and SEPARATION under `VT_POOL_EXACT` -- which is spec §5 T1.3's second clause, promised since the spec was written and asserted nowhere until now. F6 (LOW). The per-device-type memoization made `platforms::GetPlatform` a per-type call, so an unregistered platform now throws where it used to inherit the first device's cap. Correct, and it had no test; it has one. The `cached[...]` index gains the bound `platforms::Index()` already applies to the same value, in both mirrored copies. Nit: spec §5 T1 said the fakes sit on the `kXPU` slots; the test uses `kCPU` indices 0/1 and is right, because `kXPU` has no registered platform and every pool case would have measured the platform registry instead. The spec is corrected and `kXPU` now earns exactly the one case that is about that throw. New mutation evidence, on this tree: * bound-check + refusal removed (`ResolveDevicePoolPolicy` defaults an unregistered platform to cap 0) -> 8 passed / 1 failed, the T1.7 case only * `VT_POOL_EXACT` made inert in `ClassOf` -> 8 passed / 1 failed under `VT_POOL_EXACT=1`, 4 assertions, the size-class case only * `Bypass()` forced false -> 4 passed / 5 failed under `VT_POOL_BYPASS=1` Tree restored byte-for-byte after each (md5 verified). CPU host, RESOLVED tree, CLEAN rebuild: `BUILD_EXIT=0`, zero warnings under `-Werror`, zero `No space left`/`BFD assertion`, `ctest -N` 412, full `ctest` 412/412 passed exit 0. `test_device_pool` 9 cases / 30 assertions SUCCESS (24 under bypass, 31 under exact, all SUCCESS). Every §6 baseline is where it was except `test_ltx2_vae` 16/1816 -> 33/2602, which is `6c9374ebc` on the CAMPAIGN branch (the VAE encoders) and not this row -- this row does not touch that file. That drift is exactly what F1 predicted a stale base would hide. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
The previous repair (56a917c) fixed the two blocks it was pointed at and then proved completeness by grepping for `still not ported`, `` `encoder_path` is still refused `` and `` `has_encoder()` is still false ``. All three returned nothing -- but those were the phrasings it had just written, so the search could only confirm itself. Searching the vocabulary of the CLAIM instead of the sentence found six more instances of the same false statement, in three docs/USAGE.md sections that were never swept. The file asserted both "A typed prompt works." (:372, added by this PR) and "there is no text encoder" (:1817). Both cannot be true. CLOSED, each against the code site that makes it true: 1. :1816-1821, a bolded present-tense block whose six sub-claims are all false at this head. `ltx-2.5` is one of exactly two registered video families (REGISTER_VLLM_VIDEO_FAMILY, ltx2_video.cpp:1529); --encoder loads the tower and sets has_encoder (ltx2_video.cpp:893); both VAEs and the pipeline layer ship (ltx2_video_vae.cpp, ltx2_audio_vae.cpp, ltx2_pipeline.cpp); /v1/videos registers family-agnostically through LoadVideoEngine; and ltx2-gen renders, documented at :436-450 of this same file. It arrived at 3d89f6f -- the first LTX commit, where it was true -- and was never revisited as L3 through L13 built each of the six pieces it denied. 2-3. :1920 heading "(no render path yet)" and :1924-1925 "There is still no render path, so these are library entry points and not a command". Those loaders are exactly what the render path drives: --dit / --video-dit reaches Ltx2StreamDitToDevice / Ltx2LoadDitFromSafetensors at ltx2_video.cpp:576-577, and --encoder / --video-encoder reaches Ltx2LoadTextEncoderFromSafetensors at ltx2_video.cpp:851. 4-5. :1963 heading "(no render path yet)" and :1965 "There is no LTX-2.5 render path. This section documents one brick." It is one brick OF the shipped path. 6. :1927-1932 said the DiT loader refuses FIVE families, "and the two *_embeddings_connector towers", "which still reports every one of them in Ltx2DitCheckpoint::unported". There are THREE. UnportedFamilies filters the connectors at ltx2_loader.cpp:439 via LoadedElsewhere, and RefuseUnported's own message says so in capitals at ltx2_loader.cpp:461-464 -- the checker's message is the authority on what it enforces. This is the identical claim the previous repair retired 400 lines earlier, left standing precisely because the sweep was built from the text that had just been fixed. Completeness was proved this time from the claim's vocabulary and its subject rather than from the repaired sentences: no render path, no text encoder, no VAE, no pipeline, will not work, one brick, not a command, unported, refuses, embeddings_connector, not ported, is refused, still no, yet, library entry point, layout and forward, asking the video engine -- over docs/, README.md, examples/ and both --help surfaces a user sees. 189 hits read. REPORTED, not fixed, because fixing half of it is worse than reporting it: this branch bumped VLLM_ABI_VERSION 17 -> 18 and added vllm_video_engine_family (36 VLLM_API declarations, was 35) at 3db9233, and left docs/USAGE.md:1464-1465 and README.md:189 and :388 all saying "17" and "35". origin/main is self- consistent at 17/35, so this PR is what falsified them, and docs/STATUS.md:119 already says v18 -- the tree disagrees with itself today. The USAGE half alone would pass check-doc-checkpoint, but check-readme-structure refuses the README half without a landing-source edit, and no landing-source edit is legitimately owed here: examples/server/main.cpp:15's "published as vllm_server_main at ABI v17" is a statement about WHEN that symbol shipped, verified correct at c1716fd. Manufacturing an edit to a blessed surface to turn a gate green is what this protocol forbids, and correcting one of two surfaces that carry the same number would leave a reader unable to tell which is right. The owed change is three lines: 17 -> 18 and 35 -> 36 at those three sites, together. Gate, CPU Release, x86_64, this worktree, no CUDA: - CONFIGURE_EXIT=0, BUILD_EXIT=0, 0 compiler warnings; build log clean of "No space left" and "BFD assertion"; disk 79G -> 64G free, 71G after cleanup. - ctest -N: 414 registered. - full ctest -j8: 413 passed, 1 skipped (test_voxtral_e2e), 1 failed (test_engine_core_proc) -- the known -j flake, which PASSES serially: "1/1 Test #195: test_engine_core_proc ... Passed". - test_ltx2_video: 30 cases / 499 assertions, exit 0 -- unchanged from this head's recorded counts, as a documentation-only change must be. - check-doc-checkpoint --staged: OK. check-public-doc-tables: OK. The preflight "doc-checkpoint range" failure is INHERITED and re-verified: b0aa475, d67f812 and aa6aa0e are each an ancestor of the base 67a7b1c. This is instance sixteen in the campaign #604 tracks, and the second in this file and this PR. Refs #435, #604. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ures attributed (#516, #486, #614) Spec §11. The review's F3 said this row "must not land on a CUDA-unattributed gate": every direct symptom of #516 needs a GPU, both removed workarounds guard CUDA cases a CPU host never executes, and §10 had nine dgx failures with NO BEFORE arm because the box was at 99-100% disk with a single 31 GB build tree. The pair now exists: both arms clean-built from the SAME base, back to back on one box, `ctest -j 1`, all three MANDATORY confirmations printed for each and zero `No space left` in four logs. BEFORE aa6aa0e, no fix 10 failed of 449 lock 08:26:13Z-09:22:00Z AFTER this branch 9 failed of 450 lock 09:40:20Z-10:34:02Z (13m45s bounded wait, not forced) THE DENOMINATORS DIFFER BY ONE BECAUSE THIS ROW ADDS A TEST. 449 -> 450 is test_device_pool existing in the AFTER arm and not the BEFORE one; it is not drift, and 10-of-449 read against 9-of-450 without that is off by one test. On the 449 both arms share: 10 failures before, 9 after, and exactly ONE leaves. test_minimax_h3 ***Exception: SegFault 11.73s -> Passed 19.06s test_device_pool (does not exist) -> Passed The other nine are the SAME set in both arms, and each is now matched to an issue rather than left as a name: #233 (test_serve_low_tools, test_linear_method, test_glm4_moe_lite_paged_engine), #248 (test_capi, test_qwen3_apc_e2e, test_minicpm3_paged_engine, test_llama_paged_engine), #305 TENTATIVELY (test_ops_gdn), and #614, filed by this row, for test_internlm2_paged_engine, which had no issue anywhere. So all nine are MEASURED pre-existing rather than argued. This row does not fix them and does not adopt them. test_capi IS NOT THE TIMING FLAKE §10 CALLED IT, and the correction matters. §10's flake evidence -- 8/8 under --repeat until-fail, wall times 0.78s to 339.88s -- came from the CPU HOST, where the test fails without crashing. On dgx it SEGFAULTs, in BOTH arms, in 1.95s and 1.97s, under ctest -j 1. A sub-2-second segfault is not a three-orders-of-magnitude timing spread, and #248 says so in as many words. Pre-existing either way, but it goes back to #248 as a crash rather than into a flake story it does not fit. Whether the two arms' crashes are the SAME crash is NOT established: neither LastTest.log survives (both 34 GB trees were deleted to keep a 98%-full box safe) and these runs were plain ctest -j 1 with no --output-on-failure, so the recorded evidence is the signal, the index and the duration. It also re-confirms #486 in BOTH directions on ONE box, 14 minutes apart, same binary recipe, same lock: SEGFAULT with the pool device-blind, Passed with it device-keyed. §10 asserted that from two separate sessions; it is now one paired measurement. Both 34 GB build trees were deleted after their arm. The box was at 98-99% throughout and ended at 94 GB free. The pair was taken at base aa6aa0e; the branch is now rebased onto 310fa16, fifteen commits later, and the CPU gate was re-run in full there (415/415, clean rebuild, zero warnings). Chasing a moving campaign branch with a five-hour paired CUDA gate does not terminate, so §11 names the base the pair was taken at and states what would invalidate it: none of the fifteen intervening commits touches device_pool.h, dense_device_glue.h, the DBuf deleters or any pool accessor, and a tree-wide sweep for a device-less Pool()/AuxPool()/ActivePool() comes back empty on the new base. §11 also records what §10 must no longer be read as saying: every number in it was measured on a tree missing main's last 201 commits, and none of it is carried forward. `## Now` is updated to match -- the nine are closed against this row, and the two things that remain unestablished (the ATS/quiet-NaN mechanism, and `MoeAuxStreamFor`'s index-only key) are named rather than quietly dropped. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This campaign moved VLLM_ABI_VERSION 17 -> 18 and added a 36th export (vllm_video_engine_family) back in L1, and never updated the surfaces that quote those numbers. They are correct on origin/main and FALSE on this branch, which is the worst possible split: a reader diffing the two would conclude the README was right and the header wrong. Fixed together, because a number carried on several surfaces must move on all of them at once or a reader cannot tell which is authoritative: README.md:189 ABI v17, 35 functions -> v18, 36 README.md:388 VLLM_ABI_VERSION 17, 35 -> 18, 36 docs/USAGE.md:1392 VLLM_ABI_VERSION 17, 35 -> 18, 36 docs/FEATURES.md:290 vllm_server_main (ABI v17) -> (ABI v18) examples/server/main.cpp:15's "at ABI v17" is deliberately NOT touched: it is a WHEN-SHIPPED statement, verified correct at c1716fd, and rewriting it would turn an accurate historical note into a false present-tense one. How this was found is the part worth keeping. An implementer swept docs/ for stale claims with 17 patterns and read 189 hits -- thorough work that found the README/USAGE pair. It missed FEATURES.md and a second USAGE passage because its patterns were phrasings ("no render path", "not ported"), and these say "ABI v17" and "35 exported". A one-line grep for the SUBJECT -- the version number itself -- found the rest immediately. That is the same lesson as the self-confirming grep, one level up: searching for how a claim is WORDED finds the instances you already imagined. Searching for what the claim is ABOUT finds the ones you did not. No source file changed, so ctest was not re-run; check-public-doc-tables and check-readme-structure both pass. Issue: #435, #604 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…k came from Reviewed PASS at 2098683 after one FAIL round. #516 and #486 are the same bug, and the fix caught a live instance of itself on this very branch. The defect: one process-wide free list held blocks from every backend that ever used it, so a block allocated by device 0 could be handed to device 1. CUDA->CPU was a SIGSEGV; CPU->CUDA was SILENT, a uniform 0x7fff0000 all-NaN, which no token gate can see. A DevicePool is now bound to one backend at construction and the no-argument Pool()/AuxPool() are REMOVED, so "the pool" without a device is no longer expressible. That removal is what made the difference. ltx2_video.cpp:1154 called ActivePool()->Drain(backend) -- the device-less spelling, draining every retained block through one backend regardless of which allocator made it, in a stage whose own comment says it runs on the host. It arrived with L9c, is an ancestor of the dgx measurement base, and was live in both arms. minimax_h3_pipeline.cpp:559 carried it identically. A free-list keyed by device would NOT have caught either; deleting the spelling turned both into compile errors. The dgx pair settles the attribution that a prior review had to leave open: BEFORE 10 failed of 449, AFTER 9 of 450 -- denominators differ because this row adds test_device_pool, which passes. On the 449 shared tests exactly one moves: test_minimax_h3 SegFault 11.73s -> Passed 19.06s. The other nine reproduce on the base, so they are measured pre-existing rather than labelled unattributed, and each is now matched to an issue -- including #614, filed because nothing tracked it. Two things the row declined to do, both right. It did not carry the recorded test_capi timing-flake label onto a SEGFAULT that appears in BOTH arms; whether the two crashes are the same crash is recorded as NOT ESTABLISHED, because the logs went with the deleted build trees. And it did not re-run a five-hour paired CUDA gate to chase a moving branch: it pinned the pair's base and named the condition that would invalidate it, which the reviewer then re-verified against this head. Issue: #516, #486, #614 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…#435) The last round corrected the post_processor rationale in the HEADER and left the SOURCE saying the opposite thing, twelve lines above the `Encode` call it annotates. The two made contradictory predictions from the same premise: the header said a future post_processor makes us WRONG, the source said it keeps us RIGHT. F7's whole content was "the rationale was inverted", so half of it was still open. VERIFIED AGAINST UPSTREAM, not taken on trust. The reviewer could not reach the anchor; this was read at /home/mudler/_git/LTX-2 pin fd4ded7f: packages/ltx-core/src/ltx_core/text_encoders/gemma/tokenizer.py:37-43 encoded = self.tokenizer( text, padding=False, truncation=True, max_length=self.max_length, return_tensors="pt", ) That is `__call__` with its DEFAULT `add_special_tokens=True`, so upstream DOES run the post_processor. The HEADER had the direction right; the SOURCE was the wrong one, and it is the source that is corrected here. A THIRD claim in the same comment was also false. "Doing both would double the BOS" is not true: upstream's own guard at :45 (`not input_ids or input_ids[0] != bos_id`), mirrored at ltx2_text_encoder.cpp:712, absorbs a leading BOS from either call. The real exposure is a post_processor that adds anything ELSE, and the comment now says that instead. SIBLING SWEEP, built from the SUBJECT (post_processor, BOS, add_special_tokens, Encode) rather than from the sentences being fixed, since a grep built from the new text only confirms itself. Two more hits, both corrected: * gen-ltx2-prompt-tokens-goldens.py:86 annotated its `add_special_tokens=False` as transcribing ":38-43" -- the same conflation, in the generator that produces the committed goldens. Anchor corrected to :37-43 and the deliberate flag flip is now stated. * ltx2_text_encoder.h:421 called the BOS prepend "unconditional", which contradicts test_ltx2_text_encoder.cpp:2398 ("i.e. conditional") and :45 itself. It is conditional, and the guard is why. ──────────────────────────────────────────────────────────────────────────────── TWO COMMENT OVERCLAIMS, PROVED BY THE REVIEWER'S MUTATION Replacing `Ltx2ComputeRightPadOrder`'s stable partition with the identity permutation reds two OTHER cases (22 assertions) and leaves the new conditioning case entirely green. `out.conditioning` and `want_f32` both reach `sort_index` and `additive_mask` through the SAME function over the SAME `out.mask`, so a defect there cancels and those ~40 assertions cannot fire. "Held EXACTLY" was an assertion true by construction wearing a gate's label. This is NOT a coverage hole and no test changes. Both are genuinely gated by "ltx2 text: additive mask, right-pad ordering and the binary mask" and "ltx2 text: the encoder -> conditioning hand-off", which compare against committed upstream goldens rather than against a second call of the code under test. The comment now says what its assertions do establish (the mask is threaded through unaltered, and the mask itself is held to the golden) and names the two cases that actually gate the ordering. ──────────────────────────────────────────────────────────────────────────────── TWO COVERAGE LIMITS RECORDED, NOT FIXED * The loader case's "the q|k|v concat ORDER" claim holds for the LTX path ONLY. `Ltx2LoadGemmaTowerFromSafetensors` uses its own `TowerConcat` (ltx2_text_encoder.cpp:943-945); `gemma4_weights.cpp:281-295` is a SEPARATE implementation this suite never loads, and mutating it leaves every case here green. That path owes its own gate. * The rope-table instrument covers the FULL-ATTENTION arm only. The sliding layers' `default` rope at theta 1e4 has no equivalent f32 instrument and is reached only through the hidden states, where the same bf16 noise-floor argument works against resolving a config-carried angle defect. ──────────────────────────────────────────────────────────────────────────────── DOCS Comment-only in src/include/tests, so AGENTS.md owes nothing here -- but check-doc-checkpoint is path-derived and demands FEATURES.md (from src/vllm/model_executor/models/) and USAGE.md (from include/vllm/). The prior repair at 9fbc682 paid it the same way. Both edits are GENUINE rather than gate-appeasement: they record the tokenization divergence this commit just verified, which is a real user-facing caveat -- our prompt tokenization mirrors upstream only while the checkpoint's post_processor stays empty. USAGE.md:454 and :473 are left alone; they are inherited from the base branch and belong to PR #600's lane. The FEATURES.md note is folded into an existing paragraph rather than added as a new one, because check-public-doc-tables budgets prose paragraphs (21) as well as table cells (220 chars) and rows (600), and the LTX row's cell is already at 217. CODE IS BYTE-IDENTICAL. The only non-comment line in the diff is a trailing comment on an unchanged statement. GATE, CPU, Release, VLLM_CPP_CUDA=OFF: BUILD_EXIT=0, no "No space left" / "BFD assertion", df 86% used test_ltx2_text_encoder: 26 cases / 4115 assertions, Status SUCCESS, exit 0 (matches the declared baseline exactly) ctest -N: 414 (the brief said 409; this configure enumerates 414 at the unmodified head, and a comment-only diff cannot register a test) ctest: 414/414 passed, 0 failed, exit 0, first pass under -j 8, no flaky re-runs needed binary md5 unchanged across the post-docs rebuild (8b48f39b...) check-doc-checkpoint "range" still reports b0aa475, d67f812 and aa6aa0e. All three are ancestors of 9fbc682, inherited, and not repairable from here without editing #600's lane. Issue: #435, #604 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Reviewed PASS at 4e5d4a2 after two FAIL rounds, both on the record surface rather than the code. The final review read 571 hits from subject-built patterns and cross-checked every vllm_* symbol in the public docs against include/vllm.h and every repo path cited in USAGE.md, with no real misses. What lands: the chain from a prompt string through tokenizer, Gemma-4 tower, caption projections and the connector to Ltx2ModalityInput::context, closed with no gap; the last_conditioning() trace with its race fixed and gated in both directions; and the honest bound on all of it. The bound is the part worth reading. last_conditioning() is a WITNESS, not a gate: scaling the conditioning x1.5 or REVERSING its rows still passes every assertion, so it detects change rather than correctness, and the two joins have no numeric oracle. That is recorded at the code, in the append-only record, and in the public docs, which now say a change detector is not a quality measure. Three conflicts. docs/USAGE.md's first was two DIFFERENT sections git aligned badly -- the pipeline-goldens regeneration note and the Gemma-4 tower gate -- so both are kept. Its second took L13's text, because the campaign side still said "there is still no render path", which stopped being true; but L13's "an NVFP4 DiT" became "both NVFP4 DiTs", since L9a landed first-party NVFP4 after L13 branched. docs/FEATURES.md is the same keyed row for the SIXTH time (#595), and again neither side was a superset: the campaign knew both VAE encoders and all three NVFP4 arms loading, L13 knew the prompt path and that the 320x192/25f scene was REGISTER-conditioned with a prompted render still owed. L13's "first-party NVFP4 does not load" was stale on arrival. Composed to what is true of the merged tree. Issue: #435, #604 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Merging L13 produced a tree that did not compile: ltx2_text_encoder.cpp:796: error: 'Ltx2DequantTorchaoNvfp4ToBf16' was not declared in this scope; did you mean 'Ltx2DequantNvfp4ToBf16'? L9A renamed that seam and gave it a mandatory `producer` argument. L10/L13 branched before it and still called the old name. Different files, so the textual merge was clean and `git merge-tree` reported no conflict -- which is exactly why merge-tree returning clean is not evidence that the result builds. The seam then behaved as designed. `producer` is deliberately NOT defaulted (ltx2_loader.h:323-325): a default would let a caller who never thought about the question silently get the torchao reading for an nvfp4-prequant file, which is the failure that seam exists to refuse. This call site had already run ParseLtx2TorchaoNvfp4Marker one line above, so it can state kTorchao as established rather than assumed, and the comment now says so. Two things about the failure itself. The build error was the ONLY signal: the run reported "100% tests passed, 0 tests failed out of 408" against a registered count of 415, because ctest correctly did not run and the harness grepped a stale log from an earlier run. A tally that disagrees with `ctest -N` is not a pass, it is a stale artifact -- the harness now deletes the log first and prints "BUILD FAILED - ctest NOT run, no tally to report" rather than a number. Re-gated clean: BUILD_EXIT=0, zero warnings, enospc 0, ctest 415/415 exit 0, registered 415 -- tally and registration agreeing, which is the check the mismatch above was trying to make. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
, #604) The re-review found `test_ltx2_text_encoder.cpp:635` citing `ltx2_text_encoder.cpp:943-945` for "the LTX tower loader's own `TowerConcat`". Lines 943-945 are the `q_proj` `TowerModule` load. The concat calls are :951 and :953; the helper is defined at :835. The substantive claim was measured and is true, and only the number was wrong -- which is the worst shape for this defect, because a reader who follows the anchor concludes a TRUE claim is unsupported. Fixing one number and stopping is what let the class survive three review rounds, so every `file:line` this branch ADDS over `row/MODEL-DIFFUSION-LTX25` was resolved against the tree it points into. Extracted mechanically by SHAPE off the added lines of `git diff origin/row/MODEL-DIFFUSION-LTX25...HEAD` (`name.ext:NN`, `name.ext:NN-MM`, bare `:NN` / `:NN-MM`, "line(s) NN", "SS N"), not by grepping the sentences already known to be wrong -- a grep built from text you just fixed only confirms itself. 19 citations added, in 6 files. 19 resolved. 8 wrong, at 6 comment sites. site cited is ------------------------------------------------------------------------ test_ltx2_text_encoder.cpp:635 ltx2_te.cpp:943-945 q_proj load test_ltx2_text_encoder.cpp:1959 rope_utils.py:187-245 ends at 254 test_ltx2_text_encoder.cpp:1992 rope_utils.py:233 blank line gemma4.h:260 gemma4_unified.py:257-274 prev fn tail gemma4.h:260 rope_utils.py:187-245 ends at 254 gen-ltx2-gemma-tower-goldens.py:378 gemma4_unified.py:214-218 call, no lookup gen-ltx2-gemma-tower-goldens.py:378 rope_utils.py:187-245 ends at 254 ltx2_text_encoder.cpp:1055 gemma4.cpp:300-312 cuts the cast The three `modeling_rope_utils.py:187-245` sites are one anchor repeated. The function is `_compute_proportional_rope_parameters`, :187-254, and all three sentences name its ZERO PADDING -- which is `torch.zeros(nope_angles, ...)` at :246, ONE LINE past the cited end. Following the anchor is following it to the `torch.cat(` that opens two lines earlier and stopping before the argument that matters. Corrected to :187-254, with :246 named where the padding is the claim. `modeling_rope_utils.py:233` is a blank line. `rope_angles = int(rope_proportion * head_dim // 2)`, which the comment quotes, is :234. `modeling_gemma4_unified.py:257-274` opens on the PREVIOUS function's `return` (:257) and stops before `forward`'s own (:275). The `emb = torch.cat((freqs, freqs), dim=-1)` the sentence mirrors is :271, so the anchor does contain it, but both ends land in the wrong construct. Corrected to the whole method, :259-275, with :271 named. `modeling_gemma4_unified.py:214-218` is the CALL, `rope_init_fn(self.config, **rope_init_fn_kwargs)`. The sentence claims the class "routes `rope_type: proportional` to `_compute_proportional_rope_parameters`", and the routing is `ROPE_INIT_FUNCTIONS[rope_type]` at :207, outside the range -- a reader at :214-218 cannot see what `rope_init_fn` resolves to, so the claim reads unsupported. Widened to :206-218. `gemma4.cpp:300-312` opens on the weight-less V `RmsNorm` at :300 and closes on the `} else {` at :312, which EXCLUDES the `vt::CastF32` pair at :313-314 that the sentence is about ("run a CastF32 on every K and V it wrote"). The `kv.dtype != adt` arm is :306-315. Corrected, with the two `DBuf`s and the casts named separately, because those are the two costs the sentence charges. The eleven that resolve as written, checked one by one, not assumed: LTX-2 `tokenizer.py:37-43` (three sites, the `self.tokenizer(...)` call), `:44-46`, `:45`, `:12-15`, `gemma_assets.py:335-386` (`build_text_encoder_tensors_from_gemma_root`, :335 to its `return` at :386), `base_encoder.py:41`; diffusers `pipeline_ltx2.py:339` (literally `add_special_tokens=True`); ours `gemma4_weights.cpp:281-295`, which does span all three qkv arms as the previous round reported. Two non-line anchors added by the F7 commit -- the two test-case NAMES it cites as the ones that gate the ordering -- exist verbatim at :1115 and :1166. PINS. LTX-2 at `fd4ded7f`, tree clean. diffusers at `3a2f35d4`. transformers 5.12.1 at `/home/mudler/recon-cpu/venv`, the interpreter the tower generator actually ran under; both files are md5-IDENTICAL to `/home/mudler/_git/ transformers` @ `7d06b1a5`, so the numbering above is not an artifact of which install was read. F10's anchor. `:951-953` and not `:835`, because the sentence is the LTX half of a contrast whose other half is `gemma4_weights.cpp:281-295` -- assembly SITES, not a helper definition -- and because the qkv concat ORDER is what the case gates and what the previous round's mutation (`q,k,v` -> `q,v,k`, 14 assertions RED) perturbed at :951. `:835` is named alongside it so the helper is still one hop away. DOCS. The code is byte-identical; this is comments and two documents. `check-doc-checkpoint` is path-derived and demands `FEATURES.md` (from `src/vllm/model_executor/models/`) and `USAGE.md` (from `include/vllm/`), which is the same bill `9fbc68256` and `ddfcb516a` paid. Both edits are genuine. `USAGE.md` still opened its LTX-2.5 gate section with "**There is no LTX-2.5 render path yet** ... no text encoder, no VAE, no pipeline and no `/v1/videos` route for it -- asking the video engine for LTX-2.5 will not work." Every clause is false, and the SAME DOCUMENT contradicts all of them: `ltx2-gen` renders end to end at :412, `--video-family ltx-2.5` pins the family on the server at :435, and the text tower is this row. Rewritten to say what the section is for. The refusal list that follows is left byte-for-byte alone: it is inherited text, it is not part of this finding, and `ltx2.h:38` and `:47-49` do support the two clauses of it I checked. `FEATURES.md` pointed at "`ltx2_text_encoder.cpp` names the line that would have to change" -- an unresolvable pointer, in a commit about unresolvable pointers. It now names `Ltx2TokenizeGemmaPrompt`. A symbol, deliberately, because a line number in a public document has nothing that can keep it true. Folded into the existing paragraph: the 700-char prose cap and the 220-char cell cap make an addition an EVICTION of someone else's content, and the LTX row's two cells are already at 217 and 215. `docs/USAGE.md:454` and `:473` untouched -- #600's lane. GATE -- CPU, Release, `VLLM_CPP_CUDA=OFF`, `build-f10`, deleted after. cmake --build build-f10 -j 16 BUILD_EXIT=0; 0 hits for 'No space left|BFD assertion'; df 85% ctest -N 414 ctest -j 8 --output-on-failure 414/414 passed, 0 failed, exit 0 ./build-f10/tests/test_ltx2_text_encoder 26 cases | 4115 assertions | 0 failed Status: SUCCESS, exit 0 (baseline 26 / 4115 matched exactly) First pass under `-j 8`; no suite needed a serial re-run. `doc-checkpoint --staged` and `now-current --staged` both ok. `doc-checkpoint range` still reports `b0aa475a3`, `d67f8125e` and `aa6aa0ecd` -- the identical three failures this worktree recorded BEFORE any edit, all ancestors of `9fbc68256`, inherited and not repairable from here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…swept out Reviewed PASS at 131328e after three FAIL rounds. The last one is the reason to read this: told to fix ONE wrong file:line, the implementer swept every citation the branch adds -- 19 of them -- and found EIGHT wrong at six sites. Fixing the instance would have left seven. Three of the eight are one anchor repeated. modeling_rope_utils.py:187-245 was cited three times as the authority for that function's ZERO PADDING, which is at :246 -- one line past the cited end, so the range stops inside the torch.cat( argument list immediately before the argument that IS the whole claim. Our own gemma4.cpp:300-312 has the identical shape: it closes on `} else {`, leaving the vt::CastF32 pair at :313-314 that the sentence charges outside the range. A range that stops just short of its own subject is worse than a missing citation, because it looks checked. The reviewer then re-extracted the citations independently and confirmed the sweep missed none, resolved all eight corrections itself at the pins, verified the transformers files are md5-identical between the venv and the clone so the numbering is not install-specific, and proved code identity by comment-stripping and hashing rather than by eye. Three conflicts, none of them a union. docs/USAGE.md's first: the base says "A typed prompt works" because L13 landed it; L10 still opened "There is no prompt", stale on arrival. Took the base's framing but KEPT L10's tokenization-divergence paragraph, which is unique to it and still true -- upstream tokenizes through __call__ with add_special_tokens defaulting True and so runs the post_processor, while this port calls plain encode and prepends BOS; identical only because the shipped post_processor's special-token map is EMPTY, measured on the file rather than assumed. Its second and docs/FEATURES.md both took the base wholesale: the base's parity paragraph is a strict superset carrying anchors, the ltx2.h refusal list and a provenance note, and L10's row still said first-party NVFP4 does not load (L9A landed it) and described an encoder_path refusal L13 lifted. Issue: #435, #604 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Second and last absorption of main (the first was 192 commits at 5b598fc). Eleven commits, one conflict, and it is the same append-only log as last time: .agents/benchmark-record.md, where both sides appended distinct sections at the same anchor. Unioned with main's first and both verified present by content rather than by the merge exiting zero. Nothing else collided. Notably the Nemotron rows, the DSA top-k gating and the Windows Invoke-Checked fix all landed on main during this campaign without touching an LTX file. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Eight more commits landed on main while the dgx gate ran. None touches an LTX path: the closest is bd3b755's GDN output_gate_type, which changes qwen3_5.cpp and hf_config, not vt/cuda, not device_pool, not ltx2_*. So the CUDA verification taken at 105e785 still covers this work, and the eight were gated by their own PRs. That is the same pinned-base-plus-invalidation-condition reasoning the pool row used, applied honestly rather than re-run for its own sake. Five conflicts, all append-collisions between two branches doing unrelated work: docs/USAGE.md the LTX-2.5 section vs a new GDN output_gate_type section -- different subjects, both kept server_main.cpp my --video-family registry check vs main's --enable-auto-tool-choice check -- two independent validations, both kept model-matrix.md six new recipe-architecture rows vs the LTX-2.5 row, no key collision, unioned check-agent-record.py both provenance comments kept; the union left TWO "MODEL" pin entries, collapsed to one test_agent_record.py both narratives kept The architecture-count pin is the interesting one. Both branches bumped the SAME number from the same base -- mine 362->363 for LTX-2.5, main 362->369 for seven recipe rows -- so neither value is right after a merge and the arithmetic is not the check. I set a hypothesis and let `check-agent-record.py` be the authority on it: it reports MODEL=370, which is 362 + 7 + 1. My own grep of the table said 366, which is what happens when you count rows with a pattern that is not the one the checker counts with. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #435.
Lands LTX-2.5 (21.00B joint video+audio flow-matching DiT) and generalizes the video seam so this model class becomes additive. Developer-directed to keep the whole campaign on one PR.
Phase state
VideoEngineseam, H3 behind it, ABI v18row/LTX25-L1-SEAMrow/LTX25-L1-FIX, re-review runningrow/LTX25-L2-DITrow/LTX25-L3-TEXTrow/LTX25-L4-VAEEmbeddings1DConnectorflockLightricks/LTX-2.5HF gateEvery phase gates against upstream
ltx_coreexecuted at reduced dimensions on CPU, both sides rebuilding weights from one deterministic stream, so no weight byte is checked in. Upstream revisionfd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca.Measured parity (max abs diff vs upstream)
DiT forward 1.19e-07 video / 4.84e-08 audio, across split RoPE, interleaved RoPE, the float64 ladder, masks and audio-disabled. VAEs 1.16e-08 to 1.79e-06 across eight bricks. Text conditioning: hidden-state stack bit-equal, extractors 2.98e-08 to 6.52e-08. All independently reproduced by the reviewers.
What review caught that implementation did not
std::uniquehiding the collision.norm_epstolerating a 100x change and the BWE mel log clamp that real silence hits in production but reduced-dimension goldens never reach.causal=falsepath was entirely ungated. The L4 reviewer generated its own goldens and proved our code correct there.Two spec errors, corrected in place
ABI "v13" was wrong: the counter was already 17, v13 shipped long ago as
vllm_complete_tokens. Now 18.The TE projections take 188160 = 3840 x 49, not 94080 = 1920 x 49. NVFP4 packs two values per byte along the last dim, so U8 header widths are half the logical ones.
model.norm.weightis BF16 [3840] and BF16 is unpacked, which settles it. The general rule now recorded for L6: read a quantized checkpoint's UNPACKED tensors to establish a width.Recorded as owed, not discovered later
PENDING, structurally: vLLM-Omni has no native 2.5 path (vllm-omni#6066) and its generic diffusers adapter is a black box (supports_step_execution=False), so it is not vLLM's production denominator.PENDINGon-Diffusersaccess. Everything so far gates against theltx_corecross-check.Embeddings1DConnector, and the bf16/NVFP4 arms.Related: #441 (ABI struct growth is source- but not binary-compatible; predates this work).
🤖 Generated with Claude Code