record(LTX25-PROMPT-ADALN): the prompt-AdaLN defect was understated ~138x, and the keyframes claim next door was false (#644)

.agents/model-matrix.md

@@ -445,7 +445,7 @@ Transformers compatibility is capability-driven and excluded from finite counts.
 | `MODEL-MM-higgs-audio-v3-higgs-audio-v3-talker-for-conditional-generation` | `HiggsMultimodalQwen3ForConditionalGeneration`, `HiggsAudioV3TalkerForConditionalGeneration` | **OUT-OF-REPO** — not in `555967922`, and absent from `registry.py` on vLLM `main`; registered by `vllm-project/vllm-omni` at `vllm_omni/model_executor/models/registry.py:213-217` (and the explicit `HiggsAudioV3TalkerForConditionalGeneration` spelling at `:218`) @ `bbe6ccc512a404a2df8c977ea29003002f2683e8` → module `vllm_omni/model_executor/models/higgs_audio_v3/higgs_audio_v3_talker.py`, class `HiggsAudioV3TalkerForConditionalGeneration` (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention), with code2wav at `registry.py:223-227` → module `higgs_audio_v3/higgs_audio_v3_code2wav.py`, class `HiggsAudioV3Code2WavForConditionalGeneration`. Documented at `vllm-omni` `docs/models/supported_models.md:81`. Official recipe: `bosonai/higgs-audio-v3-tts-4b` (published at `recipes.vllm.ai`), whose `config.json` declares `architectures: ["HiggsMultimodalQwen3ForConditionalGeneration"]`. The distinct v2 lineage (`HiggsAudioV2ForConditionalGeneration`, `higgs_audio_v2/`) is a DIFFERENT target and has no recipe in the audited set, so it gets no row here | conditional generation / text+audio in, AUDIO OUT (TTS) | MM processor; audio detokenizer / code2wav; audio tokenizer; Qwen3-derived multimodal decoder; streaming audio output | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-voxtral-realtime-voxtral-realtime-for-conditional-generation` | `VoxtralRealtimeForConditionalGeneration` (target-pending) | **TARGET-PENDING — the architecture string is registered NOWHERE we can find.** Recipe `mistralai/Voxtral-Mini-4B-Realtime-2602` (one of the 157 recipes at `vllm-project/recipes` `86c7777a`) declares `architectures: ["VoxtralRealtimeForConditionalGeneration"]` with an inner `audio_config.model_type: "voxtral_realtime_encoder"`, but that exact string appears in none of: (a) the pinned registry `555967922`; (b) any of the ten dictionaries of `vllm/model_executor/models/registry.py` on vLLM `main`; (c) `_OMNI_MODELS` in `vllm_omni/model_executor/models/registry.py` @ `bbe6ccc512a404a2df8c977ea29003002f2683e8`; (d) `vllm-omni` `docs/models/supported_models.md`. **Related but NOT equal, recorded so nobody re-derives it:** `VoxtralRealtimeGeneration` — a DIFFERENT string — is registered at the pin (`registry.py:585`, `main` `:603`) and already has its own row here, `MODEL-MM-voxtral-realtime-voxtral-realtime-generation`; `VoxtralForConditionalGeneration` (pin `registry.py:584`) is the audio→TEXT model; and `vllm-omni` `registry.py:305-309` registers `VoxtralTTSForConditionalGeneration` for the SEPARATE `mistralai/Voxtral-4B-TTS-2603` recipe. The most likely reading is that this checkpoint's `config.json` names a renamed or not-yet-landed spelling of the existing `VoxtralRealtimeGeneration` target, but that is a hypothesis: no anchor is asserted until one of those is shown to be what actually resolves. Tracked by [#610](https://github.com/mudler/vllm.cpp/issues/610) | conditional generation / audio in, AUDIO OUT (realtime speech-to-speech) — modality inferred from the recipe and config, NOT from an upstream implementation | unresolved until the target is located | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-bailing-mm-native-bailing-mm-native-for-conditional-generation` | `BailingMMNativeForConditionalGeneration` (target-pending) | **TARGET-PENDING — the architecture string is registered NOWHERE we can find.** Recipe `inclusionAI/Ming-omni-tts-0.5B` (one of the 157 recipes at `vllm-project/recipes` `86c7777a`) declares `architectures: ["BailingMMNativeForConditionalGeneration"]`, but that exact string appears in none of: (a) the pinned registry `555967922`; (b) any dictionary of `vllm/model_executor/models/registry.py` on vLLM `main`; (c) `_OMNI_MODELS` in `vllm_omni/model_executor/models/registry.py` @ `bbe6ccc512a404a2df8c977ea29003002f2683e8`; (d) `vllm-omni` `docs/models/supported_models.md`. **Related but NOT equal:** `vllm-omni` `docs/models/supported_models.md:75` lists that SAME checkpoint under a different architecture name, `MingTTSForConditionalGeneration` (`registry.py:250-254` → module `ming_tts/ming_tts.py`, class `MingTTSForConditionalGeneration`); and `vllm-omni` `registry.py:417-421` carries a near-miss alias `BailingMM2NativeForConditionalGeneration` — note the `2` — commented "HF repo currently ships this architecture name in config.json", pointing at module `ming_flash_omni/ming_flash_omni.py`, class `MingFlashOmniForConditionalGeneration`, for the DIFFERENT Ming-flash-omni-2.0 checkpoint. So the shape of the gap is a missing config-string alias upstream, but that is an observation, not an anchor, and none is asserted here. Tracked by [#610](https://github.com/mudler/vllm.cpp/issues/610) | conditional generation / text+image+audio in, AUDIO OUT (TTS) — modality from the recipe and config, NOT from a resolved upstream target | unresolved until the target is located | ☐ required | `INVENTORIED` | none | unassigned |
-| `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model` | `LTX2VideoTransformer3DModel` | **BEYOND-PIN AND OUT-OF-REPO** — not in `555967922` (LTX-2.5 released 2026-08) and not in the vLLM repository at all. Architecture reference is Lightricks' own `LTX-2` (`packages/ltx-core/src/ltx_core/`: `model/transformer/{transformer,model,attention,rope,model_configurator}.py`, `model/audio_vae/`, `components/`, `guidance/`). vLLM-Omni carries an `ltx2` module but its recipes stop at 2.3 (`vllm_omni/diffusion/models/ltx2/ltx2_recipes.py:162-166`); 2.5 is upstream-OPEN at [vllm-omni#6066](https://github.com/vllm-project/vllm-omni/issues/6066) and [#4985](https://github.com/vllm-project/vllm-omni/issues/4985). Serving oracle reaches 2.5 through the GENERIC `DiffusersAdapterPipeline` (`pipeline_diffusers_adapter.py:116`) against `Lightricks/LTX-2.5-Diffusers`. | diffusion generation / video + audio (text/image/video in, video+audio out) | flow-matching denoise loop; TWO coupled modality streams with audio<->video cross-attention; per-head gated attention; AdaLN-single with 9-vector per-block tables; timestep-INDEPENDENT prompt K/V (cacheable once per request); split/interleaved 3D RoPE; gelu-approximate FFN; Gemma-4 12B text encoder with dual caption projections (4096 video / 2048 audio); Conv video VAE + DiffVAE; audio VAE + vocoder; latent spatial/temporal x2 upsamplers; duration head; distilled two-stage recipe | [ltx-2.5](specs/ltx-2-5.md) | `SPIKE` | **L0 LANDED 2026-08-11 (#435).** Spec committed BEFORE implementation per AGENTS.md. Geometry measured from the ungated `vonkaiser/LTX-2.5-FP8-NVFP4` FP8 checkpoint header by range request — the same no-payload technique used for H3's manifests: 21.00B params, 48 blocks, 1775 F8_E4M3 tensors each with a per-tensor F32 `weight_scale`, biases/norms BF16. Cross-modal projection shapes are ASYMMETRIC and are a named test trap: `audio_to_video_attn.to_q` is [2048, 4096] while `to_k`/`to_v` are [2048, 2048] and `to_out` is [4096, 2048], so a transposition still type-checks against a square assumption. **CHECKPOINT ACCESS (verified against the HF API 2026-08-11):** `Lightricks/LTX-2.5` is `gated: auto` (accepting the license opens it; holds the first-party NVFP4 DiT at 18.72 GB), `Lightricks/LTX-2.5-Diffusers` is RESTRICTED behind manual approval (needed for the binding oracle), and `vonkaiser/LTX-2.5-FP8-NVFP4` is UNGATED and unblocks L1-L2 today. All artifacts land under `$CHECKPOINT_ROOT` (`/mnt/nas_share/checkpoints`) so dgx.casa and the cluster nodes share one copy. **PHASES (one PR, developer-directed):** L1 `vllm::multimodal::VideoEngine` interface + checkpoint-detected registry with MiniMax-H3 moved behind it UNCHANGED (gated on frames+WAV byte-identical to the pre-refactor fold fixture) and ABI **v13 by ADDING fields only**; L2 DiT forward; L3 Gemma-4 TE reusing `gemma4.cpp`; L4 Conv video VAE + audio VAE + vocoder; L5 pipeline/recipes/upsampler/duration head; L6 NVFP4 arms + GB10 load-time residency; L7 e2e on dgx.casa under `flock $HOME/gpu.lock`. **OPEN, same as H3:** there is no vllm-omni parity PIN — `.agents/upstream-sync.md` covers the vLLM repo only, so each golden records the vllm-omni SHA inline. | `CLAIM-LTX25-L0-L7` |
+| `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model` | `LTX2VideoTransformer3DModel` | **BEYOND-PIN AND OUT-OF-REPO** — not in `555967922` (LTX-2.5 released 2026-08) and not in the vLLM repository at all. Architecture reference is Lightricks' own `LTX-2` (`packages/ltx-core/src/ltx_core/`: `model/transformer/{transformer,model,attention,rope,model_configurator}.py`, `model/audio_vae/`, `components/`, `guidance/`). vLLM-Omni carries an `ltx2` module but its recipes stop at 2.3 (`vllm_omni/diffusion/models/ltx2/ltx2_recipes.py:162-166`); 2.5 is upstream-OPEN at [vllm-omni#6066](https://github.com/vllm-project/vllm-omni/issues/6066) and [#4985](https://github.com/vllm-project/vllm-omni/issues/4985). Serving oracle reaches 2.5 through the GENERIC `DiffusersAdapterPipeline` (`pipeline_diffusers_adapter.py:116`) against `Lightricks/LTX-2.5-Diffusers`. | diffusion generation / video + audio (text/image/video in, video+audio out) | flow-matching denoise loop; TWO coupled modality streams with audio<->video cross-attention; per-head gated attention; AdaLN-single with 9-vector per-block tables; timestep-CONDITIONED prompt K/V via `prompt_adaln_single` (model.py:223-227, transformer.py:441-446), so the K/V cache is inapplicable on the shipped DiT -- the row's first claim of cacheability was RETRACTED 2026-08-12 and the module PORTED 2026-08-13 (#644, [spec](specs/ltx25-prompt-adaln.md)); on the real weights the timestep term DOMINATES the static table, RMS ratio 1347% video / 1583% audio; split/interleaved 3D RoPE; gelu-approximate FFN; Gemma-4 12B text encoder with dual caption projections (4096 video / 2048 audio); Conv video VAE + DiffVAE; audio VAE + vocoder; latent spatial/temporal x2 upsamplers; duration head; distilled two-stage recipe | [ltx-2.5](specs/ltx-2-5.md) | `SPIKE` | **L0 LANDED 2026-08-11 (#435).** Spec committed BEFORE implementation per AGENTS.md. Geometry measured from the ungated `vonkaiser/LTX-2.5-FP8-NVFP4` FP8 checkpoint header by range request — the same no-payload technique used for H3's manifests: 21.00B params, 48 blocks, 1775 F8_E4M3 tensors each with a per-tensor F32 `weight_scale`, biases/norms BF16. Cross-modal projection shapes are ASYMMETRIC and are a named test trap: `audio_to_video_attn.to_q` is [2048, 4096] while `to_k`/`to_v` are [2048, 2048] and `to_out` is [4096, 2048], so a transposition still type-checks against a square assumption. **CHECKPOINT ACCESS (verified against the HF API 2026-08-11):** `Lightricks/LTX-2.5` is `gated: auto` (accepting the license opens it; holds the first-party NVFP4 DiT at 18.72 GB), `Lightricks/LTX-2.5-Diffusers` is RESTRICTED behind manual approval (needed for the binding oracle), and `vonkaiser/LTX-2.5-FP8-NVFP4` is UNGATED and unblocks L1-L2 today. All artifacts land under `$CHECKPOINT_ROOT` (`/mnt/nas_share/checkpoints`) so dgx.casa and the cluster nodes share one copy. **PHASES (one PR, developer-directed):** L1 `vllm::multimodal::VideoEngine` interface + checkpoint-detected registry with MiniMax-H3 moved behind it UNCHANGED (gated on frames+WAV byte-identical to the pre-refactor fold fixture) and ABI **v13 by ADDING fields only**; L2 DiT forward; L3 Gemma-4 TE reusing `gemma4.cpp`; L4 Conv video VAE + audio VAE + vocoder; L5 pipeline/recipes/upsampler/duration head; L6 NVFP4 arms + GB10 load-time residency; L7 e2e on dgx.casa under `flock $HOME/gpu.lock`. **OPEN, same as H3:** there is no vllm-omni parity PIN — `.agents/upstream-sync.md` covers the vLLM repo only, so each golden records the vllm-omni SHA inline. | `CLAIM-LTX25-L0-L7` |
 | `MODEL-MM-moss-transcribe-diarize-moss-transcribe-diarize-for-conditional-generation` | `MossTranscribeDiarizeForConditionalGeneration` (v0.25.0 target-pending) | v0.25.0 target `registry.py:450-453`; `vllm/model_executor/models/moss_transcribe_diarize.py::MossTranscribeDiarizeForConditionalGeneration` @ `702f481` | conditional generation / audio | MM processor; Whisper encoder; VQ adaptor; Qwen3 decoder; speech-to-text/diarization frontend | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-lightonocr-light-on-ocrfor-conditional-generation` | `LightOnOCRForConditionalGeneration` | `registry.py:450-453`; `vllm/model_executor/models/lightonocr.py::LightOnOCRForConditionalGeneration` | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-lfm2-vl-lfm2-vlfor-conditional-generation` | `Lfm2VlForConditionalGeneration` | `registry.py:454`; `vllm/model_executor/models/lfm2_vl.py::Lfm2VLForConditionalGeneration` | conditional generation / image | MM processor; encoder/merge; Mamba/SSM state; vision encoder; video path | ☐ required | `INVENTORIED` | none | unassigned |

.agents/roadmap_v1.md

@@ -151,6 +151,7 @@ issue is not yet placed. Keyed record: update in place, never append.
 | [#560](https://github.com/mudler/vllm.cpp/issues/560) | `ROAD-V1-LTX25` | Stabilizing constants unreachable by any gate: FIVE instances found by sweep, incl. the DiT's own `norm_eps` field default | bug |
 | [#567](https://github.com/mudler/vllm.cpp/issues/567) | `ROAD-V1-LTX25` | `OpNameImpl` makes a SECOND file every new `OpId` must edit; collapse the enum and its names into one derived list | bug |
 | [#644](https://github.com/mudler/vllm.cpp/issues/644) | `ROAD-V1-LTX25` | LTX-2.5 FULL PORT campaign. Row 0 `LTX25-PROMPT-ADALN` (spec [`ltx25-prompt-adaln.md`](specs/ltx25-prompt-adaln.md)): `use_prompt_adaln_single` defaults TRUE in BOTH references and the shipped DiT carries its 18 tensors, but `ltx2_loader.cpp:988` cleared the flag unconditionally, so every render dropped the timestep-conditioned half of the prompt K/V modulation — invisible to every shape and finiteness check | feature |
+| [#673](https://github.com/mudler/vllm.cpp/issues/673) | `ROAD-V1-LTX25` | #644's checkpoint-derived evidence is MANUAL and host-local: `LTX2_CHECKPOINT_ROOT` is set by no workflow (`grep -rn CHECKPOINT_ROOT .github/` exits 1, positive control in `tests/`), so CI runs **502 of 8734 assertions — 5.7%** of `test_ltx2_video` at an unchanged case count of 30, and `scripts/measure-ltx2-prompt-adaln.py` is a manual tool no gate invokes. Recorded as visible debt; the skip itself is already announced in four places and that disposition stands | bug |
 | [#615](https://github.com/mudler/vllm.cpp/issues/615) | `GATE-PR-SIZE-BINARY` | `check-pr-size` fail-closes on every binary path with no exemption route, so no golden-bearing PR can merge: it blocks #431 and post-dates the golden precedent it rejects, spec [`gate-pr-size-binary.md`](specs/gate-pr-size-binary.md) | bug |
 | [#238](https://github.com/mudler/vllm.cpp/issues/238) | `SAMPLE-LOGPROB-TOKEN-IDS` | `logprobs_mode`: three of four modes are runtime-refused stubs | bug |
 | [#264](https://github.com/mudler/vllm.cpp/issues/264) | `SAMPLE-LOGPROB-TOKEN-IDS` | `logprob_token_ids`: generative scoring over an explicit token set is unported | feature |