diff --git a/.agents/NOW.md b/.agents/NOW.md index 8a23d813d..4a6706e1a 100644 --- a/.agents/NOW.md +++ b/.agents/NOW.md @@ -38,8 +38,7 @@ both gate models, reproduced 2–3x on an idle box. See [gates.md](gates.md) and (0.26.0.dev0). Method rules hardened (AGENTS.md): the STRUCTURAL lens (same kernel, different -throughput ⇒ audit the context; scan the REFERENCE's own rationale; per-shape -MEASUREMENT arbitrates; distrust aggregate bytes/time and CROSS-TOOL comparisons). +throughput ⇒ audit the context; per-shape MEASUREMENT arbitrates). ## Next actions @@ -52,19 +51,20 @@ MEASUREMENT arbitrates; distrust aggregate bytes/time and CROSS-TOOL comparisons f32-out caller) once the Laguna fix proves the mechanism. 4. **Restore `local-ai-worker`** on dgx when the GPU campaign ends (`docker update --restart=always` + `docker start`). -5. **Protocol substrate — partly done.** Claim triage DONE; `docs/STATUS.md` - under a shrink-only ratchet; roadmap compacted; `AGENTS.md` tiered. REMAINING: - anchor backfill (98 rows `SPIKE`/`ACTIVE`, need code/test anchors; 6 model rows - need a DECISION, architecture unregistered); record-era rollover BLOCKED on - `check-agent-record.py` binding `DONE` rows to `parity-ledger.md` LINE anchors - (re-anchor by ROW ID first; `state.md`/`benchmark-record.md` can roll now). +5. **Protocol substrate — partly done.** Claim triage + live-state audit DONE + (10 unevidenced rows → `READY`, 11 claims retired, 9 amended); `STATUS.md` + ratcheted; roadmap compacted; `AGENTS.md` tiered. REMAINING: anchor backfill + (6 model rows need a DECISION); record-era rollover BLOCKED on `DONE` rows + bound to `parity-ledger.md` LINE anchors (re-anchor by ROW ID). + ★ The gate SELF-BLINDS on those same 10 (audit §➁a); its fix owes an 8-row + adjudication. workflow.md now states the `ACTIVE` precondition. **Operator/helper protocol** ([spec](specs/operator-helper-protocol.md)): roles DECLARED then MATERIALIZED into a lock or worktree+PR; operator merges PRs first and does features only via sub-agents; helpers use worktrees on `row/` and open a DRAFT PR at the START, which IS the claim. **W0-W5 LANDED**; role discipline ENFORCING, -`--require-role` still opt-in. Queue: 4 rows. Backfill: 79 rows, 30 anchored; blocker is claim FAMILIES. +`--require-role` still opt-in. Queue: 10 rows — 6 are audit-vacated, with LANDED gate anchors; READ before picking. Backfill: 79 rows, 30 anchored; blocker is claim FAMILIES. **Upstream inventory** ([spec](specs/upstream-derived-inventory-2026-08-05.md), drift-gated, arch parity BOTH ways): SM060/061/070 below vLLM's floor = OUT-OF-SCOPE; COMP-*/DISTRIBUTED-* are REAL unported work; **all 362 archs now have rows**; llama.cpp's 11 extra devices are IN SCOPE, spike-gated diff --git a/.agents/backend-matrix.md b/.agents/backend-matrix.md index 26799c4d2..e8da7ae62 100644 --- a/.agents/backend-matrix.md +++ b/.agents/backend-matrix.md @@ -270,7 +270,7 @@ the rest are `SPIKE`. | ID | Item | Upstream | Our code | Tests/evidence | Spike/spec | State | Owner | |---|---|---|---|---|---|---|---| | `BACKEND-DISTRIBUTED-COMM` | The unifying `vt::Communicator` / process-group abstraction — rank/world_size + AllReduce(sum/max/min/prod)/AllGather/Send/Recv, stream-ordered (each takes a `Queue&`). **W1 LANDED**: abstraction (`include/vt/communicator.h`) + a CPU in-process multi-rank transport (`src/vt/communicator.cpp`, N ranks = N host threads over one barrier+staging+mailbox) proven by `tests/vt/test_communicator.cpp` (2/4-rank AllReduce-sum + AllGather exact on every rank, Send/Recv rendezvous, RED-verified; 8 cases/50 assertions). `world_size==1` ⇒ every collective a byte-identical no-op (asserted). **W2 LANDED**: collectives now ROUTE through `OpProvider`/`OpId` (`kAllReduce`/`kAllGather`/`kSend`/`kRecv`, keyed on the queue's DeviceType) — the CPU in-process reduce registered on kCPU (`test_communicator` still 50/50 through the OpId path), the NCCL provider on kCUDA. W2+ residuals: RDMA/TCP (Spark), MLX-ring (kMETAL) transports | vLLM `device_communicators/base_device_communicator.py:147` (DeviceCommunicatorBase interface, the port template) + `distributed/parallel_state.py:358` (GroupCoordinator dispatch; world_size==1 bypass :638) | LANDED: `include/vt/communicator.h` + `src/vt/communicator.cpp` (sibling of `vt::Queue` `include/vt/device.h:50`); OpId routing via `include/vt/op_provider.h:108` (`OpId::kAllReduce/…`); stream-order hooks reused `include/vt/backend.h:87-104` | CPU exact-gate (`test_communicator`, 50/50, via OpId path) | [scale-out spike](specs/scale-out-distributed.md) | `ACTIVE` | `CLAIM-SCALE-OUT-W2` | -| `BACKEND-DISTRIBUTED-TP` | Tensor parallel (intra-node multi-GPU) — sharded Column/Row/QKV linears, vocab-parallel embed + LM head, attention-head split, MoE expert-parallel; all-reduce after o_proj/MLP-down and the EP combine. **W2 LANDED (CPU-gated)**: the `TensorParallel`/`TpShard`/`TpAllReduceSum` wiring (`include/vllm/model_executor/models/tensor_parallel.h`) threaded into the Qwen3-dense forward (o_proj all-reduce `dense_attn_block.h`, MLP-down `qwen3.cpp`) + the MergedColumn shard at the loader chokepoint (`dense_weight_loaders.h`), proven by `tests/vt/test_tp_forward.cpp` — sharded-matmul + RowParallel all-reduce **== the unsharded tp=1 forward** over the W1 CPU communicator (RED-verified: dropping the all-reduce fails 24 assertions). `tp_size==1`/nullptr ⇒ every helper a byte-identical no-op (asserted). RESIDUAL (HW-gated, no ≥2-GPU box): QKV head-aware KV replication, vocab/LM-head + MoE-EP sharding, and a real TP-2 GPU forward | vLLM `layers/linear.py:418` (Column, out-dim shard) / `:1612` (Row, all-reduce :1766) / `:1021` (QKV heads :1074) + `vocab_parallel_embedding.py:198` + `fused_moe/expert_map_manager.py:22` | LANDED `include/vllm/model_executor/models/tensor_parallel.h`; seams `dense_attn_block.h` (o_proj all-reduce) + `qwen3.cpp` (MLP-down); weight chokepoint `dense_weight_loaders.h:131` (column shard) | CPU multi-rank TP gate (`test_tp_forward`, 60/60, RED-verified) | [scale-out spike](specs/scale-out-distributed.md) | `ACTIVE` | `CLAIM-SCALE-OUT-W2` | +| `BACKEND-DISTRIBUTED-TP` | Tensor parallel (intra-node multi-GPU) — sharded Column/Row/QKV linears, vocab-parallel embed + LM head, attention-head split, MoE expert-parallel; all-reduce after o_proj/MLP-down and the EP combine. **W2 LANDED (CPU-gated)**: the `TensorParallel`/`TpShard`/`TpAllReduceSum` wiring (`include/vllm/model_executor/models/tensor_parallel.h`) threaded into the Qwen3-dense forward (o_proj all-reduce `dense_attn_block.h`, MLP-down `qwen3.cpp`) + the MergedColumn shard at the loader chokepoint (`dense_weight_loaders.h`), proven by `tests/vt/test_tp_forward.cpp` — sharded-matmul + RowParallel all-reduce **== the unsharded tp=1 forward** over the W1 CPU communicator (RED-verified: dropping the all-reduce fails 24 assertions). `tp_size==1`/nullptr ⇒ every helper a byte-identical no-op (asserted). RESIDUAL (HW-gated, no ≥2-GPU box): QKV head-aware KV replication, vocab/LM-head + MoE-EP sharding, and a real TP-2 GPU forward | vLLM `layers/linear.py:418` (Column, out-dim shard) / `:1612` (Row, all-reduce :1766) / `:1021` (QKV heads :1074) + `vocab_parallel_embedding.py:198` + `fused_moe/expert_map_manager.py:22` | LANDED `include/vllm/model_executor/models/tensor_parallel.h`; seams `dense_attn_block.h` (o_proj all-reduce) + `qwen3.cpp` (MLP-down); weight chokepoint `dense_weight_loaders.h:131` (column shard) | CPU multi-rank TP gate (`test_tp_forward`, 60/60, RED-verified) | [scale-out spike](specs/scale-out-distributed.md) | `READY` | - | | `BACKEND-DISTRIBUTED-PP` | Pipeline parallel — PP stage split (`PPMissingLayer` analogue) + inter-stage `IntermediateTensors` send/recv over the comm layer + multi-worker executor fan-out | vLLM `models/utils.py:785` (PPMissingLayer) / `:798` (make_layers) + `distributed/utils.py:127` (get_pp_indices) + `parallel_state.py:957` (send_tensor_dict) | fan-out seam `src/vllm/v1/executor/executor.cpp:7-34` (direct single-worker call today) | - | [scale-out spike](specs/scale-out-distributed.md) | `SPIKE` | `CLAIM-SCALE-OUT-SPIKE` | | `BACKEND-DISTRIBUTED-DP` | Data parallel — N independent engine replicas over the SAME weights + a DP coordinator (global "request wave" so all DP ranks step together) + a per-step token-count all-reduce; DP×EP is the large-scale DeepSeek serving topology (DP-replicated attention + EP-sharded experts). NOT part of `world_size` (DP is outside: `world_size_across_dp = world_size × DP`) | vLLM `v1/engine/coordinator.py:23` (`DPCoordinator`, wave :33-56) + `v1/worker/dp_utils.py:164` (`coordinate_batch_across_dp`; per-step `num_tokens_across_dp` all-reduce :53) + group `distributed/parallel_state.py:1866` + flags `config/parallel.py:129-145` | reuses W1 `Communicator::AllReduce` for the token-count sync; NEW engine-replica executor + coordinator; **depends on the multi-worker executor (`executor.cpp:7-34`)** | - | [parallelism-modes spike](specs/parallelism-modes.md) | `SPIKE` | `CLAIM-PARALLELISM-MODES-SPIKE` | | `BACKEND-DISTRIBUTED-EP` | Expert parallel — each rank owns a DISJOINT subset of WHOLE experts (vs TP's every-rank-all-experts intermediate-dim shard); all-to-all dispatch (tokens→expert-owner) + all-to-all combine. EP group = DP×PCP×TP ranks (MoE-only), a re-grouping NOT a new `world_size` knob; EPLB is a separate same-ranks group. DeepEP HT/LL/V2 all-to-all backends present upstream | vLLM `fused_moe/expert_map_manager.py:22` (`determine_expert_map`, local_num_experts :69) + `use_ep` `fused_moe/config.py:1204` + EP group `distributed/parallel_state.py:1892` + all-to-all backends `fused_moe/all2all_utils.py:44` (`prepare_finalize/deepep_{ht,ll,v2}.py`); flags `--enable-expert-parallel` `config/parallel.py:165` + `--all2all-backend` `:188` | NEW `OpId::kAllToAll` (dispatch/combine) on the same `vt::Communicator` + the expert-subset loader (`qwen3_moe_weights.cpp:35-38` per-expert loop); `vt::MoeCombine` (`include/vt/ops.h:104`) becomes the combine | - | [parallelism-modes spike](specs/parallelism-modes.md) | `SPIKE` | `CLAIM-PARALLELISM-MODES-SPIKE` | diff --git a/.agents/coordination.md b/.agents/coordination.md index 672bc58fe..64982d198 100644 --- a/.agents/coordination.md +++ b/.agents/coordination.md @@ -1399,7 +1399,7 @@ table, tests, CMake. Details in the state-log entry of the same date. | `CLAIM-DEEPSEEK-V4-W6` | `KERNEL-MOE-SQRTSOFTPLUS-HASH` (NEW row, under `CLAIM-DEEPSEEK-V4-IMPL` — does NOT own a new model-matrix row; the `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` row stays owned by `CLAIM-DEEPSEEK-V4-IMPL`, still `SPIKE`) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w6-moe` (CPU-only; NO GPU/download; foreground; NOT pushed) | worktree branch `deepseek-v4-w6-moe`, base `main` `5b843be5` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W6 — the sqrtsoftplus + hash-routed MoE, as portable host references + unit gate. Owns the additive TUs `deepseek_v4_moe.{h,cpp}` + `tests/vllm/models/test_deepseek_v4_moe.cpp` + the new `KERNEL-MOE-SQRTSOFTPLUS-HASH` kernel-matrix row (+ its `check-agent-record.py` count bump 41→42) + the W6 section of `.agents/specs/deepseek-v4-flash.md` + the record surfaces. SACRED-inert: additive ONLY — does NOT touch any existing forward (the shared DeepSeek-V2 MoE router / grouped GEMM / shared experts + the W3/W4/W5 DSA/compressor/MHC TUs are untouched beyond additive) and does NOT touch the GGUF/quant TUs owned by the `CLAIM-DSV4-GGUF-*` lane; only the NEW scoring + hash-route + clamp are net-new (the shared grouped-GEMM machinery is REUSED, not re-ported); the device kernels + forward assembly are NAMED W7 residuals. | `SPIKE` | 2026-07-29 — **W6 LANDED (foreground, NOT pushed).** Ported + unit-gated the three genuinely-new-vs-V2/V3 MoE primitives as portable host references: `SqrtSoftplus` (the V4 router score `sqrt(softplus(x))`), `SqrtSoftplusRouteTopk` (score→bias-for-selection-only top-k OR `tid2eid` hash bypass→GATHER weights from UNBIASED scores→renormalize→×routed_scaling_factor), `ClampedSwiGLU` (`SiluAndMulWithClamp`: gate max-clamp, up ±clamp, `gate·σ(αgate)·(up+β)`). Ported 1:1 from vLLM `fused_topk_bias_router.py:75-118` (`_topk_softplus_sqrt_torch`) + `activation.py:197-201`, cross-checked SGLang `v0.5.15` `moe/{topk.py, hash_topk.py}`. Gate `test_deepseek_v4_moe` **12/12·716** — hand-derived literals (sqrt∘softplus composition softplus=4⇒score=2; bias flips selection but weight stays UNBIASED 1.0 not 3.0; renormalize by the unbiased sum; routed_scaling_factor; hash picks {3,1} where top-k would pick {2,0}; asymmetric clamp gate=-5 kept vs up clamped to -2; clamp boundaries; alpha/beta) + from-first-principles DOUBLE-PRECISION references (router f32==f64 rel-L2 < 1e-5 + exact ids; SqrtSoftplus f64 + monotonicity; ClampedSwiGLU rel-L2 < 1e-6). **RED-first PROVEN all three levers:** drop the sqrt → 8 cases/493 assertions fail; gather weights from the BIASED scores → 2 cases/181 fail; symmetric-clamp the gate → 2 cases/6 fail; revert restores 12/12·716. Honest gate form: host-reference + hand-case + structural review vs vLLM+SGLang `file:line` (fixed-config 167B not constructible at a tiny shape ⇒ NOT a dumped-oracle rel-L2). No OPEN QUESTIONS — every constant grounded (MegaMoE SM100-only is NOT the GB10 target; the FusedMoE-fallback router GB10 runs is mirrored). CPU Debug full-library build (the pre-existing GCC-13 `-O2` `-Werror=array-bounds` false positive in `voxtral.cpp` forces Debug; the new TUs are `-Wall -Werror -Wextra`-clean, verified by explicit strict compile). SACRED inertness PROVEN: no existing forward touched; `test_deepseek_v4_mhc` still 14/14·125, `test_deepseek_v4_compressor` 12/12·164, `test_deepseek_v4_dsa` 13/13·38, `test_deepseek_v4_scaffold` 4/4·40. New kernel row `KERNEL-MOE-SQRTSOFTPLUS-HASH` (`SPIKE`). Residuals: device kernels (reuse the existing grouped-GEMM) + `DeepseekV4Model::Forward` assembly (W7), strict/near-tie engine gate (W8) — all multi-Spark. | | `CLAIM-DEEPSEEK-V4-W7` | `deepseek_v4.{h,cpp}` forward (owns the FORWARD TUs via `CLAIM-DEEPSEEK-V4-W7`; does NOT own a new model-matrix row — `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` stays owned by `CLAIM-DEEPSEEK-V4-IMPL`, still `SPIKE`; advanced to note the W7 forward-assembly landed) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w7-forward` (CPU-only `build-cpu` Debug; NO GPU/download; foreground; NOT pushed) | worktree branch `deepseek-v4-w7-forward`, base `main` `a856383c` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W7 — the `DeepseekV4Model::Forward` ASSEMBLY: compose the four landed host-reference primitives (W3 DSA/MLA seams, W4 compressor + fp8_ds_mla KV, W5 MHC + Sinkhorn, W6 sqrtsoftplus/hash MoE) into a REAL forward at a tiny synthetic config, replacing the `VT_CHECK(false)` stub. Owns `deepseek_v4.{h,cpp}` (the forward + the host-float weight tower + the RED-first miswire enum + the structural trace) + `tests/vllm/models/test_deepseek_v4_forward.cpp` + the `tests/CMakeLists.txt` test wiring + the W7 section of `.agents/specs/deepseek-v4-flash.md` + the record surfaces. SACRED-inert: additive to the V4-only path ONLY — does NOT touch the shared MLA/MoE (`mla_attention.{h,cpp}`, `cuda_mla_attn.cu`, the DeepSeek-V2 MoE), README, Metal, SACRED, apex, darwin, the W3-W6 primitive TUs (beyond calling them), or the GGUF/quant TUs (`CLAIM-DSV4-GGUF-*`). No new kernel-matrix row (a forward assembly, not a new kernel family) ⇒ no `check-agent-record.py` count bump. | `SPIKE` | 2026-07-29 — **W7 LANDED (foreground, NOT pushed).** Replaced the `VT_CHECK(false, "W3-W8 pending")` stub with a REAL `DeepseekV4Model::Forward` + `DeepseekV4ForwardHost` that composes the four landed host primitives on the portable CPU path at a SMALL synthetic config, producing logits end-to-end. Interleave grounded 1:1 (`nvidia/model.py:1080-1148` model.forward + `:866-957` decoder.forward): embed → per layer [first-layer MHC-pre stream EXPAND `[T,H]→[T,hc,H]`, else fused MhcPost(prev-ffn)+MhcPre(attn)] → 512-wide MLA (q/kv proj+norms, RoPE, DSA indexer→topk→compressor→fp8_ds_mla KV round-trip, sink softmax, grouped o-LoRA) → fused MhcPost(attn)+MhcPre(ffn) → MoE (sqrtsoftplus/hash router + shared+routed clamped-SwiGLU) → final MhcPost → hc_head collapse → norm → lm_head. Gate `test_deepseek_v4_forward` **6/6·26** (STRUCTURAL/composition, tiny shape): finite logits end-to-end + deterministic + shape `[T,vocab]`; MHC stream `[T,hc,H]` (residual_stream_elems==T*4*H, hc_mult==4); hash layers route by `tid2eid` (`layer_hash_routed=={1,1,0,0}`) vs gated learned top-k; DSA indexer SELECTS (layers 1,3, index_topk=3) + compressor POOLS (layers 1,2,3); `logits_indices` gather. **RED-first PROVEN 3 levers** (each changes the output): route hash layers as gated (ignore tid2eid), skip the final MhcPost fold, drop the per-head attention sink. Honest 3-state: the CPU forward assembly at tiny shape = DERIVED + BUILD-VERIFIED (structural) — does NOT claim V4 "runs" a real model; documented tiny-vs-167B divergences (W=2 compressor window, full-latent MLA value, single rope_theta, 1-block quant). SACRED-inert PROVEN: only `deepseek_v4.{h,cpp}` + the new test + CMake changed; the shared MLA/MoE + W3-W6 TUs empty-diff; prior V4 tests unchanged (scaffold 4/40, dsa 13/38, compressor 12/164, mhc 14/125, moe 12/716). CPU Debug full-library build (the voxtral `-O2` `-Werror=array-bounds` false positive forces Debug; the new TU is `-Wall -Werror -Wextra`-clean). Residuals: W7-device CUDA kernels (MHC Sinkhorn, DSA indexer/compressor, sqrtsoftplus router, clamped SwiGLU; expert GEMM REUSES NVFP4/FP8 grouped-GEMM) + `ForwardDevice`; W2b real-checkpoint tower materialization; W8 strict/near-tie engine gate (multi-Spark, 156.7 GiB); the single-Spark IQ2_XXS-GGUF vehicle additionally needs the GGUF `blk.N.*` name-map (W2, download-blocked on the 1328-tensor manifest). | | `CLAIM-DEEPSEEK-V4-W7-DEVICE` | `KERNEL-DSV4-W7-DEVICE` (NEW row, under `CLAIM-DEEPSEEK-V4-IMPL` — does NOT own a new model-matrix row; the `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` row stays owned by `CLAIM-DEEPSEEK-V4-IMPL`, still `SPIKE`) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w7-device` (CPU `-Werror` build-verify + **DGX GB10 CUDA gate under `flock /tmp/gpu`**; `docker stop local-ai-worker` for the run then restored `--restart=always`+start; foreground; NOT pushed) | worktree branch `deepseek-v4-w7-device`, base `main` `33016f34` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W7-DEVICE — the CUDA kernels for the four NEW V4 op families + `ForwardDevice` wiring. Owns the NEW additive TUs `src/vt/cuda/cuda_deepseek_v4.cu` + `include/vllm/model_executor/models/deepseek_v4_device.h` + `src/vllm/model_executor/models/deepseek_v4_device.cpp` + `tests/vllm/models/test_cuda_deepseek_v4.cpp`, the `ForwardDevice` wiring in `deepseek_v4.{cpp,h}` (the `V4Backend` policy that runs the ONE composition on host refs OR the device kernels), the 4 new `kDeepseekV4*` OpIds in `include/vt/ops.h`, the `KERNEL-DSV4-W7-DEVICE` kernel-matrix row (+ its `check-agent-record.py` count bump 42→43), the CMake source/test wiring, the W7-device section of `.agents/specs/deepseek-v4-flash.md`, and the record surfaces. Also neutralized the pre-existing GCC-13 `-O2` `-Werror=array-bounds`/`-Wstringop-overflow` FALSE POSITIVE in `voxtral.cpp` (project #155) with a MINIMAL LOCAL scoped `#pragma GCC diagnostic` around the in-bounds `BuildPaddedDecodeAttn` copies so the `-Werror` build links (advances #155). SACRED-inert: does NOT touch the shared MLA/MoE (`mla_attention.{h,cpp}`, `cuda_mla_attn.cu`, `cuda_moe*.cu` — the 512-wide MLA attn + expert grouped-GEMM REUSE the existing NVFP4/FP8 kernels), the W3-W6 host-ref TUs (beyond calling them via the seam), README, Metal, SACRED, apex, darwin. | `SPIKE` | 2026-07-29 — **W7-DEVICE LANDED + DGX-GATED (foreground, NOT pushed).** The four families' CUDA kernels (MHC Sinkhorn/pre/post/head; DSA indexer weight-fold/MQA-ReLU-logits/causal-topk + sink softmax + grouped output-LoRA; compressor pool+norm + save-APE + fp8_ds_mla KV encode/decode; sqrtsoftplus/hash router + clamped SwiGLU), each a 1:1 device port of the landed host reference, registered through the OpProvider seam (`kDeepseekV4{Mhc,Dsa,Compressor,Moe}`) and dispatched by `DeepseekV4Model::ForwardDevice`. **DGX GB10 (sm_121a) UNIT GATE `test_cuda_deepseek_v4` 11/11 cases · 153 assertions GREEN** — BIT-EXACT ids (DSA causal top-k + router selection), `-inf` mask exact, near-tie rel-L2 < 1e-4 for the fp reductions (device `expf`/`sqrtf`/`rsqrt` vs host), fp8_ds_mla round-trip within e4m3 granularity + bf16 rope bit-exact; PLUS the ForwardDevice composition gate (device forward == host forward, rel-L2 < 2e-3). **compute-sanitizer memcheck 0 errors.** **RED-first PROVEN:** dropping the device sqrtsoftplus sqrt fails 3 cases/6 assertions; revert restores 11/11·153. CUDA `-Werror` clean; CPU `-Werror` clean (voxtral #155 FP neutralized). SACRED-inert PROVEN: shared MLA/MoE CUDA + W3-W6 host TUs empty-diff; host oracle `test_deepseek_v4_forward` still 6/6·26 + prior V4 tests unchanged. Honest 3-state: kernels RUNTIME-VERIFIED at small shape on real GB10; the real-checkpoint paged-engine e2e stays W8 (156.7 GiB NVFP4 does not fit ONE GB10) and the single-Spark IQ2_XXS-GGUF vehicle also needs the W2 `blk.N.*` name-map. | -| `CLAIM-DEEPSEEK-V4-W8` | `KERNEL-QUANT-CIQ-IQUANT` (NEW kernel-matrix row) `QUANT-GGUF-IQ2_XXS` `QUANT-GGUF-IQ3_XXS` (NEW) `QUANT-GGUF-Q2_K` (all advanced to keep-quant compute, ownership transferred from `CLAIM-DSV4-GGUF-LOADER`); does NOT own a new model-matrix row — `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` stays owned by `CLAIM-DEEPSEEK-V4-IMPL`, still `SPIKE` | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w8-run` (CPU-only `build-cpu` Debug gate; DGX SSH used ONLY read-only to confirm host/disk/mem + HF HTTP-range header fetch — NO 91 GiB download, NO GPU run this lane; foreground; NOT pushed) | worktree branch `deepseek-v4-w8-run`, base `main` `4d618f59` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W8 — the single-Spark GGUF memory enabler + name-map. (1) Keep-quant `vec_dot` for IQ2_XXS/IQ3_XXS/Q2_K (`KERNEL-QUANT-CIQ-IQUANT`): owns the NEW `src/vt/cpu/cpu_quant_iq_tables.h` (shared grids) + the three `VecDot*` kernels in `cpu_quant_dot.cpp` + `DequantIQ3_XXS` in `cpu_quant_dequant.cpp` + three Q8_K traits rows in `cpu_quant_traits.cpp` + `kIQ3_XXS` in `dtype.{h,cpp}` + the three block structs in `cpu_quant_blocks.h` + `kIQ3_XXS` in the two exhaustive `DType` switches (`ops.cpp`) + ggml id-18 sizing in `gguf_reader.cpp` + the `kWeightCases` extension in `tests/vt/test_ops_quant_dot.cpp`. (2) The GGUF `blk.N.*`→V4 name map + FULL 1328-tensor coverage gate: NEW `scripts/check-dsv4-gguf-namemap.py` + `scripts/dsv4_gguf_manifest_names.txt` (the real manifest, HTTP-range-fetched). (3) A precise honest registry message advance in `deepseek_v4_registry.cpp`. Owns the `KERNEL-QUANT-CIQ-IQUANT` kernel-matrix row (+ `check-agent-record.py` count 43→44), the quantization-matrix IQ rows, the W8 section of `.agents/specs/deepseek-v4-flash.md`, and the record surfaces. SACRED-inert: additive to the vt quant tier + V4 GGUF path ONLY — does NOT touch the six existing k-quant kernels' numerics (their gate is UNCHANGED at 130426 of the 130444 assertions), the shared MLA/MoE, README, Metal, SACRED, apex, darwin. | `SPIKE` | 2026-07-29 — **W8 KEEP-QUANT ENABLER + NAME-MAP LANDED (foreground, NOT pushed); the RUN is the honest residual.** (1) Keep-quant `vec_dot` for IQ2_XXS/IQ3_XXS/Q2_K, 1:1 ports of ggml `vec_dot_{q2_K,iq2_xxs,iq3_xxs}_q8_K_generic` (`quants.c:514/855/999`) → `HasQuantDotKernel` TRUE ⇒ the loader keeps these blocks COMPRESSED (the ~91 GiB vs ~316 GiB bf16 OOM enabler). **HONEST CORRECTION of the brief's "IQ2_XXS + Q2_K":** the real `UD-IQ2_XXS` manifest uses IQ2_XXS (gate/up experts) + **IQ3_XXS** (down experts); Q2_K is the sibling `UD-Q2_K_XL` vehicle — all three landed so both vehicles keep-quant. **HONEST finding:** there is NO CUDA keep-quant vec_dot for ANY k-quant (`kMatmulBTQuant` is `kCPU`-only) — on GB10 keep-quant runs on the 20 ARM cores against the unified pool, so IQ2_XXS/IQ3_XXS/Q2_K match the six existing types (CPU tier). Gate `test_ops_quant_dot` **19 cases / 130444 assertions GREEN** (CPU Debug, new TUs `-Werror` clean): vec_dot vs INDEPENDENT f64 dequant-dot (≤1e-5·L1), NMSE ≤5e-4 vs dequant-f32 GEMM, bit-exact across threads, ragged-K reject; **RED-first PROVEN** (perturb IQ2_XXS `0.125` fold → 2 cases/18 fail, revert → 19/130444). (2) `blk.N.*`→V4 name map with **EXACT 1328/1328 coverage, 0 unmapped, 0 leftover** (`check-dsv4-gguf-namemap.py` rc=0) against the REAL manifest fetched via HF HTTP-range (no 91 GiB download); topology verified (43 layers, hash{0,1,2}, indexer 21 = compress_ratio==4, compressor 41). (3) Registry reject replaced with a precise "keep-quant+name-map landed; W2b tower materialization pending" message. **HONEST RESIDUAL — the RUN did NOT execute** and was NOT faked: it is blocked on the unimplemented **W2b** (materialize the keep-quant blocks + F32 MHC/DSA tensors into the DeepseekV4 towers via the name map) — a genuine code brick, not just memory/download. Resume: land W2b → free DGX disk ≥100 GiB → `flock $HOME/gpu.lock` + `docker stop local-ai-worker` → download `unsloth/DeepSeek-V4-Flash-GGUF/UD-IQ2_XXS` (3 shards, ~91 GB) → keep-quant load → `ForwardDevice` greedy → self-consistency + coherence gate. **W8-FINAL UPDATE 2026-07-29 (base `376e186b`, W2b now landed): the top-level `deepseek4` GGUF ENTRYPOINT ARM is LANDED + GATED, and the real RUN is re-scoped to a CODE blocker.** Wiring: `DeepseekV4HfConfigFromGguf` (`deepseek_v4_weights.cpp`) maps `general.architecture=deepseek4`→the registered `DeepseekV4ForCausalLM` (republishing geometry into `config.raw` for `ParseDeepseekV4Config`); `LoadedEngine::FromModelDir` routes via new anon-ns `HfConfigFromGgufDispatch`. Gate `test_deepseek_v4_gguf_load` **6/6·168** (CPU Release `-Werror`-clean; +1 case: config maps + `ModelRegistry::Resolve`→V4 factory), qwen path byte-neutral (`test_model_registry` 24/24). **The RUN did NOT execute — BLOCKED on a CODE residual (not download/box), NOT attempted (would OOM-reboot the box):** the forward (`ForwardComposeImpl`) reads the FULLY-DEQUANTIZED f32 `weights.host` tower, and `LoadDeepseekV4FromGguf` builds it unconditionally (every routed expert `HostVec`→f32) ≈ ~24 GiB/layer × 43 ≈ **~1.0 TiB** f32, past the 119 GiB pool by ~layer 5; the keep-quant `weights.gguf` (~91 GiB) is built but never read by the forward. **Named residual W2c:** rewire the forward onto the CIQ `kMatmulBTQuant` keep-quant blocks + gate off the host dequant, THEN download + GB10 gen + gate + benchmark. NO tokens generated (not faked); DGX left as found (worker untouched, no download). Owns ONLY: the `deepseek4` config-builder + dispatch + the +1 test case + these record surfaces (model-matrix cell, spec §W8.5, STATUS/BENCHMARKS/roadmap/ledger/state, this row); no README/Metal/SACRED. Isolated worktree `.claude/worktrees/dsv4-w8-entrypoint` off `main` `376e186b`. | +| `CLAIM-DEEPSEEK-V4-W8` | `KERNEL-QUANT-CIQ-IQUANT` (NEW kernel-matrix row) `QUANT-GGUF-IQ2_XXS` `QUANT-GGUF-Q2_K` (all advanced to keep-quant compute, ownership transferred from `CLAIM-DSV4-GGUF-LOADER`); does NOT own a new model-matrix row — `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` stays owned by `CLAIM-DEEPSEEK-V4-IMPL`, still `SPIKE`. The IQ3_XXS quantization row was DROPPED from this claim 2026-08-06 by the live-state audit (`ACTIVE`→`READY`, no Git evidence for the claim); see the Closing and archival note. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w8-run` (CPU-only `build-cpu` Debug gate; DGX SSH used ONLY read-only to confirm host/disk/mem + HF HTTP-range header fetch — NO 91 GiB download, NO GPU run this lane; foreground; NOT pushed) | worktree branch `deepseek-v4-w8-run`, base `main` `4d618f59` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W8 — the single-Spark GGUF memory enabler + name-map. (1) Keep-quant `vec_dot` for IQ2_XXS/IQ3_XXS/Q2_K (`KERNEL-QUANT-CIQ-IQUANT`): owns the NEW `src/vt/cpu/cpu_quant_iq_tables.h` (shared grids) + the three `VecDot*` kernels in `cpu_quant_dot.cpp` + `DequantIQ3_XXS` in `cpu_quant_dequant.cpp` + three Q8_K traits rows in `cpu_quant_traits.cpp` + `kIQ3_XXS` in `dtype.{h,cpp}` + the three block structs in `cpu_quant_blocks.h` + `kIQ3_XXS` in the two exhaustive `DType` switches (`ops.cpp`) + ggml id-18 sizing in `gguf_reader.cpp` + the `kWeightCases` extension in `tests/vt/test_ops_quant_dot.cpp`. (2) The GGUF `blk.N.*`→V4 name map + FULL 1328-tensor coverage gate: NEW `scripts/check-dsv4-gguf-namemap.py` + `scripts/dsv4_gguf_manifest_names.txt` (the real manifest, HTTP-range-fetched). (3) A precise honest registry message advance in `deepseek_v4_registry.cpp`. Owns the `KERNEL-QUANT-CIQ-IQUANT` kernel-matrix row (+ `check-agent-record.py` count 43→44), the quantization-matrix IQ rows, the W8 section of `.agents/specs/deepseek-v4-flash.md`, and the record surfaces. SACRED-inert: additive to the vt quant tier + V4 GGUF path ONLY — does NOT touch the six existing k-quant kernels' numerics (their gate is UNCHANGED at 130426 of the 130444 assertions), the shared MLA/MoE, README, Metal, SACRED, apex, darwin. | `SPIKE` | 2026-07-29 — **W8 KEEP-QUANT ENABLER + NAME-MAP LANDED (foreground, NOT pushed); the RUN is the honest residual.** (1) Keep-quant `vec_dot` for IQ2_XXS/IQ3_XXS/Q2_K, 1:1 ports of ggml `vec_dot_{q2_K,iq2_xxs,iq3_xxs}_q8_K_generic` (`quants.c:514/855/999`) → `HasQuantDotKernel` TRUE ⇒ the loader keeps these blocks COMPRESSED (the ~91 GiB vs ~316 GiB bf16 OOM enabler). **HONEST CORRECTION of the brief's "IQ2_XXS + Q2_K":** the real `UD-IQ2_XXS` manifest uses IQ2_XXS (gate/up experts) + **IQ3_XXS** (down experts); Q2_K is the sibling `UD-Q2_K_XL` vehicle — all three landed so both vehicles keep-quant. **HONEST finding:** there is NO CUDA keep-quant vec_dot for ANY k-quant (`kMatmulBTQuant` is `kCPU`-only) — on GB10 keep-quant runs on the 20 ARM cores against the unified pool, so IQ2_XXS/IQ3_XXS/Q2_K match the six existing types (CPU tier). Gate `test_ops_quant_dot` **19 cases / 130444 assertions GREEN** (CPU Debug, new TUs `-Werror` clean): vec_dot vs INDEPENDENT f64 dequant-dot (≤1e-5·L1), NMSE ≤5e-4 vs dequant-f32 GEMM, bit-exact across threads, ragged-K reject; **RED-first PROVEN** (perturb IQ2_XXS `0.125` fold → 2 cases/18 fail, revert → 19/130444). (2) `blk.N.*`→V4 name map with **EXACT 1328/1328 coverage, 0 unmapped, 0 leftover** (`check-dsv4-gguf-namemap.py` rc=0) against the REAL manifest fetched via HF HTTP-range (no 91 GiB download); topology verified (43 layers, hash{0,1,2}, indexer 21 = compress_ratio==4, compressor 41). (3) Registry reject replaced with a precise "keep-quant+name-map landed; W2b tower materialization pending" message. **HONEST RESIDUAL — the RUN did NOT execute** and was NOT faked: it is blocked on the unimplemented **W2b** (materialize the keep-quant blocks + F32 MHC/DSA tensors into the DeepseekV4 towers via the name map) — a genuine code brick, not just memory/download. Resume: land W2b → free DGX disk ≥100 GiB → `flock $HOME/gpu.lock` + `docker stop local-ai-worker` → download `unsloth/DeepSeek-V4-Flash-GGUF/UD-IQ2_XXS` (3 shards, ~91 GB) → keep-quant load → `ForwardDevice` greedy → self-consistency + coherence gate. **W8-FINAL UPDATE 2026-07-29 (base `376e186b`, W2b now landed): the top-level `deepseek4` GGUF ENTRYPOINT ARM is LANDED + GATED, and the real RUN is re-scoped to a CODE blocker.** Wiring: `DeepseekV4HfConfigFromGguf` (`deepseek_v4_weights.cpp`) maps `general.architecture=deepseek4`→the registered `DeepseekV4ForCausalLM` (republishing geometry into `config.raw` for `ParseDeepseekV4Config`); `LoadedEngine::FromModelDir` routes via new anon-ns `HfConfigFromGgufDispatch`. Gate `test_deepseek_v4_gguf_load` **6/6·168** (CPU Release `-Werror`-clean; +1 case: config maps + `ModelRegistry::Resolve`→V4 factory), qwen path byte-neutral (`test_model_registry` 24/24). **The RUN did NOT execute — BLOCKED on a CODE residual (not download/box), NOT attempted (would OOM-reboot the box):** the forward (`ForwardComposeImpl`) reads the FULLY-DEQUANTIZED f32 `weights.host` tower, and `LoadDeepseekV4FromGguf` builds it unconditionally (every routed expert `HostVec`→f32) ≈ ~24 GiB/layer × 43 ≈ **~1.0 TiB** f32, past the 119 GiB pool by ~layer 5; the keep-quant `weights.gguf` (~91 GiB) is built but never read by the forward. **Named residual W2c:** rewire the forward onto the CIQ `kMatmulBTQuant` keep-quant blocks + gate off the host dequant, THEN download + GB10 gen + gate + benchmark. NO tokens generated (not faked); DGX left as found (worker untouched, no download). Owns ONLY: the `deepseek4` config-builder + dispatch + the +1 test case + these record surfaces (model-matrix cell, spec §W8.5, STATUS/BENCHMARKS/roadmap/ledger/state, this row); no README/Metal/SACRED. Isolated worktree `.claude/worktrees/dsv4-w8-entrypoint` off `main` `376e186b`. | | `CLAIM-DEEPSEEK-V4-W2B` | `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` (advanced: GGUF→tower materialization landed; row still owned by `CLAIM-DEEPSEEK-V4-IMPL`, stays `SPIKE` — this claim owns only the GGUF loader wiring, not the model row) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-w2b-tower` (CPU-only `build-cpu` Debug; NO GPU/download; foreground; NOT pushed) | branch `deepseek-v4-w2b-gguf-tower`, base `main` `341dfbb9` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W2b — the GGUF keep-quant TOWER materialization: wire the landed `blk.N.*` name-map + keep-quant blocks into the `DeepseekV4` weight towers so the model LOADS from `UD-IQ2_XXS`. Owns `LoadDeepseekV4FromGguf` + `DeepseekV4ParamsFromGguf` + the `DeepseekV4GgufWeights` tower in `deepseek_v4_weights.cpp`/`deepseek_v4.h`, the registry GGUF-reject lift in `deepseek_v4_registry.cpp`, the new `tests/vllm/models/test_deepseek_v4_gguf_load.cpp` + its CMake wiring, the W2b section of `.agents/specs/deepseek-v4-flash.md`, and the record surfaces. Loader-only + SACRED-inert: does NOT touch the forward (`deepseek_v4.cpp`), the W3-W6 primitive TUs, the shared MLA/MoE (`mla_attention`, `cuda_mla_attn`), the W8 keep-quant/name-map TUs (reused as-is), README, Metal, SACRED, apex, darwin. No new kernel-matrix row (loader wiring, not a kernel family) ⇒ no `check-agent-record.py` count bump. | `SPIKE` | 2026-07-29 — **W2b LANDED (foreground, NOT pushed).** `LoadDeepseekV4FromGguf` routes EVERY GGUF tensor through `GgufLoadPolicy::Route` with its name-map role: MW/SEW (MLA linears + router gate + shared + 256 routed experts + lm_head) KEEP their blocks COMPRESSED (`OwnGgufQuantBlocks`); V/ET/HASH (norms, MHC, DSA, sinks, embed, `tid2eid`, `exp_probs_b`) dequant. Accounts for every tensor (throws on unmapped/leftover), lifts the registry `kGguf` reject, and dequants the tiny CPU composition `host` tower so a loaded model forwards. Gate `test_deepseek_v4_gguf_load` **5/5·149** (tiny synthetic `deepseek4` GGUF, real `blk.N.*` names + Q8_0 keep-quant): accounting 126/126 == file set (none unmapped/leftover), keep-quant residency (down-experts stay Q8_0, bytes < dequant-f32; expand-policy RED → bf16), load→`DeepseekV4ForwardHost` finite+deterministic, RED-first (missing/leftover throw). CPU Debug `-Werror`-clean; SACRED-inert PROVEN (forward + W3-W6 + shared MLA/MoE empty-diff; prior V4 tests 6/26 + 4/40 unchanged). Honest 3-state: tiny-synthetic load→forward = DERIVED+BUILD-VERIFIED; the real 91 GB `UD-IQ2_XXS` load+generate = W8-FINAL (download + DGX keep-quant load + `ForwardDevice` greedy + self-consistency/coherence gate + benchmark). Row stays `SPIKE`. | | `CLAIM-DEEPSEEK-V4-W2C` | `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` (advanced: the GGUF forward now consumes the keep-quant tower — memory-feasible; row still owned by `CLAIM-DEEPSEEK-V4-IMPL`, stays `SPIKE`; this claim owns only the forward rewire + loader memory fix, not the model row) | Claude Code (opus-4-8) | isolated worktree `/tmp/.../scratchpad/wt-w2c` (CPU-only `build-w2c` Release `-DVLLM_CPP_CUDA=OFF`; NO GPU/download; foreground; NOT pushed) | branch `claim-deepseek-v4-w2c`, base `main` `328e6a50` (confirmed via `git rev-parse HEAD`) | DeepSeek-V4-Flash W2c — rewire the forward to CONSUME the keep-quant tower (the last CODE brick before the single-Spark run; W8-final proved the forward read the ~1 TiB f32 `host` tower and would OOM the box). Owns EXACTLY: `deepseek_v4.{h,cpp}` (the `V4Backend::gguf` weight source + the `Gemm`/`GemmRowSlice`/`GroupedOutputLoraGguf` keep-quant GEMM helpers + `ForwardComposeImpl` rewire + `DeepseekV4ForwardGguf` + the `has_gguf_weights` gate in `Forward`), `deepseek_v4_weights.cpp`/`deepseek_v4.h` (drop the big-weight f32 host expansion + the load-time memory `VT_CHECK` + `DeepseekV4{Host,Gguf}ResidentBytes`), `tests/vllm/models/test_deepseek_v4_gguf_load.cpp` (the keep-quant-forward + memory-bound gate), the spec §W2c, and the record surfaces. SACRED-inert: does NOT touch the shared MLA/MoE (`mla_attention.{h,cpp}`, `cuda_mla_attn.cu`, DeepSeek-V2 MoE), the W3-W6 primitive TUs (the correctness oracle — kept UNTOUCHED + still passing), README, Metal, SACRED, apex, darwin, the W7/W7-device CUDA path (untouched). No new kernel-matrix row (REUSES the landed `kMatmulBTQuant` CIQ GEMM) ⇒ no `check-agent-record.py` count bump. | `SPIKE` | 2026-07-29 — **W2c LANDED (foreground, NOT pushed); the OOM-infeasibility is FIXED.** `LoadDeepseekV4FromGguf` no longer f32-expands the big MLA/MoE/lm_head weights (only small norms/embed/MHC/DSA/hash dequant); `DeepseekV4ForwardGguf` runs the SAME composition with the 512-wide MLA linears + 256 routed/shared experts + lm_head consuming the COMPRESSED `weights.gguf` blocks via `vt::MatmulBT`→`kMatmulBTQuant`. `Forward` gates on `has_gguf_weights` (safetensors/NVFP4 + tiny-synthetic host path byte-identical). Gate `test_deepseek_v4_gguf_load` **7/7·185** (CPU Release `-Werror`-clean): keep-quant forward RUNS finite+deterministic; keep-quant(Q8_0)==dequant(bf16) RelL2 **0.0116** (< 0.05 near-tie); RED-first (no-sink miswire diverges 0.122; a rebuilt f32 tower fails the loader `VT_CHECK` + the host0) via a new `inverse` param on `RopeInplaceLayer`. **Proof on the real 80.7 GB model:** L00 q **0.9646→0.0013**, attn_out **0.5956→0.0175**, and the FULL 43-layer folded-state curve COLLAPSED to the keep-quant floor (MAX **0.0334**; L33 **0.5286→0.0029**; L34 **6.0239→0.0031**). **Generation** (greedy, chat-templated): **"The capital of France is Paris.<｜end▁of▁sentence｜>"** (ids `671 6102 294 8760 344 11111 16 1` — correct + EOS), deterministic/self-consistent. Benchmark: ours CPU-tier decode ~3.3 s/tok, peak resident 85.8 GiB (ds4 GPU: prefill 358, decode 16.5 tok/s). Named residuals (row ACTIVE, not DONE): GPU-expert dispatch (CPU queue), DSA-sparse ctx>512 (dense-fallback, exact short gen), paged-engine integration. Spec §W8-run.9 + 9 structured sections appended; model-matrix row SPIKE→ACTIVE (rollup ACTIVE 21→22 / SPIKE 9→8). Box restored (worker up, flock free, file retained). All 7 record checkers rc=0. Row `ACTIVE`. | @@ -1433,10 +1433,9 @@ table, tests, CMake. Details in the state-log entry of the same date. | `CLAIM-MLA-PREFIX-CACHE-ASSERT` | `MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm` (bugfix — restores its SACRED gate under asserts-enabled builds; the shared prefix-cache manager assertion it relaxes stays owned by `CLAIM-PREFIX-PROMPT-CACHING`, no row-state change to that row) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-mla-fix` (branch `fix/mla-prefix-cache-assert`, base `6abe09c`); dgx repro `/home/mudler/repro_deepseek` (unfixed) + `/home/mudler/repro_fixed` (fixed) | `fix/mla-prefix-cache-assert` | `src/vllm/v1/core/single_type_kv_cache_manager.cpp` (assert only) + `tests/vllm/v1/test_single_type_kv_cache_manager.cpp` (new MLA cases) | `DONE` (2026-07-23) | 2026-07-23 root-caused + fixed the DeepSeek-V2 assert abort | | `CLAIM-PREFIX-PROMPT-CACHING` | `ENG-CASCADE-ATTN`, `KV-MAMBA-ALIGN`, `KV-EVENTS` | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-af1befeda4d311f2c` (spike only — no build, no GPU work, no download) | worktree branch, base `1cb5f64` | SPIKE ONLY, no implementation. Owns `.agents/specs/prefix-prompt-caching-parity.md` — the UMBRELLA parity surface for prompt/prefix caching — plus the three `SPIKE` rows above. It also CORRECTS three already-evidenced rows in place without claiming them: `KV-BLOCK-POOL` (`ANCHOR-BACKFILL` -> `PARTIAL`, spike gap closed and the row's scope was undersold), `KV-HYBRID-COORD` (stays `PARTIAL`; the full cross-group intersection is ported verbatim, residue is assert-guarded), `KV-PREFIX-CACHE` (stays `PARTIAL`; gaps re-stated to what the source actually shows). **EXPLICIT NON-COLLISION:** `.agents/specs/prefix-caching.md` remains the accepted cache-policy/coordinator-selection leaf and is NOT edited or superseded — the new spec cross-references it and flags exactly one stale statement in it (its W3 "blocked on a supported non-hybrid model family", now false since five dense models landed and dense defaults APC ON). Does NOT own `KV-OFFLOAD` / `KV-EXTERNAL-CACHE` / `KV-CONNECTORS` (`ROAD-V1-D4`), `SERVE-METRICS` or `SERVE-UTILITY-ENDPOINTS`; the spec only states what each owes the caching feature. **Key dispositions:** `ENG-CASCADE-ATTN` is VERIFIED NOT OWED on the MRV2 path we mirror (default-off + absent from the V2 runner + unreachable on Blackwell) and is deliberately NOT scheduled; upstream `cache_partial_block` is DEAD CODE, so it is to be ported as an unwired primitive only **W1 IMPLEMENTATION 2026-07-22.** Owns `include/vllm/v1/metrics/stats.h` (new) plus the statistics wiring in `src/vllm/v1/core/kv_cache_manager.cpp`, `include/vllm/v1/core/kv_cache_manager.h`, `src/vllm/v1/core/sched/scheduler.cpp`, `include/vllm/v1/core/sched/scheduler.h`, `include/vllm/v1/engine/core.h`, `include/vllm/v1/engine/llm_engine.h`, `include/vllm/v1/request.h` (`num_preemptions` un-deferred), and `tests/vllm/v1/test_prefix_cache_stats.cpp`. Landed in the same commit as `CLAIM-KV-PERSISTENCE-LMCACHE` W1-W3 by design: that claim's gate 6 inherits these counters. W2-W9 are NOT started | `SPIKE` | 2026-07-22 — **W1 IMPLEMENTED AND GATED.** Prefix-cache statistics exist for the first time; `log_stats` defaults ON per the parity-enabler rule. **First measured hit rate 0.75** (1920/2560 tokens over 16 requests sharing a 128-token prefix), caching-OFF control 0.0. The `BACKEND-GATE-CUDA-SGLANG-PREFIX` hit-proof blocker is CLOSED. Resume point: W2 extra keys, then W3 the first cache-ON dense model gate on dgx | | `CLAIM-ROADMAP-D4-KV-EVENTS` | `KV-EVENTS` | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-add5bb82fe2773676` (CPU build `build-cpu` + CPU doctest only; no GPU, nothing downloaded; reference bytes captured from a local `msgspec` 0.21.1 pip in the scratchpad) | worktree branch, base `ffff4eb` | Owns ONLY these NEW files: `include/vllm/distributed/kv_events.h` + `src/vllm/distributed/kv_events.cpp` (the `BlockStored`/`BlockRemoved`/`AllBlocksCleared` + `KVEventBatch` types, the `msgpack` payload encoder, and the `EventPublisher`/`NullEventPublisher`/`CollectingEventPublisher`/`EventPublisherFactory` seam), `tests/vllm/v1/test_kv_events.cpp`, `.agents/specs/kv-events.md`. Additive edits to shared files, all KV-EVENTS-scoped (no collision with the sibling `CLAIM-KV-PERSISTENCE-LMCACHE`, `CLAIM-PREFIX-PROMPT-CACHING`, `CLAIM-ROADMAP-D4APC`, or the concurrent MM claim): `include/vllm/v1/core/kv_cache_utils.h` + `src/vllm/v1/core/kv_cache_utils.cpp` (`ExternalBlockHash` + `maybe_convert_block_hash` ONLY — appended after `none_hash_provenance`), `include/vllm/v1/core/block_pool.h` + `src/vllm/v1/core/block_pool.cpp` (the `KVCacheEvent` alias replacing the SPIKE placeholder + the three emission sites the spike left as marked-out no-ops + `emit_cached_block_events`/`_build_block_stored_event`/`_emit_block_removed_events`), and one source line in `CMakeLists.txt` + one test line in `tests/CMakeLists.txt`. MUST NOT touch any MM record/file or any other sibling's files | `ACTIVE` | **2026-07-27 — event GENERATION + PAYLOAD IMPLEMENTED AND GATED (CPU).** The KV-cache event system is ported 1:1 from `vllm/distributed/kv_events.py` @ `555967922`: the event data types, the emission at the BlockPool store/remove/clear sites (guarded by `enable_kv_cache_events`, default OFF ⇒ default path byte-identical), and the `msgpack` payload encoder that is BYTE-EXACT vs `msgspec.msgpack.Encoder()` on the upstream struct definitions (two golden vectors: default int-truncated hashes AND raw-bytes hashes). Gate `test_kv_events` 6 cases / 62 assertions: byte-exact serialization + the store→reuse→evict→reset event SEQUENCE with correct hashes/token_ids/parent/group_idx, RED-first proven (mis-wired hash + dropped `AllBlocksCleared` ⇒ 4 failing asserts). Default-off + APC unchanged: `test_block_pool` 132/132, `test_prefix_cache_stats` 36/36, `test_kv_cache_manager` 74/74, coordinator 106/106, utils 253/253. Clean full-library CPU `-Werror` 0 warnings. **DEFERRED (honest D4 residual): the LIVE ZMQ transport** (PUB/ROUTER sockets, replay buffer, publisher thread, DP port offset) is stubbed behind the faithful seam (`EventPublisherFactory` throws loudly on `"zmq"`), and the engine/scheduler wiring of the batch envelope + `report_mode=="full"` reuse path is not wired (our `Request` has no `kv_cache_report_mode`). W6/W7 + the larger-model LMCache grid remain the broader D4 residuals | -| `CLAIM-GLM-DSA-LATEST-DEEPSEEK` | `MODEL-TEXT-chatglm-chat-glmfor-causal-lm`, `MODEL-TEXT-glm-glm-for-causal-lm`, `MODEL-TEXT-glm4-glm4-for-causal-lm`, `MODEL-TEXT-glm4-moe-glm4-moe-for-causal-lm`, `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm` | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-aea3b1eba5776ab41` (spike only — no build, no GPU work, no download) | worktree branch, base `aa65ce7` | SPIKE ONLY, no implementation. Owns `.agents/specs/glm-dsa-latest-deepseek.md` plus the seven model-matrix rows above. Covers: the complete GLM family inventory and per-variant GB10 hardware fit; the two NEW shared primitives GLM needs (partial rotary factor, sandwich norms — we have neither); the full DSA (DeepSeek Sparse Attention) delta over dense MLA and the sm_121 sparse-backend verdict; and the V3.2-vs-V3 / V4-vs-V3 architectural deltas including the V4 tokenizer risk. **EXPLICIT NON-COLLISION:** `CLAIM-MLA-DEEPSEEK` owns `.agents/specs/mla-deepseek-campaign.md` and the five rows including `MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm` (which carries DeepSeek-V3.2). This claim does NOT edit that file or those rows; its DSA findings are supplied as CROSS-REFERENCED input, and the three places where it CORRECTS or EXTENDS that spike are enumerated in its §0.1 for the user to reconcile. **SHARED WORK to coordinate before implementing:** the `vt::MoeRouterTopK` extension (sigmoid + `e_score_correction_bias` + group masking + `routed_scaling_factor`) is the same gap both campaigns identified; whoever lands it first unblocks the other, and it must not be implemented twice. **2026-07-28: RELEASED `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` to `CLAIM-DEEPSEEK-V4-IMPL` (user-directed DeepSeek-V4-Flash implementation pickup — this claim held that row SPIKE-only with no worktree on it).** | `ACTIVE` | **2026-07-24 — G2 LANDED: `Glm4ForCausalLM` (GLM-4-9B-0414), the first GLM-family model, SACRED gate 16/16 vs vLLM 0.25.0 (STRICT token-exact 13/16 + near-tie 3/16, max gap 0 nats; vLLM K=5 self-deterministic ⇒ STRICT bar). Row `MODEL-TEXT-glm4-glm4-for-causal-lm` SPIKE→ACTIVE (correctness DONE, speed PENDING). New files (glm4.{h,cpp}, glm4_weights.cpp, glm4_registry.cpp) + one REGISTER; the two "new primitives" reduced to existing infra (partial+interleaved `RopeFromCache` `is_neox_style=false`; standalone `vt::RmsNorm` sandwich norms). Clean `-Werror` 0 warnings; rope unit 6692/6692; loader 523 tensors. Worktree `glm4-g2-bringup`, base `99a43c6`.** 2026-07-21 — spike committed. Recommended order: G2 (partial rotary + sandwich norms, fully independent of MLA) FIRST, then G1 (`Glm4MoeLite` on GLM-4.7-Flash) once `CLAIM-MLA-DEEPSEEK` reaches its W6. Nothing claims READY/ACTIVE/DONE **2026-07-22 — CROSS-CLAIM DISPOSITION recorded by `CLAIM-MLA-DEEPSEEK` W10 (the blocked-row honesty pass), with this claim's consent-by-record and no other edit to its rows or spec:** the GLM-5 row (`GlmMoeDsa`) is RELEASED from this claim's active row list and set to `BLOCKED` — HW-BLOCKED at 1404.2 GiB against 119 GiB of unified memory, and additionally DEP-BLOCKED because GLM-5.x is DeepSeek-V3.2 verbatim and therefore a SPARSE/DSA model, for which the sm_121 backend filter leaves `FLASHINFER_MLA_SPARSE_SM120` as the sole candidate and that path dispatches to flashinfer's dense-only XQA kernel (this spike's own §0.2 finding). Nothing else in this claim moves; its six remaining rows stay `SPIKE`, and `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm` (GLM-4.7-Flash, 31.2B / 58.2 GiB, FITS GB10) is now the MLA campaign's NAMED NEXT VEHICLE because it is the only reachable checkpoint that closes both of that campaign's permanent coverage gaps (`q_lora_rank=768` and `noaux_tc` with correction bias). **2026-07-24 — G1 LANDED: `Glm4MoeLiteForCausalLM` (GLM-4.7-Flash, 31.2B), SACRED gate 8/8 vs vLLM 0.25.0 (STRICT token-exact 1/8 + near-tie 7/8, 69/128 tokens strictly exact, max teacher-forced gap 0.0 nats, 0 forward-divergent; vLLM K=5 self-deterministic ⇒ STRICT bar). Row `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm` SPIKE→ACTIVE (correctness COMPLETE, speed PENDING).** The scope was much smaller than the spike estimated: the noaux_tc router ALREADY landed in the MLA campaign's W3, and GLM reuses the ENTIRE DeepSeek-V2 MLA stack over the SAME `DeepseekV2Weights`. Genuinely-new = an additive head_dim-256 MLA-prefill dispatch (256 kernel already compiled for 27B/35B; 192 byte-identical), MTP-tolerant parse/loader (`allow_mtp_tail`, defaulted false), the GLM registry TU, and a scoring-func fix (GLM omits `scoring_func`, noaux_tc defaults to sigmoid; DeepSeek greedy→softmax UNCHANGED). CLOSES the MLA campaign's C2 gaps (q_lora branch + noaux_tc router now e2e-gated). Loader 3/3·57117 (9491 mapped, 212 MTP-tail skipped, 0 unmapped/0 missing); router unit at GLM dims CUDA==CPU, memcheck 0; eager==graph; clean `-Werror` 0 warn. Regressions ALL byte-identical STANDALONE under flock: 27B 235/235, 35B 315/315, Coder 138/138, dense 184/184, OPT 63/63, DeepSeek-V2 223/223 (Release AND asserts-on exit 0, the shared-TU canary), Llama 92/92, Mistral 92/92. Worktree `glm4-moe-lite-g1`, base `b568d20`. Not pushed. G3/G4/G5 remain blocked-honesty passes.| +| `CLAIM-GLM-DSA-LATEST-DEEPSEEK` | `MODEL-TEXT-chatglm-chat-glmfor-causal-lm`, `MODEL-TEXT-glm-glm-for-causal-lm`, `MODEL-TEXT-glm4-moe-glm4-moe-for-causal-lm`, `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm`. The GLM-4-9B dense row was DROPPED from this claim 2026-08-06 by the live-state audit (`ACTIVE`→`READY`, no Git evidence for the claim); see the Closing and archival note. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-aea3b1eba5776ab41` (spike only — no build, no GPU work, no download) | worktree branch, base `aa65ce7` | SPIKE ONLY, no implementation. Owns `.agents/specs/glm-dsa-latest-deepseek.md` plus the seven model-matrix rows above. Covers: the complete GLM family inventory and per-variant GB10 hardware fit; the two NEW shared primitives GLM needs (partial rotary factor, sandwich norms — we have neither); the full DSA (DeepSeek Sparse Attention) delta over dense MLA and the sm_121 sparse-backend verdict; and the V3.2-vs-V3 / V4-vs-V3 architectural deltas including the V4 tokenizer risk. **EXPLICIT NON-COLLISION:** `CLAIM-MLA-DEEPSEEK` owns `.agents/specs/mla-deepseek-campaign.md` and the five rows including `MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm` (which carries DeepSeek-V3.2). This claim does NOT edit that file or those rows; its DSA findings are supplied as CROSS-REFERENCED input, and the three places where it CORRECTS or EXTENDS that spike are enumerated in its §0.1 for the user to reconcile. **SHARED WORK to coordinate before implementing:** the `vt::MoeRouterTopK` extension (sigmoid + `e_score_correction_bias` + group masking + `routed_scaling_factor`) is the same gap both campaigns identified; whoever lands it first unblocks the other, and it must not be implemented twice. **2026-07-28: RELEASED `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` to `CLAIM-DEEPSEEK-V4-IMPL` (user-directed DeepSeek-V4-Flash implementation pickup — this claim held that row SPIKE-only with no worktree on it).** | `ACTIVE` | **2026-07-24 — G2 LANDED: `Glm4ForCausalLM` (GLM-4-9B-0414), the first GLM-family model, SACRED gate 16/16 vs vLLM 0.25.0 (STRICT token-exact 13/16 + near-tie 3/16, max gap 0 nats; vLLM K=5 self-deterministic ⇒ STRICT bar). Row `MODEL-TEXT-glm4-glm4-for-causal-lm` SPIKE→ACTIVE (correctness DONE, speed PENDING). New files (glm4.{h,cpp}, glm4_weights.cpp, glm4_registry.cpp) + one REGISTER; the two "new primitives" reduced to existing infra (partial+interleaved `RopeFromCache` `is_neox_style=false`; standalone `vt::RmsNorm` sandwich norms). Clean `-Werror` 0 warnings; rope unit 6692/6692; loader 523 tensors. Worktree `glm4-g2-bringup`, base `99a43c6`.** 2026-07-21 — spike committed. Recommended order: G2 (partial rotary + sandwich norms, fully independent of MLA) FIRST, then G1 (`Glm4MoeLite` on GLM-4.7-Flash) once `CLAIM-MLA-DEEPSEEK` reaches its W6. Nothing claims READY/ACTIVE/DONE **2026-07-22 — CROSS-CLAIM DISPOSITION recorded by `CLAIM-MLA-DEEPSEEK` W10 (the blocked-row honesty pass), with this claim's consent-by-record and no other edit to its rows or spec:** the GLM-5 row (`GlmMoeDsa`) is RELEASED from this claim's active row list and set to `BLOCKED` — HW-BLOCKED at 1404.2 GiB against 119 GiB of unified memory, and additionally DEP-BLOCKED because GLM-5.x is DeepSeek-V3.2 verbatim and therefore a SPARSE/DSA model, for which the sm_121 backend filter leaves `FLASHINFER_MLA_SPARSE_SM120` as the sole candidate and that path dispatches to flashinfer's dense-only XQA kernel (this spike's own §0.2 finding). Nothing else in this claim moves; its six remaining rows stay `SPIKE`, and `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm` (GLM-4.7-Flash, 31.2B / 58.2 GiB, FITS GB10) is now the MLA campaign's NAMED NEXT VEHICLE because it is the only reachable checkpoint that closes both of that campaign's permanent coverage gaps (`q_lora_rank=768` and `noaux_tc` with correction bias). **2026-07-24 — G1 LANDED: `Glm4MoeLiteForCausalLM` (GLM-4.7-Flash, 31.2B), SACRED gate 8/8 vs vLLM 0.25.0 (STRICT token-exact 1/8 + near-tie 7/8, 69/128 tokens strictly exact, max teacher-forced gap 0.0 nats, 0 forward-divergent; vLLM K=5 self-deterministic ⇒ STRICT bar). Row `MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm` SPIKE→ACTIVE (correctness COMPLETE, speed PENDING).** The scope was much smaller than the spike estimated: the noaux_tc router ALREADY landed in the MLA campaign's W3, and GLM reuses the ENTIRE DeepSeek-V2 MLA stack over the SAME `DeepseekV2Weights`. Genuinely-new = an additive head_dim-256 MLA-prefill dispatch (256 kernel already compiled for 27B/35B; 192 byte-identical), MTP-tolerant parse/loader (`allow_mtp_tail`, defaulted false), the GLM registry TU, and a scoring-func fix (GLM omits `scoring_func`, noaux_tc defaults to sigmoid; DeepSeek greedy→softmax UNCHANGED). CLOSES the MLA campaign's C2 gaps (q_lora branch + noaux_tc router now e2e-gated). Loader 3/3·57117 (9491 mapped, 212 MTP-tail skipped, 0 unmapped/0 missing); router unit at GLM dims CUDA==CPU, memcheck 0; eager==graph; clean `-Werror` 0 warn. Regressions ALL byte-identical STANDALONE under flock: 27B 235/235, 35B 315/315, Coder 138/138, dense 184/184, OPT 63/63, DeepSeek-V2 223/223 (Release AND asserts-on exit 0, the shared-TU canary), Llama 92/92, Mistral 92/92. Worktree `glm4-moe-lite-g1`, base `b568d20`. Not pushed. G3/G4/G5 remain blocked-honesty passes.| | `CLAIM-QUANT-NVFP4-CT-W4A16` | `QUANT-NVFP4-CT-W4A16`, `MODEL-TEXT-qwen3-qwen3-for-causal-lm` (extend) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a141e107f210eb6c6`; dev-box host build `build-host`; DGX build/gate `~/scratch/nvfp4a16-src` (production flags, `flock /tmp/gpu` per gate series) | `main` (worktree) | compressed-tensors **NVFP4A16 (W4A16)** bring-up on the ALREADY-DONE dense `Qwen3ForCausalLM` forward — the breadth-sweep QUANT-SCHEME additivity row (rank 3, `.agents/specs/breadth-sweep-plan.md` §B.3). OWNS the new files: `include/vllm/model_executor/models/dense_nvfp4_gemm.h`, `include/vllm/model_executor/models/dense_device_glue.h`, `tests/vllm/models/test_qwen3_32b_nvfp4a16_{load,paged_engine}.cpp`, `scripts/qwen3-32b-nvfp4a16-oracle-capture.py`, `tests/parity/goldens/qwen3_32b_nvfp4a16_greedy/`, `.agents/specs/sweep-qwen3-32b-nvfp4a16.md`. APPEND-ONLY additions to shared files: `include/vllm/model_executor/models/dense_weight_loaders.h` (+`IsCtNvfp4Projection`/`LoadCtNvfp4W4A16`/`LoadMergedCtNvfp4W4A16`/`ReadCtF32Scalar`; NO existing helper touched), `tests/CMakeLists.txt` (two test rows), `tests/vllm/models/test_qwen3_forward.cpp` (one NEW case). Qwen3-dense-only edits: `include/vllm/model_executor/models/qwen3.h` (fp4 weight fields + `IsNvfp4()`), `src/vllm/model_executor/models/qwen3_weights.cpp` (per-Linear `.weight_packed` probe), `src/vllm/model_executor/models/qwen3.cpp` (`MlpBlock` branch). ONE shared-header structural change: `include/vllm/model_executor/models/dense_attn_block.h` — a PURE VERBATIM RELOCATION of `Dev`/`DBuf`/`MakeTensor`/`Reshape`/`DevicePoolPolicy` into the new `dense_device_glue.h` (same namespace ⇒ every consumer resolves unchanged) plus two NVFP4 dispatch branches in `AttnBlock`. MUST NOT touch `qwen3_5*.{h,cpp}` (the 27B/35B hot path stays byte-untouched by construction), any CUDA kernel, the runner, scheduler, platforms, the attention registry, or `hf_config.{h,cpp}` | `ACTIVE` | **2026-07-21 W4b DIAGNOSIS LANDED (worktree `agent-aa857f4152f4e9b03`, base `80d1da0`) — the strict-gate 4/6 is RESOLVED and CORRECTNESS IS CLOSED.** The named teacher-forcing isolation was RUN (new `scripts/qwen3-32b-nvfp4a16-neartie-gap.py` + a `VT_DUMP_IDS=1` bootstrap in the gate): **all 29 divergent positions gap <= 0.0625 nats, 28/29 EXACTLY 0.0** (our token IS vLLM's own argmax given OUR prefix). prompt[5] tok1 is an EXACT bf16 tie where vLLM's teacher-forced argmax is OURS and **vLLM contradicts its own incremental greedy**; prompt[2] tok1 gaps 0.0625 nats against **0.125 nats of vLLM's OWN batch-composition jitter at that position**. **VERDICT: NOT a W4A16 defect — the pre-existing dense-forward bf16 near-tie drift, recorded against `MODEL-TEXT-qwen3-qwen3-for-causal-lm`, NOT this quant row.** Gate converted to the ratified near-tie-robust bar WITH the evidence committed as goldens (`our_ids.npy` / `neartie_gap_mnats.npy`) and a hard anchor `REQUIRE`: **6/6 PASS (strict 4/6 + band 2/6, max gap 0.062 nats, 0 forward-divergent)**. NO engine/library source changed (diff = 1 test + 1 script + 2 goldens) ⇒ memcheck 0 and every regression preserved by construction and RE-CONFIRMED: 27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36, loader 5144/5144, `-Werror` 0-warn. SPEED (W5) still NOT started ⇒ row stays `ACTIVE`. Prior context: 2026-07-21 W0-W3 LANDED + dgx W4 gate run. Checkpoint premise VERIFIED FIRST (unlike OPT the plan's §B.1 claim was CORRECT: 5 shards / 20.6 GB / 1603 tensors, oracle loads it). vLLM 0.25.0 greedy MEASURED **DETERMINISTIC** on this checkpoint (K=5, **0 multi-valued cells**) ⇒ SACRED gate is **STRICT token-exact**. vLLM's dispatch OBSERVED (not inferred): `Using MarlinNvFp4LinearKernel for NVFP4 GEMM` — `use_a16` FORCES Marlin on sm_121. ADDITIVITY VERDICT: the KERNEL layer was fully additive (the vendored 35B Marlin W4A16 GEMM carries this scheme with ZERO new kernel code); the leaks were the device-glue header layering (fixed by a verbatim relocation) and — the standing one — the still-absent `QuantizationConfig`/`LinearMethod` abstraction (scheme selection remains a per-model tensor-name probe). SPEED bar explicitly PENDING — row stays `ACTIVE`, not `DONE` | | `CLAIM-MODEL-QWEN3-DENSE` | `MODEL-TEXT-qwen3-qwen3-for-causal-lm`, `ENG-RUNNER-MODELSHAPE` | Claude Code (opus-4-8) | `include/vllm/model_executor/models/qwen3.h`, `include/vllm/model_executor/models/dense_weight_loaders.h` (W2 shared helpers), `src/vllm/model_executor/models/qwen3_dense.cpp`, `src/vllm/model_executor/models/qwen3_weights.cpp` (W2 loader), `src/vllm/model_executor/models/qwen3_5_dense_weights.cpp` (W2 helper extraction, byte-identical), `src/vllm/v1/worker/gpu/runner.cpp` (W1 generalization only), `CMakeLists.txt` (TU add), `tests/CMakeLists.txt` (W2 test), `tests/vllm/models/test_model_registry.cpp`, `tests/vllm/models/test_qwen3_load.cpp` (W2 load gate), `tests/vllm/v1/worker/test_runner.cpp`, `tests/vllm/test_model_loader_gguf.cpp` (arch-list msg), `scripts/qwen3-neartie-gap.py`, `tests/parity/test_qwen3_paged_engine.cpp`, `tests/parity/goldens/qwen3_greedy_0_6b/*`, `tests/parity/goldens/qwen3_greedy_4b/*` (W4), `src/vt/cuda/cuda_flash_attn_fa2.cu` + `src/vt/cuda/cuda_paged_attn.cu` + `tests/vt/test_ops_paged_attn.cpp` (SPEED: d128 FA2 prefill + decode default flip), `include/vllm/model_executor/models/device_pool.h` (NEW: shared DevicePool extraction), `src/vllm/model_executor/models/qwen3_5.cpp` (uses the extracted header, byte-identical), `src/vllm/model_executor/models/qwen3.cpp` (pooled DBuf + `VT_QWEN3_ROPE_CACHE` opt-in rope) [TTFT-levers 2026-07-20, worktree `agent-aea71a6a5024b17d4`], `include/vt/ops.h` + `src/vt/ops.cpp` + `src/vt/cuda/cuda_glue.cu` + `src/vt/cpu/cpu_ops.cpp` (NEW `QkvSplit`/`kQkvSplit` GQA merged-qkv split op) + `src/vllm/model_executor/models/qwen3.cpp` (`VT_QWEN3_QKV_MERGE` merged-QKV path, default-off) + `docs/bench-evidence/qwen3-4b-binding-20260721.log` [SPEED re-binding + qkv-merge NEUTRAL 2026-07-21, worktree `agent-a5dfd17cc56194abc`] | worktree `agent-af01f4e66c3cc6c98` (W4; W0-W3 in `agent-a4eabfb705beb4ce9`) | First additive-model bring-up: Qwen3 dense (`Qwen3ForCausalLM`) on `Qwen3-0.6B` BF16 — spike [first-additive-model-qwen3-dense.md](specs/first-additive-model-qwen3-dense.md). **W0+W1 LANDED 2026-07-20:** W0 = new registry TU + header (`REGISTER_VLLM_MODEL(qwen3,"Qwen3ForCausalLM")` + full-attention-only `MakeQwen3ForCausalLMKVCache` + stub factory; forward throws until W3) resolving via the CLEAN model-factory seam (zero shared-array edit); W1 = the runner generalization (`ENG-RUNNER-MODELSHAPE`, seam gaps #1 `runner.cpp:458` layer_types indexing on pure-dense, #2 `:651-680` unconditional GDN metadata/step) — model-agnostic `has_mamba_group`/`gdn_group_id_>=0` guards. GATES PASSED: dgx CUDA `-Werror` 0-warn; **27B 235/235 + 35B 315/315 token-exact UNCHANGED** (behaviour-preservation); new CPU runner tests RED(SIGSEGV)→GREEN; registry resolves `Qwen3ForCausalLM`; ASan/UBSan clean on affected paths; full CPU ctest green (test_capi flaky-passes on rerun). **W2 LANDED 2026-07-20:** weight loader `qwen3_weights.cpp` (`LoadQwen3ForCausalLMWeights`) fills `Qwen3DenseWeights` from Qwen3-0.6B safetensors — merged qkv/gate_up raw-NK (vLLM `packed_modules_mapping`), per-head q/k RMSNorm, tied `lm_head` aliasing `embed_tokens` (checkpoint `lm_head.weight` skipped, mirroring vLLM `skip_prefixes=["lm_head."]`); `tie_word_embeddings`/`attention_bias` parsed from `config.raw` (deferred from W0). Shared BF16 helpers (`LoadBf16Direct`/`LoadBf16Transposed`/`LoadMergedBf16RawNK` + `MakeOwned`/`TransposeBf16`) extracted from `qwen3_5_dense_weights.cpp` anon-ns to `dense_weight_loaders.h` (`vllm::dense_loaders`) — 27B load byte-identical (SEAM GAP #3). GATES PASSED: dgx CUDA `-Werror` **0-warn**; W2 load gate `test_qwen3_load` **1567/1567** on dgx (all 311 tensors mapped, shapes exact, tied lm_head resolves, NO leftover); **27B 235/235 + 35B 315/315 token-exact UNCHANGED** (extraction behaviour-preserving); `compute-sanitizer memcheck` **0 err / 0 leak** on the load path; full CPU ctest **125/125**. **W3 LANDED 2026-07-20:** dense forward `qwen3.cpp` (`Qwen3DenseModel::Forward/ForwardDevice`) composed from public vt:: ops + the fusion catalog, wired into the factory (replacing the stub). 2 NEW recipes in `include/vt/recipes.h` (`kFusedAddRmsNormStd` non-gemma add-RMSNorm at the 3 norm sites via `vt::FusedChain`; `kAttnQkNormRope` non-gated qk-norm-rope composite) each byte-exact composite-tested. bf16 attention numerics (qkv/q-k-norm/RoPE/query/attn all bf16) mirror vLLM's bf16 per-op rounding. Two GENUINE latent bugs the first pure-dense bf16 model forced out + FIXED: (a) tokenizer `kQwen2Classic` (the classic Qwen2/Qwen3 pre-tokenizer regex was hard-rejected — `pretokenizer.{h,cpp}`,`tokenizer.cpp`; test_bpe/test_pretokenizer updated + new mark-split case); (b) `cuda_paged_attn.cu` WMMA prefill gated to head_dim 256 (the tensor-core ladder mistokenizes at d≠256 — validated only for the gate models). New files `qwen3.cpp`, `test_qwen3_forward.cpp`, `test_qwen3_paged_engine.cpp`, `scripts/qwen3-oracle-capture.py`. **W4 GATING 2026-07-20 — NOT token-exact yet:** Qwen3-0.6B greedy vs vLLM 0.25.0 oracle = **11/16 prompts** (all 16 first-tokens exact; 5 fail at late-decode SEMANTIC near-ties). Isolated as per-op bf16-rounding drift vs vLLM's kernels (CPU-exact-f32 also 10/16, overlapping divergences; RMSNorm rounding verified matching vLLM csrc) — structurally correct, NOT a bug. GATES: dgx CUDA `-Werror` 0-warn CLEAN rebuild; **27B 235/235 + 35B 315/315 UNCHANGED**; forward memcheck 0 err; recipes byte-exact CPU. **W4 near-tie RAZOR 2026-07-20 — 16/16-EXACT IS ILL-POSED (record-only, no code change):** ISOLATE-then-BIT-MATCH showed vLLM's OWN bf16 greedy (FA2, head_dim 128) is non-deterministic on these near-ties — vLLM ↔ its golden = 16·15·16 (N=3), flips 7/16 prompts over N=10 (p6 4/10, p15 3/10); all 5 of our divergences sit on gaps ≤0.125 nats (p5@5 EXACT 0.0 tie) and we emit vLLM's MAJORITY on p6/p15. `vt::GreedyArgmax` already mirrors torch.argmax lowest-index tiebreak. ⇒ not a kernel bug, no fixed golden to bit-match to; correct closure = a near-tie-robust distributional gate (user gate-design decision). Regression preserved by construction (binary byte-identical to `ec7f8a0`). Row stays `ACTIVE`. **W4 CORRECTNESS COMPLETE 2026-07-20 (worktree `agent-af01f4e66c3cc6c98`):** CORRECTED the razor's premise — vLLM greedy non-determinism was a BATCHING artifact; PER-PROMPT (batch=1, the gate's regime) vLLM 0.25.0 is DETERMINISTIC (0.6B 0-multi/K=10, 4B 0-multi/K=5). Near-tie robustness now comes from TEACHER-FORCING vLLM on OUR exact prefix (`scripts/qwen3-neartie-gap.py`): at all-but-2 positions vLLM's own argmax GIVEN OUR PREFIX is our token (gap 0.0000, bit-identical logprobs) — our forward matches vLLM's prefill logits; residual flips are bf16 near-ties (0.6B ≤0.125 nats, 4B ≤0.25) where vLLM's OWN prefill argmax disagrees with its decode. Gate (`test_qwen3_paged_engine.cpp` rewritten, 2 cases): PASS if our token within 0.5 nats of vLLM's teacher-forced argmax. **0.6B 16/16** (strict 12 + near-tie 4, max 0.125) + **BIGGER-MODEL PROOF Qwen3-4B 16/16** (36L, GQA 32/8, hidden 2560, same forward code; strict 10 + near-tie 6, max 0.25). GATES: CUDA `-Werror` 0-warn; 27B 235/235 + 35B 315/315 UNCHANGED (no engine source); memcheck 0; CPU syntax RC=0. New files `scripts/qwen3-neartie-gap.py` + goldens `tests/parity/goldens/qwen3_greedy_{0_6b,4b}/{greedy_dist,our_ids,neartie_gap_mnats}.npy`. Row stays `ACTIVE` until vLLM-throughput SPEED parity (DONE = correctness + speed). **STRICT-DECODE RAZOR 2026-07-20 (worktree `agent-a7e66dd02ed286cb2`, INVESTIGATION only — reverted, no code landed):** the group-swap FA2 split-KV d128 route measured WORSE (0.6B 12→11/16, 4B 10→9/16); vLLM runs FA2 varlen (`flash_attn_varlen_func`, fa_version 2 sm_121), not the group-swap kvcache path — remaining scope = FA2 VARLEN d128 decode, spec `specs/qwen3-decode-strict-bitmatch.md`. Byte-identical to HEAD (27B 235/235 + 35B 315/315 + near-tie band gate unchanged by construction). **FA2 VARLEN d128 RESOLVED 2026-07-20 (worktree `agent-a31b006ff4766f03b`):** vendored the EXACT vLLM varlen decode — NEW `src/vt/cuda/flash_attn/src/flash_fwd_split_hdim128_bf16_{,causal_}sm80.cu` + `LaunchDecodeVarlenFA2Bf16` (`src/vt/cuda/cuda_flash_attn_fa2.cu`: plain varlen, no group swap, num_splits=exact heuristic) routed by `Fa2DecodeQwen3Enabled`/`fa2_decode_qwen3` (`src/vt/cuda/cuda_paged_attn.cu`, `VT_FA2_DECODE_QWEN3` **default OFF**, d128-scoped so d256 arms untouched) + op-parity in `tests/vt/test_ops_paged_attn.cpp` + `CMakeLists.txt`. The kernel BIT-MATCHES vLLM's decode attention OUTPUT (teacher-forced gap 0.0000 at all-but-near-tie positions) but STRICT 16/16 is bf16-tie-bounded, NOT reached: FA2-varlen 0.6B 11/16 & 4B 9/16, ONE WORSE than the CUDA-core fallback (12/16 & 10/16). Shipped OPT-IN; near-tie-robust gate stays the closure; engine gate NOT tightened. GATES: `-Werror` 0-warn, op-parity 23/23 (454433 assertions), memcheck 0, **27B 235/235 + 35B 315/315 + default Qwen3 gate 16/16 UNCHANGED**. | `ACTIVE` (correctness COMPLETE — 0.6B + 4B gates PASS 16/16; **SPEED re-bound 2026-07-21 same-session matching-recipe: c1 EFFECTIVE EVERY-AXIS PARITY** (tput 0.98× / TTFT 0.90× WIN / TPOT 1.01× / P99 ITL 0.996× WIN); **c8 decode residual** (tput 0.93× / TTFT 0.38× WIN / TPOT 1.10× / P99 ITL 1.12×). TTFT residual RESOLVED — the prior 2.27×/5.85× + c8 ITL 4.3× were bad-denominator/num-prompts artifacts (fresh vLLM c1 TTFT ~152 ms; ours WINS TTFT both concurrencies). qkv-merge (`QkvSplit` op, mirrors vLLM `QKVParallelLinear`) implemented + MEASURED NEUTRAL (c8 decode 93% GPU-busy/compute-bound) ⇒ DEFAULT-OFF. cutlass RESOLVED (27B 235/235 on flashinfer-cutlass). Named residual = c8 decode-GEMM efficiency; claim does NOT close) | 2026-07-20 (W0-W4 landed; near-tie gate PASS 0.6B + 4B, re-passed with FA2 prefill+decode ON; SPEED `Qwen3-4B` vs graphed vLLM: d128 FA2 varlen prefill + FA2 decode default-ON → total tput 0.90×/0.62× (from 0.80×/0.48×), c1 TPOT 1.04× / ITL P99 0.98× WIN, prefill A/B +7%/+41% total & −55%/−48% TTFT; residual TTFT 5.85×/10.2× + c8 TPOT 1.38×; ledger 2026-07-20 FA2-prefill SPEED row. **TTFT LEVERS 2026-07-20 (base `812a57a`):** DevicePool extracted to shared `device_pool.h` (byte-identical; A/B-MEASURED PERF-NEUTRAL — "44% GPU-idle" premise disproven); RoPE cos/sin cache shipped OPT-IN `VT_QWEN3_ROPE_CACHE` (default OFF/RopeNeox) — MEASURED the dominant dense-TTFT lever (c1 median TTFT 209→135 ms = 0.87× vLLM parity, c8 316→144 ms = 0.38×) but not default: byte-identity unreachable on CUDA (FMA) + exposes FA2-split-KV near-tie nondeterminism (flaky strict gate); default gate 16/16 both models deterministic 4/4 + 27B 235/235 + 35B 315/315 UNCHANGED, `-Werror` 0-warn). **RoPE cos/sin cache flipped DEFAULT-ON 2026-07-20 (worktree `agent-a8c0b2005b8b5ff51`, base `4f84850`):** the opt-in blocker (b) — the alleged FA2-split-KV-combine run-to-run nondeterminism → flaky gate — was GROUNDED + DISPROVEN (engine byte-deterministic run-to-run: RoPE-off gate 4/4 + RoPE-on md5 3/3 + flipped gate 16/16 K=5; and `num_splits==1` on the short gate contexts ⇒ the combine kernel never launches). `RopeCacheEnabled` default-true (`src/vllm/model_executor/models/qwen3.cpp`); near-tie goldens regenerated on the CANONICAL `$HOME/cutlass-4.5.0` build (the flashinfer-bundled cutlass copy tips the 27B tok6 razor to 271→234/235; cutlass-4.5.0 gives 198→235/235 — Qwen3 near-ties are likewise cutlass-content-sensitive). Gate 16/16 both (0.6B strict 10+nt 6 max 0.0; 4B strict 11+nt 5 max 0.25) deterministic K=3; **27B 235/235 + 35B 315/315 on the canonical build** (RoPE flip isolated to the Qwen3-dense TU); `-Werror` 0-warn. SPEED (RoPE-ON default vs vLLM 0.25.0 production, in1024/out128): c1 total 0.97× / TPOT 1.00× / ITL P99 0.96× WIN / TTFT 2.27×; c8 total 0.82× / TTFT 1.90×; RoPE A/B c1 TTFT −34%, c8 −43% / total +10%; total tput 0.90×→0.97× (c1)/0.62×→0.82× (c8). Row stays `ACTIVE` — residuals = prefill-STEP non-attention glue (TTFT) + c8 split-KV decode occupancy. Files: `qwen3.cpp` (RoPE default), `tests/parity/goldens/qwen3_greedy_{0_6b,4b}/{our_ids,neartie_gap_mnats}.npy` (canonical-build regen), README/BENCHMARKS/model-matrix/roadmap/ledger/state | -| `CLAIM-PR3` | `KERNEL-GDN-AOT-BF16`, `KERNEL-GDN-SCRATCH` | root takeover of stopped `validate_pr3` / `complete_pr3` stream | primary recovery tree `/home/mudler/_git/vllm.cpp-pr3-validate`; 27B default/component integration in `/home/mudler/_git/vllm.cpp-nvfp4-small-m`; DGX evidence `~/work/vllm.cpp-noPy` plus `~/work/vllm.cpp-nvfp4-small-m/debug/gdn-out-bf16-c16-ab-20260712` | recovery branch integrated into `main` by `a767188`; current checkpoint on `codex/nvfp4-small-m` | PR #3 files and rows `KERNEL-GDN-AOT-BF16`+`KERNEL-GDN-SCRATCH`; 27B-only `GdnOutDType` default/f32 override in `qwen3_5.cpp`; ledger/inventory/roadmap evidence. `KERNEL-GDN-PACKED-DECODE` closed `DONE` (W1D3 EQUIVALENCE, `e47b4d6`); this claim retains prefill AOT and scratch/teardown only. No 35B default change | `ACTIVE` | 2026-07-14 (vendored BF16 H32/H48 AOT/safety evidence and native 16/16 correctness remain green. Binding `3f256ab` has c16 total at 1.027889× but mean TPOT/ITL at 0.987450×. Pure-decode packed execution is split into its own row/claim; all 35B paths keep f32) | | `CLAIM-BACKEND-METAL-M3A-1` | `BACKEND-METAL-MLX` (study work row **`M3a`** — THE FIRST MODEL ON A NON-CUDA BACKEND; the row stays `ACTIVE`, because correctness is met and SPEED is a separate, unmet bar) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a5bedee70b3431835`, base `72f5db2`; M4 build/gate `192.168.68.103:~/vllmcpp-metal` with an UNMODIFIED `72f5db2` baseline tree at `~/vllmcpp-base` for the pre-existing-failure A/B, and the md5-verified OPT-125m bf16 checkpoint at `~/models/opt-125m-bf16-st`; Linux CPU build `build-cpu` in the worktree; dgx build/gate `dgx.casa:~/vllmcpp-metal`, the whole regression series under ONE `flock $HOME/gpu.lock` | worktree branch, base `72f5db2` | Bring OPT-125m (`OPTForCausalLM`) up END TO END on Apple GPU and gate it. Owns EXACTLY: the five new MSL kernels + their host encoders and registrations in `src/vt/metal/metal_msl.h` and `src/vt/metal/metal_ops.mm`; `MetalPlatform::get_attn_backend_priority` + the new `supports_model_architecture` override in `src/vllm/platforms/metal.cpp`; the `Platform::supports_model_architecture` DEFAULT-`true` virtual in `include/vllm/platforms/interface.h`; `SelectQueue` in `src/vllm/entrypoints/model_loader.cpp`; the `kFlashAttnMetal` NAME registration in `src/vllm/v1/attention/backend.cpp`; the KV-cache residency predicate at `src/vllm/v1/worker/gpu/runner.cpp:516`; and `tests/vt/test_metal_backend.cpp` / `tests/vllm/models/test_opt_paged_engine.cpp`. Owns NO op wrapper body, NO CUDA/Vulkan/CPU kernel, NO model forward, and NO scheduler/runner algorithm — and touched none of them | **LANDED + TRI-BOX GATED 2026-07-22.** **OPT-125m runs on Apple GPU and is STRICT token-exact 6/6 prompts / 96/96 tokens** vs the SAME committed dgx-captured vLLM 0.25.0 goldens the CUDA arm uses — a DEVICE-INDEPENDENT bar, so Metal met the bar CUDA already met rather than one re-derived on Metal. Registered Metal ops **10 -> 15 of 75** (`kEmbedding`, `kQkvSplit`, `kReshapeAndCache`, `kPagedAttention`, `kGreedyArgmax`). **METAL EXECUTION PROVEN, NOT INFERRED:** `runner().device().type == kMETAL` plus, for all NINE ops OPT dispatches, `selections > 0` AND `declines == 0` (`kPagedAttention` selections **1152**); per-op unit tests NaN-poison every output so an un-executed kernel cannot pass numerically. Per-op vs the CPU oracle: four ops **BIT-EXACT**, `kPagedAttention` **NMSE 4.99e-13** (bar 5e-4, bit-exactness explicitly NOT claimed — online vs materialized softmax is a different reduction order by construction). **SEAM FIXES: study predicted 4 — 2 CONFIRMED+fixed (hardcoded `kCUDA` loader queue; empty Metal attn priority), 1 REFUTED BY MEASUREMENT (the runner's `vt/cuda/` include is declaration-only and links fine on a Metal-only macOS build — no change needed or made), and 1 NEW BUG THE STUDY MISSED: `runner.cpp:516` gated KV-cache DEVICE RESIDENCY on `is_cuda()`, handing a HOST pointer to a device kernel** (same class as the earlier `dense_attn_block.h` bug, invisible for the same reason — no test sees it until a model runs off-NVIDIA). NEW SEAM forced by the above: `Platform::supports_model_architecture()` (default `true` => CUDA/CPU byte-unchanged), because once `SelectQueue` asks the platform, "which device am I on" stops being "which device can run THIS model"; **caught by the macOS regression suite, not by design**, and recorded as such. Tree-friction counts RE-JUDGED: OPT-on-Metal needed **2 of the 13** `is_cuda()` sites and **0 of the 5** CUDA includes — no tree-wide unpicking campaign started or warranted. **EVIDENCE — M4:** clean `-Werror` **0 warn** on a CLEAN FULL rebuild (AppleClang 21, CLT-only, MSL at RUNTIME); `test_metal_backend` **12 cases / 18,535 assertions**; full ctest **154/156**, both misses documented pre-existing platform gaps, and `test_capi`'s standalone macOS failure PROVEN pre-existing by reproducing the identical `CHECK(1 == 2)` on an unmodified `72f5db2` build on the same box. **Linux CPU:** clean `-Werror` 0 warn. **dgx:** clean CUDA `-Werror` **0 warn**; ALL SIX REGRESSIONS UNCHANGED, each STANDALONE. **NO METAL SPEED NUMBER IS CLAIMED OR OWED** — the M4 could not be quieted (root LaunchDaemon needs interactive sudo; the desktop aerial wallpaper is the larger contender), so any timing is void under the standing rule; correctness and speed are separate bars and only correctness is met, which is why the row stays `ACTIVE`. `BACKEND-GATE-METAL-MLXLM` correctly stays `INVENTORIED` — it binds on Qwen3-dense (`M3b`), the model MLX also runs. Still owed: `M2r`, `M3b` (+RoPE), `M3c` (batched encoders), `M4`, `W0b-3` | 2026-07-22 | | `CLAIM-BACKEND-METAL-M3B-1` | `BACKEND-METAL-MLX` (study work row **`M3b`** — Qwen3-dense, the SECOND model on a non-CUDA backend; row stays `ACTIVE`, correctness met, SPEED unmet) + `BACKEND-GATE-METAL-MLXLM` (the native-competitor MLX benchmark cell) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm-metal-qwen3` (branch `metal-qwen3-dense`), base `origin/main` `4884d03`; M4 build/gate/bench `192.168.68.103:~/work/m3b-qwen3` (Qwen3-0.6B gate + Qwen3-1.7B bench); dgx CUDA-build + Qwen3-dense-gate regression check `dgx.casa:~/work/m3b-cuda-check` under `flock $HOME/gpu.lock` | worktree branch, base `4884d03` | Bring Qwen3-dense (`Qwen3ForCausalLM`) up END TO END on Apple GPU, gate it token-exact, and produce the binding MLX-vs-ours benchmark on the model MLX also runs. Owns EXACTLY: the three new RoPE MSL kernels + host encoders + registrations in `src/vt/metal/metal_msl.h` and `src/vt/metal/metal_ops.mm`; the `Qwen3ForCausalLM` entry in `MetalPlatform::supports_model_architecture` (`src/vllm/platforms/metal.cpp`); the Metal-aware branch + provider proof in `tests/parity/test_qwen3_paged_engine.cpp`; the three new RoPE cases in `tests/vt/test_metal_backend.cpp`. Owns NO op wrapper body, NO CUDA/Vulkan/CPU kernel, NO model forward, NO scheduler/runner algorithm | **LANDED 2026-07-23 (M4-gated; dgx CUDA-build check DONE).** **dgx:** clean CUDA `-Werror` 0 warn on the only dgx-compiled change (`test_qwen3_paged_engine.cpp`); device correctly stays CUDA (no metal branch); CUDA/CPU anchor path diff-verified behaviour-preserving. **PRE-EXISTING stale-golden finding (NOT an M3b regression):** the Qwen3-dense CUDA gate is currently RED at the hard our_ids anchor — dgx CUDA gives 15344 at prompt[0] tok=5 vs committed golden 9625; airtight attribution — `git diff 4884d03 HEAD` is 5 files (3 Metal-only not compiled on Linux + 2 tests), CUDA engine byte-identical to base, AND the M4 Metal run produced the SAME 15344, so two devices corroborate the committed goldens (captured 2026-07-20 `e510e85` on canonical cutlass-4.5.0) are stale vs the current runtime (same class as the 27B NVFP4-emulation control). Goldens NOT modified (refresh needs a fresh dgx oracle capture, out of M3b scope). Other 5 suites have zero changed compiled code on dgx → unchanged by construction, not re-run. **Qwen3-dense runs on Apple GPU; Qwen3-0.6B SACRED gate 16/16 on Metal** (10 STRICT token-exact vs vLLM 0.25.0 greedy, 2 near-tie-band, 4 cross-device near-ties all at gap 0 nats, 0 forward-divergent). Metal op set **15→18/75** (`kRopeFromCache` bit-exact, `kRopeCosSinCache`/`kRopeNeox` NMSE 0/4e-15). Metal execution PROVEN (device==kMETAL, `selections>0 ∧ declines==0` all 9 ops, kRopeFromCache/kPagedAttention 7168). Study prediction CORRECTED: the default `VT_QWEN3_ROPE_CACHE`-ON path needs kRopeCosSinCache + kRopeFromCache (not kRopeFromCache alone); kRopeNeox is the cache-off opt-out. `test_metal_backend` **15 cases / 19,331 assertions**, `-Werror` 0 warn on the M4. DSR holds **86**. **BENCHMARK (INDICATIVE / BLOCKED-ON-SUDO):** Qwen3-1.7B bf16 p512 g128 same box/session — ours (Metal) decode 4.29→2.14 tok/s/stream, TTFT 4.9→47.2 s, peak 7.36→8.78 GB vs MLX 27.77→211.55 tok/s, TTFT 0.47→7.16 s, peak 3.78→5.28 GB (b=1→16); ours a knowingly-unoptimised FLOOR ~6–11× slower decode, ~7–10× TTFT, ~2× memory. The Mac could NOT be quieted (no passwordless sudo; root `com.localai.worker` + aerial wallpaper up), so NOT binding. Metal TUs `VLLM_CPP_METAL` AUTO→OFF on Linux so the only dgx-compiled change is `test_qwen3_paged_engine.cpp` (behaviour-preserving on CUDA). Still owed: `M2r`, `M3c` (batched encoders — the named speed lever), `M4`, `W0b-3` | 2026-07-23 | @@ -1550,20 +1549,16 @@ items a-runner/b stay with the async/GDN `runner.cpp` owners. | `CLAIM-CUDA-AMPERE-FA2-WA1` | `BACKEND-CUDA-SM080`, `BACKEND-CUDA-SM086`, `BACKEND-CUDA-SM087`, `BACKEND-CUDA-SM089` (FA2 leg) — WA-1 of `.agents/specs/cuda-arch-ampere-fastpath.md` | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a85d473507104438b`; dgx GB10 build-verify scratch `~/ampere-fa2-verify` (`git archive` transfer), all GPU under `flock $HOME/gpu.lock` | worktree branch `worktree-agent-a85d473507104438b`, base local `main` `2bf1aaa9` | **DERIVE-AND-SHIP (build-verify).** Owns the `fa2` FEATURE-TABLE cell (`cmake/CudaArchFeatures.cmake`), the `VLLM_CPP_CUTLASS_HEADERS` + FA2-gate host-predicate in `CMakeLists.txt`, the `fa2` assertions in `cmake/CudaArchFeaturesTest.cmake`, and the FA2-leg cells of `BACKEND-CUDA-SM080/086/087/089` + `BACKEND-CUDA-COMP-FA` Ampere leg + the WA-1 intro note in `.agents/backend-matrix.md`; records (`docs/STATUS.md`, `docs/BENCHMARKS.md`, `roadmap_v1.md` `ROAD-V1-D1-CUDA`, `specs/roadmap-v1-completion.md`, `specs/cuda-arch-ampere-fastpath.md` WA-1, `.agents/parity-ledger.md`, `.agents/state.md`, this claim). MUST NOT touch README/Metal/demo (concurrent session owns those), or any kernel/model/runner TU. Ships LABELED `DERIVED+BUILD-VERIFIED (testing-welcome)` — NO Ampere board here. | `DONE (DERIVED+BUILD-VERIFIED)` | 2026-07-27 — `fa2` cell → `8.0,8.6,8.7,8.9,12.0a,12.1a` + FA2 gate decoupled onto `VLLM_CPP_CUTLASS_HEADERS`; dgx `87`+`80` `-Werror` 0-warn + `cuobjdump` sm_87/sm_80 FA2 cubins (7 TUs); resolver test ENABLED 80/86/87/89; sm_121a OLMo-2 SACRED gate 16/16; records + checkers green; NOT pushed | | `CLAIM-MLA-DEEPSEEK` | `MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm`, `MODEL-TEXT-deepseek-v2-deepseek-for-causal-lm`, `MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm` | Claude Code (opus-4-8) | worktree `agent-a5c1b57e0f74e4423` (W8; dgx goldens/build/gates at `~/w8mla`); earlier legs `agent-a06bba75ea226aadc` (W7, `~/w7mla_src`), `agent-a05498a1005f452af` (W6, `~/w6mla`), `agent-a43819c9776dc1585` (W5, `~/w5mla`) | worktree branch, base `04f5c01` (W8); earlier legs `ce43c51` (W7), `2846467` (W6), `5395203` (W5), `ed2c342` (W4), `a05437f` (W2+W3), `fb3fd5d` (W0+W1), `b4f14ee` (spike) | SPIKE ONLY, no implementation. Owns `.agents/specs/mla-deepseek-campaign.md` plus the five model-matrix rows above and the corrective dependency-cell edits on the two MiniMax-M3 rows (which stay `INVENTORIED`). Determined: (1) MLA architecture + the full upstream chain incl. the dependency chain below vLLM (Triton MLA decode = vLLM <- SGLang <- lightllm); (2) **GB10/sm_121 selects `TRITON_MLA`** for dense MLA (`vllm/platforms/cuda.py:129-133` — the `major == 12` list has exactly two entries and the other is sparse-only) and **`FLASH_ATTN`** for MLA prefill (`mla/prefill/selector.py:74-77`), so every sm90/sm100-only MLA kernel (FlashMLA, CUTLASS MLA, FlashInfer MLA, TokenSpeed) is out of reach and out of scope; (3) the CROSS-CUTTING COST = the compressed-latent KV cache (one 576-wide latent per token, `num_kv_heads=1`, no separate V; upstream cache shape is 3D at `mla_attention.py:1216-1224`), which our tree cannot express — `runner.cpp:487-492` hardcodes the K+V factor 2 and derives shape from the HF config, and `vt::ReshapeAndCache`/`vt::PagedAttention` (`include/vt/ops.h:1167,1201`) have K/V-pair signatures. Block manager/prefix caching are NOT affected (upstream maps MLA onto the stock `FullAttentionManager`, `single_type_kv_cache_manager.py:1539`); (4) HW verdict — **only DeepSeek-V2-Lite (~29.3 GiB bf16) fits GB10**; V3/V3.2 (~1250 GiB), Kimi-K2.5 (~2000 GiB), MiniMax-M2 (~428 GiB) and M3 are HW-BLOCKED e2e (dgx has 238 GiB free disk, 119 GiB unified); Kimi-Linear-48B (~89.4 GiB) is HW-MARGINAL; (5) three inventory corrections — Kimi-Linear IS MLA (NoPE), MiniMax M2/M3 are NOT MLA, `DeepseekForCausalLM` is NOT MLA (plain MHA); plus the user-premise correction that no "Kimi K3" exists at the pin (K2.5 is newest). W0-W10 breakdown recorded in the spike §10. **W0 + W1 LANDED 2026-07-21** (worktree `agent-a95c0c3038971eb12`, base `fb3fd5d`; dgx scratch `~/scratch_mla_w1`). **W0 = ground the facts, NO code:** DeepSeek-V2-Lite fetched to dgx (30 GB, 4 shards, snapshot `604d5664`) and loaded in the vLLM 0.25.0 oracle; **both flagged source reads CONFIRMED BY OBSERVATION on sm_121** — the oracle prints `Using TRITON_MLA attention backend out of potential backends: ['TRITON_MLA']` and `Using FLASH_ATTN MLA prefill backend`, so W4/W5 target the right kernels and the sm90/sm100-only MLA family stays out of scope; the real `config.json` confirms every geometry number (576-wide latent = `kv_lora_rank=512`+`qk_rope=64`, `qk_nope=128`, `v_head_dim=128`, `q_lora_rank=null`, `n_group=topk_group=1` softmax/greedy, 64+2 experts, 27 layers) plus `is_neox_style=False` and the mscale^2 correction; both recorded coverage gaps are real (no `fused_qkv_a_proj` branch, no `e_score_correction_bias` -> unit-gated only). Incidental W9 input: the oracle's MoE backend is **FlashInfer CUTLASS**, not Triton `fused_moe`. **W1 = spec-driven KV allocation, ZERO MLA math:** `MLAAttentionSpec` (factor-1 page formula, `KVCacheSpecKind::kMlaAttention`, registered against the ORDINARY `FullAttentionManager` per `single_type_kv_cache_manager.py:1539`), `ResolveKvCacheDType()` centralising the KV storage dtype onto the spec, and `runner.cpp` sizing every attention buffer as `num_blocks * spec->page_size_bytes()` + building its `PagedKvCache` view from the spec instead of the HF config. Owns additionally: `include/vllm/v1/kv_cache_dtype.h` (new), `include/vllm/v1/kv_cache_interface.h`, `src/vllm/v1/kv_cache_interface.cpp`, `src/vllm/v1/kv_cache_spec_registry.cpp`, `src/vllm/v1/worker/gpu/runner.{h,cpp}`, the three model KV factories (`qwen3_5_common.cpp`, `qwen3_dense.cpp`, `qwen3_moe_registry.cpp`), `tests/vllm/v1/test_kv_cache_interface.cpp`, `tests/vllm/v1/worker/test_runner.cpp`. Gated: clean dgx CUDA `-Werror` 0 warn/0 err; **27B 235/235 + 35B 315/315 + Qwen3-Coder 6/6 + Qwen3-dense 16/16 UNCHANGED**; `test_runner` 15/15, `test_kv_cache_interface` 21/21, `test_llm_engine` 5/5; new path proven EXERCISED via `fa_page_size_bytes()` + a `page_size_padded` case. NO MLA ops, NO MLA model, NO backend-selection change — **no model row moves off `SPIKE`** **W2 + W3 LANDED 2026-07-21** (worktree `agent-afc317d70ffbd733f`, base `a05437f`). **W2 = the MLA branch of `_get_backend_priorities`, as DATA:** the whole of `cuda.py:84-176` (BOTH branches — MLA sm_100 incl. the `:96-115` adaptive sparse tail, MLA sm_12x, MLA `else`, and the two pre-existing non-MLA arms) is now a TABLE in the NEW header `include/vllm/platforms/cuda_attn_priority.h`, one row per upstream arch arm keyed on `(use_mla, major)`; a new arch is a ROW. It is a HEADER, not the CUDA-only TU, so the CPU test tier asserts the REAL table and the pre-W2 hand-copied `FakeCudaPlatform` duplicate is deleted. `AttnSelectorConfig` (`interface.h`) carries `use_mla`/`use_sparse`/`num_heads`/`quantized_kv_cache`, all defaulting to the pre-W2 dense answer. **The DSA/sparse seam is left OPEN and unit-proven:** GB10 keeps BOTH upstream entries and the sparse one is eliminated by a real FILTER — `AttentionBackend::is_mla()`/`is_sparse()` compared against the request in `SelectAttentionBackendName`, mirroring `backend.py:307-360 validate_configuration` — so a future DSA backend is selected by declaring `is_sparse() == true`, with ZERO edit to the table or the selector (the test registers a stand-in sparse backend and shows both directions). `TritonMLABackend` registers the NAME + upstream's 3-D `get_kv_cache_shape` (`mla_attention.py:1216-1224`, no K/V axis, `num_kv_heads != 1` REFUSED) with `get_impl_cls()` deliberately still `nullptr`. MLA prefill priority ported too (`mla/prefill/selector.py:47-76`; GB10 -> `[FLASH_ATTN]` alone). **W3 = two new `vt::` ops, both CPU-reference-gated.** (a) `vt::ConcatAndCacheMla` — `OpId::kConcatAndCacheMla` + validation + CPU ref + CUDA, ported 1:1 from `csrc/libtorch_stable/cache_kernels.cu:401-442` (+ wrapper `:842-905`). Per the whole-chain rule this was VERIFIED rather than assumed: `_custom_ops.py:2532` binds straight to `torch.ops._C_cache_ops.concat_and_cache_mla` from vLLM's OWN csrc — there is no flashinfer/cutlass/TRT-LLM variant in the dense-bf16 path, the only sibling being the out-of-scope `concat_and_cache_ds_mla_kernel` fp8 layout (which our dtype check refuses); the only thing that displaces it is the COMPILE-TIME `mla_rope_kvcache_cat_fusion.py:40` RoPE fold, deferred to W9. It concatenates the latent + rope part into ONE 576-wide entry — the write `ReshapeAndCache`'s `(k,v,k_cache,v_cache)` signature cannot express — and is STRIDE-driven on every operand so a per-layer cache slice and the two column halves of `kv_a_proj_with_mqa` both work copy-free. (b) The **SHARED** `vt::MoeRouterTopK` grouped-topk (`noaux_tc`) extension this table flagged as owed to both campaigns — **landed HERE, so `CLAIM-GLM-DSA-LATEST-DEEPSEEK` consumes it and must NOT reimplement it.** Additive `MoeRouterTopKArgs` fields (`scoring_func`/`num_expert_group`/`topk_group`/`routed_scaling_factor`) + an optional trailing `e_score_correction_bias` tensor; ported 1:1 from `grouped_topk_router.py:106-161`. The existing router is byte-identical BY CONSTRUCTION, not by measurement: `num_expert_group == 0` dispatches the ORIGINAL kernel on both devices and a separate `MoeRouterGroupedTopKKernel` implements the new path. Recorded deviation: upstream `torch.topk` has unspecified tie order; we keep lowest-index-wins for both group and expert selection, which is what makes CPU and CUDA agree bit-for-bit. **The `noaux_tc` evidence is UNIT-ONLY and says so:** V2-Lite has `n_group=topk_group=1` and no bias, so the e2e vehicle exercises none of it; `tests/vt/test_ops_moe_router_grouped.cpp` runs at V3's REAL dims (256 experts, n_group=8, topk_group=4, top_k=8, sigmoid, scaling 2.5, WITH the bias) against an INDEPENDENT sort-based transcription of the upstream formula. Owns additionally: `include/vllm/platforms/cuda_attn_priority.h` (new), `include/vllm/platforms/interface.h`, `src/vllm/platforms/{cuda,cpu}.cpp`, `include/vllm/v1/attention/{backend,registry}.h`, `src/vllm/v1/attention/{backend,registry}.cpp`, `include/vt/ops.h`, `src/vt/ops.cpp`, `src/vt/cpu/{cpu_cache,cpu_ops}.cpp`, `src/vt/cuda/{cuda_cache.cu,cuda_moe.cu}`, `tests/vllm/v1/attention/test_attn_backend_registry.cpp`, `tests/vt/test_ops_mla_cache.cpp` (new), `tests/vt/test_ops_moe_router_grouped.cpp` (new), `tests/CMakeLists.txt`. **NO MLA attention math, NO MLA model, NO forward — no model row moves off `SPIKE`.** **W4 LANDED 2026-07-22** (worktree `agent-a3de4e8cccbae9c23`, base `ed2c342`, dgx scratch `~/w4mla`; rows STAY `SPIKE` — W4 adds a kernel and fills a `nullptr`, it makes no model supported). **W4 = the MLA DECODE kernel `vt::MlaDecodeAttention`** — the MQA decode over the compressed latent (QK 576 / V 512, `num_kv_heads == 1`) reading the paged 3-D cache W3 writes. STRUCTURE port with `file:line` on both sides, of the pair W0 OBSERVED EXECUTING: `MlaDecodeStage1` <- `triton_decode_attention.py:278-458 _fwd_grouped_kernel_stage1` (the `IS_MLA` branch), `MlaDecodeStage2` <- `:575-639 _fwd_kernel_stage2`, launchers <- `:470-573`/`:642-682`/`:719-754`, `ComputeNumKvSplits` <- `triton_mla.py:40-47`, `TritonMLAImpl::forward_mqa` <- `:189-260`, the split workspace <- `:57-78` (as the house grow-only per-stream scratch with `RetireGraphScratch`), and the CPU reference's numerics <- `csrc/cpu/mla_decode.cpp`. The ACTUAL Triton was read, not the wrapper — which is how `:424-431`'s `v = tl.trans(k)` was ported correctly: under `IS_MLA` NO V tile is loaded at all, V is the leading 512 columns of the SAME latent row already loaded as K, so one shared-memory tile is both. **Honest FA-2 reuse verdict, recorded in the TU header:** our FA-2 split+combine fit at the ALGORITHM level (split schedule + LSE merge algebra + the fixed-ascending / never-atomicAdd determinism rule) and NOT at the code level — the vendored launcher takes separate 4-D k/v caches and is instantiated only for symmetric head_dim {128,256}, which cannot express a 3-D single-buffer cache with QK 576 / V 512; generalizing it is W5's PREFILL job. **Evidence is UNIT-LEVEL and deliberately strong** (no e2e model until W7): `tests/vt/test_ops_mla_attn.cpp` ports `test_mla_decode_cpu.py` — its `ref_mla` as an INDEPENDENT TWO-PASS oracle (a different algorithm from the streaming online-softmax both impls use), its full parametrization, both `varlen` arms, and its NaN-padding out-of-bounds detector — plus the `test_mla_backends.py` shape sweep, all at the REAL V2-Lite geometry (576/512/64, block 16, the mscale^2-corrected scale, reverse-interleaved block tables so any page-stride assumption fails): ragged 1/15/16/17/255/256/257/300, multi-block, single-block single-token, EVERY split boundary ({1,2,3,4,5,8,16,17,64,300,512} incl. splits > seq_len, the empty-split path both stages must skip), 128-head V3 (head_tiles > 1), head counts 1/3/17 that do not fill a `BLOCK_H` tile, a 288/256 block-32 geometry, bf16 + f32, run-to-run BIT-exact over 5 runs. dgx/sm_121: **11/11 cases, 2,303,193 assertions**, with the 7 CUDA cases proven to EXECUTE (2,106,080 alone) and the output pre-poisoned to NaN. Memory safety run DELIBERATELY per the W3 lesson: memcheck **0 errors**, and because this is a `__syncthreads`-coordinated shared-memory kernel (a class memcheck does NOT cover) racecheck **0 hazards** and synccheck **0 errors** too. `TritonMLABackend::get_impl_cls()` now returns a real `TritonMLAImpl` + `MLACommonMetadata`; PREFILL is W5 and `forward()` refuses a prefill-shaped batch BY NAME. Recorded deviations: `RegisterOp` table not `cuda_arch_tactics` (no per-arch tactic exists to choose between yet — a W9 concern); fp8 KV cache + `logit_cap` NOT ported (out of scope / unreachable, both refused); `seq_len == 0` writes zeros; `TritonMLAImpl::queue` replaces torch's ambient stream. Gated: clean dgx CUDA build **0 warn / 0 err**; **27B 235/235 + 35B 315/315 + Coder 6/6 + Qwen3-dense 16/16 + OPT 6/6 UNCHANGED**; `benchmark_binding=false` (decode perf is W9). Owns additionally: `include/vt/ops.h`, `src/vt/ops.cpp`, `src/vt/cpu/cpu_mla_attn.cpp` (new), `src/vt/cuda/cuda_mla_attn.cu` (new), `include/vllm/v1/attention/backend.h`, `src/vllm/v1/attention/backend.cpp`, `tests/vt/test_ops_mla_attn.cpp` (new), `tests/vllm/v1/attention/test_attn_backend_registry.cpp`, `CMakeLists.txt`, `tests/CMakeLists.txt`. **NO MLA PREFILL, NO MLA MODEL, NO FORWARD, NO WEIGHT ABSORPTION — no model row moves off `SPIKE`.** **W5 LANDED 2026-07-22** (worktree `agent-a43819c9776dc1585`, base `5395203`, dgx scratch `~/w5mla`; rows STAY `SPIKE`). **W5 = MLA PREFILL + the CHUNKED-CONTEXT LOOP.** Three new `vt::` ops plus a device-agnostic driver, each `file:line`-cited both sides: `vt::MlaPrefillAttention` <- `mla/prefill/flash_attn.py:153-248` (`FlashAttnPrefillBackend`), the ONLY MLA prefill backend reachable on sm_121 (`mla/prefill/selector.py:66-76`, which HARD-RAISES with no fallback at `:191-194`) and the one W0 OBSERVED the oracle logging; `vt::GatherMlaCache` <- `csrc/libtorch_stable/cache_kernels.cu:992-1064`; `vt::MergeAttnStates` <- `csrc/libtorch_stable/attention/merge_attn_states.cu:18-192`; and the loop itself in the NEW header `include/vllm/model_executor/layers/attention/mla_chunked_context.h` <- `mla_attention.py:1422-1451` (workspace sizing), `:1667-1745`+`:1837-1855` (the chunk grid), `:2094-2199` (`_compute_prefill_context`), `:2344-2425` (`forward_mha`). **THE FA-2 LAUNCHER WAS GENERALIZED — W4's prediction held, and the reason is that upstream does not ask FA-2 for asymmetric head dims either:** `requires_v_padding` is TRUE on GB10 (`flash_attn.py:88-99`), so V is ZERO-PADDED 128->192 and the output sliced back (`:164-168`, `:196-197`), leaving a plain SYMMETRIC head_dim-192 kernel. The whole generalization is (a) two new explicit instantiations of the UNCHANGED generic `run_mha_fwd_splitkv_dispatch`, (b) a NEW entry point `LaunchMlaPrefillFA2Bf16` for the CONTIGUOUS-varlen mode (`cu_seqlens_k` instead of `block_table`+`seqused_k`, already supported at `flash_fwd_kernel.h:584-590`), (c) the pad/slice pair. **Existing models proven byte-identical STRUCTURALLY and BY GATE:** the diff of `cuda_flash_attn_fa2.cu` is 211 insertions / **0 deletions**, the vendored FA-2 tree gains only 2 files, and `LaunchPrefillFA2Bf16` — the paged launcher every non-MLA prefill calls — is textually untouched; 27B 235/235 + 35B 315/315 + Coder 138/138 + Qwen3-dense 664/664 + OPT 36/36 ALL UNCHANGED. **Evidence is UNIT-ONLY and held to the W4 bar:** `tests/vt/test_ops_mla_prefill.cpp` **4/4 cases / 2,377,052 assertions** (port of `test_mla_backends.py` + the bf16 arm of `test_mla_prefill_quant_output.py`; its fp8 arms are NOT ported and say so — they need device-capability family 100, unreachable on sm_121) and `tests/vt/test_ops_mla_chunked_context.cpp` **5/5 / 306,037 assertions**, both on dgx sm_121 at the REAL V2-Lite prefill geometry (QK 192 = 128 nope + 64 rope, V 128, latent 576, block 16, mscale^2 scale). The oracle is INDEPENDENT (double-precision TWO-PASS softmax, a different algorithm from FlashAttention's streaming online-softmax), and the chunked loop is additionally gated against a SINGLE-SHOT whole-sequence oracle that NEVER CHUNKS — exact / +1 / -1 chunk boundaries, a request with no context at all, a chunk in which a request contributes ZERO keys, ragged multi-chunk, V3's 128 heads; ADVERSARIAL reverse-interleaved block tables throughout; NaN-poisoned outputs; run-to-run BIT-exact over 5 runs. memcheck **0 errors**, racecheck **0 hazards**, synccheck **0 errors** on BOTH binaries; clean dgx CUDA build **0 warn / 0 err**. **An upstream FA-2 QUIRK was found and worked around, recorded as a deviation:** with `unpadded_lse = true` the MAIN path writes LSE at the unpadded offset (`flash_fwd_kernel.h:1038-1041`) but the EMPTY-K early exit (`:1030-1043`) ignores the flag and writes at the PADDED offset — and a zero-key request IS reachable in the chunked loop (the very case `merge_attn_states.cu:100-106` documents), so the mixed layout would clobber valid rows and write out of bounds. We run with `unpadded_lse = false` into a `[b,h,max_seqlen_q]` scratch and convert, normalizing FA-2's `+INFINITY` empty-row LSE to `-inf` (the value the merge normalizes it to anyway). Other deviations: `MergeAttnStates` is SCALAR not 128-bit-packed (same arithmetic, any head_size/stride; vectorization is W9); the `kv_b_proj` up-projection is a CALLBACK because it is a MODEL weight (W6); DCP/context-parallel and fp8-prefill NOT ported (single-GPU; capability-family-100 only). Owns additionally: `src/vt/cpu/cpu_mla_prefill.cpp` (new), `src/vt/cuda/cuda_mla_prefill.cu` (new), `src/vt/cuda/flash_attn/src/flash_fwd_split_hdim192_bf16{,_causal}_sm80.cu` (new), `include/vllm/model_executor/layers/attention/mla_chunked_context.h` (new), `tests/vt/test_ops_mla_prefill.cpp` (new), `tests/vt/test_ops_mla_chunked_context.cpp` (new), plus append-only additions to `include/vt/ops.h`, `src/vt/ops.cpp`, `src/vt/cuda/cuda_flash_attn_fa2.cu`, `CMakeLists.txt`, `tests/CMakeLists.txt`. **NO MLA MODEL, NO MLA FORWARD, NO WEIGHT ABSORPTION — no model row moves off `SPIKE`.** **W6 LANDED 2026-07-22** (base `2846467`; rows STAY `SPIKE` — W6 adds an attention LAYER, not a model). **The MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION** — the layer that finally COMPOSES W3's cache write, W4's MQA decode and W5's MHA prefill ([`mla_attention.h`](../include/vllm/model_executor/models/mla_attention.h), [`mla_attention.cpp`](../src/vllm/model_executor/layers/attention/mla_attention.cpp)): the projections with BOTH `q_lora_rank` branches (`fused_qkv_a_proj` -> `q_a_layernorm` -> `q_b_proj`, or the direct `q_proj`), the two RMSNorms (the rope part deliberately NOT normed, `deepseek_v2.py:516`), the DECOUPLED RoPE (`is_neox_style=False`, only the 64-dim trailing slice rotates) with its YaRN cos/sin cache and the SEPARATE mscale^2 softmax-scale correction, the `kv_b_proj -> W_UK/W_UV` split, the prefill-MHA / decode-MQA dispatch (decode tokens packed FIRST, `mla_attention.py:700-709`), and the `kv_b_proj` up-projection callback W5 left open. **Absorption needed NO new attention kernel** — it is a load-time weight transform plus TWO BATCHED GEMMs exactly as the spike's §2.2 predicted — so the whole new-kernel surface is two general primitives: `vt::BatchedMatmul` <- `torch.bmm` (`mla_attention.py:789`, `:1034`; on CUDA torch resolves that to cuBLAS `gemmStridedBatchedEx`, and ours is the cuBLASLt strided-batched form of the same GEMM) and `vt::ConcatMlaNopeRope` <- `concat_mla_q` (`csrc/libtorch_stable/concat_mla_q.cuh` + `cache_kernels.cu:1555-1600`) generalized to also serve `_concat_k_nope_k_pe` (`:2063-2092`). **THE ABSORBED-vs-UNABSORBED EQUIVALENCE IS PROVEN NUMERICALLY, THREE WAYS, not argued:** (1) an INDEPENDENT double-precision block oracle computes the attention BOTH ways and they agree to **< 1e-11** (the identity itself); (2) our absorbed decode reproduces the UNABSORBED oracle to **< 2e-4** in f32; (3) — the strongest — the SAME batch is driven once through our ABSORBED MQA decode kernel (QK 576 / V 512, one KV head, K/V never materialized) and once through our UNABSORBED materialized-MHA prefill path (QK 192 / V 128 over `kv_b_proj` output plus the chunked-context loop), agreeing to **< 3e-4** (CPU f32) / **< 4e-2** (CUDA bf16) — nothing but the weights is shared, so an absorption bug cannot cancel out. Evidence on dgx sm_121: `test_mla_attention_block.cpp` **10/10 cases / 2,372,644 assertions** and `test_ops_mla_absorb.cpp` **9/9 / 1,644,807 assertions** (CUDA cases proven to EXECUTE, 124,941 + 290,835 when run alone; NaN-poisoned outputs; run-to-run BIT-exact). Ports `tests/kernels/test_concat_mla_q.py` (both arms, incl. the transposed-nope case, compared bit-exactly). Clean dgx CUDA build **0 warn / 0 err**; `compute-sanitizer` memcheck **0**, racecheck **0 hazards**, synccheck **0** (the last needing `--num-cuda-barriers 65536` — the default barrier table OVERFLOWS on a binary that drives this many kernel families and the tool then reports a bogus `unspecified launch failure`; recorded because it looks exactly like a real defect). **REGRESSION SET UNCHANGED: 27B 235/235, 35B 315/315, Qwen3-Coder 138/138, Qwen3-dense 664/664, OPT 36/36.** **Coverage gap stated plainly: the `q_lora` query branch has NO e2e coverage and cannot get any here** — DeepSeek-V2-Lite has `q_lora_rank=null`, so it is UNIT-GATED ONLY (at DeepSeek-V3's real 7168/128-head/1536 dimensions); GLM-4.7-Flash (`q_lora_rank=768`, 58.2 GiB, FITS GB10) is what would close it later and was deliberately NOT attempted. Deviations: the A-projections are issued per weight ROW-SLICE rather than one fused GEMM (`vt::RmsNorm` needs contiguous inputs; the checkpoint PACKING is unchanged and this is the dense block's own default — a fused A-GEMM is a W9 A/B); `vt::ConcatMlaNopeRope` is SCALAR not 128/256-bit vectorized (a concat is a pure copy, so the bytes are identical; W9); and two ADDITIVE relaxations — `vt::RopeFromCache` stride-driven on q/k and `vt::MatmulBT` accepting a row-strided ACTIVATION — both integer-identical for contiguous tensors and so bit-identical for every existing model by construction AND by gate. An upstream ORDERING INVARIANT was found and documented: within the prefill tail the WITH-CONTEXT requests must come FIRST (`prefill_tokens_with_context = query_start_loc[num_prefills_with_context]`, `:1806-1810`), which W7 must honour. **NO MLA MODEL, NO MODEL FORWARD, NO REGISTRY ENTRY — no model row moves off `SPIKE`.** Owns additionally: `include/vllm/model_executor/models/mla_attention.h` (new), `src/vllm/model_executor/layers/attention/mla_attention.cpp` (new), `tests/vt/test_ops_mla_absorb.cpp` (new), `tests/vllm/model_executor/layers/attention/test_mla_attention_block.cpp` (new), plus additive edits to `include/vt/ops.h`, `src/vt/ops.cpp`, `src/vt/cpu/cpu_ops.cpp`, `src/vt/cuda/cuda_matmul.cu`, `src/vt/cuda/cuda_mla_attn.cu`, `src/vt/cuda/cuda_ops.cu`, `CMakeLists.txt`, `tests/CMakeLists.txt`. **W7 LANDED 2026-07-22** (base `ce43c51`; dgx build + gates at `~/w7mla_src`). **THE DEEPSEEK-V2 MODEL — registry + config parse + loader + forward.** The first MLA MODEL in the tree: [`deepseek_v2.h`](../include/vllm/model_executor/models/deepseek_v2.h), [`deepseek_v2_weights.cpp`](../src/vllm/model_executor/models/deepseek_v2_weights.cpp), [`deepseek_v2.cpp`](../src/vllm/model_executor/models/deepseek_v2.cpp), [`deepseek_v2_registry.cpp`](../src/vllm/model_executor/models/deepseek_v2_registry.cpp), plus the ONLY shared-code edit — a two-line additive condition in `runner.cpp` recognising a `KVCacheSpecKind::kMlaAttention` group as the model's attention group (upstream maps MLA onto the ordinary `FullAttentionManager`, `single_type_kv_cache_manager.py:1539`, so block tables/prefix caching/eviction are untouched). **LOADER GATE PASSED on the real 4-shard checkpoint: 5291/5291 tensors accounted for, ZERO unmapped and ZERO leftover** (`test_deepseek_v2_load.cpp` 4/4 cases / 37,331 assertions), every shape asserted including the LOAD-TIME `kv_b_proj -> W_UK_T [16,128,512]` / `W_UV [16,512,128]` absorption split (mla::AbsorbKvBProjBf16 applied while the checkpoint bytes are in hand — the same transform, at the same lifecycle point, as upstream's `process_weights_after_loading`, `mla_attention.py:875-962`). **V2-Lite takes the DIRECT `q_proj` query branch** (`q_lora_rank: null`, `deepseek_v2.py:1028-1034`) — asserted, with `fused_qkv_a_proj`/`q_a_layernorm`/`q_b_proj` EMPTY on every layer; the fused branch is implemented (the loader un-fuses `packed_modules_mapping["fused_qkv_a_proj"]`) but stays e2e-uncovered on this box, exactly as the spike §5.1 recorded. **FORWARD GATE PASSED and obviously right, not merely finite: the real DeepSeek-V2-Lite prefill of `The capital of France is` -> argmax ` Paris`** (top-5 ` Paris`, ` the`, ` a`, ` one`, ` also`; run-to-run bit-exact) — the direct analogue of the Qwen3-Coder W3 sanity case. **BATCH-ORDERING GATE:** `BuildMlaBatchSplit` is a PURE HOST reproduction of `split_decodes_and_prefills` (`mla_attention.py:1640-1649`, `reorder_batch_threshold == 1` at `:1420`) + `prefill_tokens_with_context` (`:1806-1810`) that VALIDATES both orderings instead of assuming them — a decode after a prefill THROWS and a with-context prefill after a context-free one THROWS, each naming the request index and citing the upstream line; W6 measured **0.86 relative error** from exactly the second violation, which is now unrepresentable rather than silent. **SHARED EXPERTS (new for this family) gated TWO ways:** DeepSeek's shared expert is a plain MLP ADDED to the routed sum (`deepseek_v2.py:344-357`, `moe_runner.py:407`) with NO sigmoid gate unlike Qwen3.6's — (1) with every routed expert zeroed a MoE layer is BIT-IDENTICAL (`memcmp`) to a dense layer holding the same MLP, (2) turning the shared expert off CHANGES the logits, so the path demonstrably runs. **THE CUDA PATH IS EXERCISED, NOT MERELY COMPILED:** a CUDA case at the REAL MLA head geometry (QK 192 / V 128 / 576 latent — the only head_dim W5's FA-2 MLA prefill is instantiated for) drives the CUDA MLA kernels, `vt::ConcatAndCacheMla`, `vt::BatchedMatmul` and the CUDA-ONLY grouped bf16 MoE GEMM branch, bit-exact run to run on device and agreeing with the CPU reference path to **0.0061 worst relative logit error**; `test_deepseek_v2_forward.cpp` **11/11 cases**; `compute-sanitizer` memcheck **0**, racecheck **0 hazards**, synccheck **0** (`--num-cuda-barriers 65536`). Clean dgx CUDA build **0 warn / 0 err**. **A LATENT TREE-WIDE HAZARD WAS FOUND (pre-existing, NOT MLA):** the shared `DevicePool` (`device_pool.h`) is a process-wide singleton keyed only on a byte size class and documents itself "backend-agnostic" — safe for the engine (one device per process) but NOT for a test binary running both a CPU and a CUDA forward, which hands the second backend the first's recycled pointers (observed as a SIGSEGV dereferencing a CUDA block on the host). Worked around TEST-LOCALLY with a per-arm `DevicePool` via the existing `ActivePoolScope`; the hazard itself is recorded, not papered over. **DEVIATIONS:** (1) **only `DeepseekV2ForCausalLM` is REGISTERED, not the four aliases the plan named** — `DeepseekForCausalLM` is plain MHA (`deepseek_v2.py:1201-1211`), V3 is fp8/671B (no bf16 loader path, no hardware), V3.2 needs the DSA indexer; all three are REFUSED BY NAME in the config parse and W10 owns their honest rows; (2) the MoE block does NOT reuse `RunMoeBlock` — DeepSeek's router is grouped and its shared expert ungated, so reusing Qwen's block would have applied Qwen's semantics; writing it directly over the same `vt::` ops is also why the regression set is UNCHANGED despite the shared-expert path activating; (3) `routed_scaling_factor` scales the ROUTING WEIGHTS rather than the combined routed OUTPUT (same function, different rounding order; V2-Lite has 1.0 so they are bit-identical on the W8 vehicle); (4) the chunked-prefill workspace is sized from `config.max_position_embeddings` + this step's `num_reqs` because a model forward has no `VllmConfig`; (5) no decode CUDA-graph sibling and no speed number (W9). **REGRESSION SET UNCHANGED. NO TOKEN-EXACT GATE — that is W8 — so NO MODEL ROW MOVES OFF `SPIKE`: a loading, forwarding model is not a supported model.** Owns additionally: the four `deepseek_v2*` files (new), `src/vllm/v1/worker/gpu/runner.cpp`, `tests/vllm/models/test_deepseek_v2_load.cpp` (new), `tests/vllm/models/test_deepseek_v2_forward.cpp` (new), `tests/vllm/models/test_model_registry.cpp` + `tests/vllm/test_model_loader_gguf.cpp` (registry-inventory updates: 5 -> 6 architectures), `CMakeLists.txt`, `tests/CMakeLists.txt`.| `ACTIVE` (W0-W10 landed; the W-plan is COMPLETE — the claim stays open only because the V2 row is `ACTIVE`, not `DONE`) | 2026-07-22 (W0+W1, W2+W3, W4, W5, W6, then W7, committed on worktree branches; **NOT pushed**. Next: **W8 — the SACRED token-exact gate on DeepSeek-V2-Lite**: wire the paged engine to produce the MLA batch order `BuildMlaBatchSplit` already validates (decodes first; with-context prefills leading the prefill tail), capture goldens from the vLLM 0.25.0 oracle and run the STRICT token-exact form W0 determined, with the regression set unchanged and memcheck 0. Superseded next-step, kept for context: **W7 — the DeepSeek-V2 model TU**: registry (all four aliases; `DeepseekForCausalLM` takes the MHA branch), config parse, the MLA-only KV group, the per-expert bf16 loader with shared experts and the `kv_b_proj -> W_UK/W_UV` absorption applied at LOAD time, and the forward composing W6's MLA block with `RunMoeBlock` and the first `first_k_dense_replace` dense layers. It must honour the W6-documented ordering invariant (with-context prefills first). Superseded next-step, kept for context: W6 — the MLA attention BLOCK + load-time weight absorption. Superseded next-step, kept for context: W5 — MLA prefill: generalize the vendored FA-2 varlen launcher to qk 192 / v 128 (GB10's MLA prefill backend is `FLASH_ATTN`, confirmed at W0), mirror `requires_v_padding` (V zero-padded 128 -> 192, sliced back), and implement the workspace-bounded chunked-context loop with the LSE merge; ports of `test_mla_backends.py` + `test_mla_prefill_quant_output.py`) | **W8 LANDED 2026-07-22 — THE SACRED CORRECTNESS GATE PASSES 8/8, and the row `MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm` moves `SPIKE` -> `ACTIVE` (correctness COMPLETE, speed PENDING). It does NOT reach `DONE`: `DONE` requires vLLM-speed parity on every axis, which is W9 and has no number yet. The other four rows in this claim STAY `SPIKE`.** Result: **8/8 prompts PASS — STRICT token-exact 5/8, near-tie band 3/8, 92/128 tokens strictly exact, max teacher-forced gap 0.25 nats, 0 forward-divergent** (223 assertions). The bar was ARRIVED AT by measurement, not chosen: vLLM is DETERMINISTIC here at batch=1 (W0's K=5 8/8, re-confirmed by W8's capture at T=16 with 0 multi-valued cells), so the STRICT form ran FIRST and came out 5/8; the ratified TEACHER-FORCING diagnostic then showed 36 divergent positions with **35 at gap EXACTLY 0.0000 nats** (vLLM's own argmax on OUR prefix IS our token), **one root flip at 0.2500 nats** inside the ratified 0.5-nat band, and **ZERO tokens outside vLLM's top-20** — with the per-position nats COMMITTED as goldens. **W8's first job — the scheduler/runner wiring — needed NO new code:** `runner.cpp:671` already reorders with `decode_threshold = 1`, exactly MLA's `reorder_batch_threshold`, and its `decode -> short_extend -> long_extend -> pure_prefill` ordering satisfies both MLA invariants; W8 PROVES that end to end with new diagnostic `MlaBatchSplitStats` counters (phase 2: **MIXED=7 steps at max_num_reqs=8** with `BuildMlaBatchSplit` never throwing; phase 3: a prefix-cache-driven **with-context prefill**) rather than duplicating the reorder. **The real blocker was the TOKENIZER:** the first run REFUSED to load (`unsupported normalizer Sequence`) and behind it sat a whole NEW pre-tokenizer family — DeepSeek's HF `Sequence` PIPELINE of SEVEN stages — landed as `SplitPattern::kDeepSeek` and MEASURED token-for-token against the REAL HF `tokenizers` library (6/6 cases / 2461 assertions). **ADDITIONALLY OWNED (new files/edits beyond the campaign's usual surface, flagged because they are SHARED code):** `include/vllm/tokenizer/pretokenizer.h`, `src/vllm/tokenizer/pretokenizer.cpp`, `src/vllm/tokenizer/tokenizer.cpp`, `tools/parity/dump_tokenizer.py` (a `--label` arg), `tests/vllm/test_bpe.cpp`, NEW `tests/vllm/test_tokenizer_parity_deepseek.cpp`, NEW `tests/vllm/models/test_deepseek_v2_paged_engine.cpp`, NEW `scripts/deepseek-v2-{oracle-capture,neartie-gap}.py` + `scripts/deepseek-v2-dgx-gate.sh`, NEW goldens `tests/parity/goldens/{deepseek_v2_greedy,tokenizer_deepseek_v2}/`, plus the diagnostic counters in `deepseek_v2.{h,cpp}` and the `tests/CMakeLists.txt` glue. The tokenizer edits are the shared-code risk and are gated as such: the Qwen3.6 parity corpus, `test_bpe` 17/17, `test_pretokenizer`, `test_detokenizer`, `test_input_processor` and `test_op_parity` are all UNCHANGED, as are 27B 235/235, 35B 315/315, Coder 6/6, Qwen3-dense 16/16 and OPT 6/6 (run SERIALLY). Clean CUDA rebuild 0 warn/0 err; memcheck/racecheck/synccheck 0. **W9 SPEED CLOSE LANDED 2026-07-22 — an ATTRIBUTED MISS; the row stays `ACTIVE`.** Owns additionally `src/vt/cuda/cuda_mla_attn.cu` (the split-KV occupancy fill + the capture guard) and the `DeepseekV2DecodeGraph` additions in `deepseek_v2.{h,cpp}` / `deepseek_v2_registry.cpp`. **The denominator question is SETTLED with evidence:** vLLM's auto-selected FlashInfer CUTLASS unquantized MoE has now rebooted dgx **five times** (three at W8, two more at W9 with the Qwen3-Coder mitigations applied — the second on a PRISTINE freshly-rebooted box with a 0 GiB page cache; `sudo` is password-gated here so `drop_caches` was never available), both W9 deaths at the identical post-`torch.compile` memory-profiling phase, so `--moe-backend triton` IS vLLM's best STABLE graphed configuration on this hardware and is the bar. **Two levers landed, both default-ON, gate 8/8 UNCHANGED:** the MLA split-KV occupancy fill (`VT_MLA_SPLIT_FILL=0` rolls back) — `nsys` found `MlaDecodeStage1` at 44.7% of all GPU time and ~180x off its own memory-bound floor from a 2-CTA grid at batch 1, and filling the grid made it **18.3x faster** for **+69.5%/+53.3%/+32.0%/+19.5%** end-to-end at c1/c2/c4/c8 — and the decode CUDA-graph sibling (`VT_DEEPSEEK_CUDAGRAPH=0` rolls back), worth only ~+2% because this model's decode is GPU-bound. **Grid: 0.86-0.95x vLLM output throughput (was 0.50x), TTFT now BEATS vLLM at c4/c8, TPOT/ITL pass at c2 only** — see [docs/BENCHMARKS.md](../docs/BENCHMARKS.md) § "Binding DeepSeek-V2-Lite (MLA) every-axis grid". **NEXT LEVER (named, not vague): route the batch-1 dense projections off cuBLAS `gemvx` (31.8% of our GPU time) onto a tensor-core GEMM — vLLM splits the same work `gemvx` 12.7% + `nvjet_sm121_tst_mma_*` 6.6%.** Then W10, the blocked-row honesty pass. **W10 LANDED 2026-07-22 — the BLOCKED-ROW HONESTY PASS, which closes the campaign's W-plan. Records only: no code, no kernels, no build, no GPU work, nothing downloaded, no new number claimed.** Rows moved to their FINAL HONEST state: `MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm` (V3 + V3.2, and Kimi-K2/K2.5's text backbone by config composition) `SPIKE` -> `BLOCKED` — HW-BLOCKED at ~642 GiB fp8 / ~1250 GiB bf16 (671B) against 119 GiB unified memory, and V3.2 additionally DEP-BLOCKED because for a SPARSE model the XOR filter eliminates `TRITON_MLA` and leaves `FLASHINFER_MLA_SPARSE_SM120` alone, whose sm12x dispatch goes to flashinfer's DENSE-ONLY XQA backend that discards `sparse_mla_top_k` (and upstream's own test monkeypatches the capability probe and asserts nothing numerical); `MODEL-TEXT-minimax-m2-mini-max-m2-for-causal-lm` `SPIKE` -> `BLOCKED` — HW-BLOCKED at ~428 GiB and DISJOINT from MLA (dense GQA + sliding window; belongs behind `ROAD-V1-C5`); `MODEL-TEXT-deepseek-v2-deepseek-for-causal-lm` STAYS `SPIKE` with the record repaired to say plainly that it is plain MHA and needs NO MLA at all; `MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm` STAYS `SPIKE` (MLA half unlocked, KDA is a separate kernel campaign, HW-MARGINAL at ~89.4 GiB). Both BLOCKED rows are RELEASED from this claim's row list per the record contract (a claim may only hold `SPIKE`/`ACTIVE` rows) and each states what CAN still be gated (config/registry resolution, weight-map on a slice, unit parity at real dimensions) versus what CANNOT (anything e2e, now or ever on this box). One CROSS-CLAIM edit, recorded in both claims: `MODEL-TEXT-deepseek-v2-glm-moe-dsa-for-causal-lm` (GLM-5) `SPIKE` -> `BLOCKED` under `CLAIM-GLM-DSA-LATEST-DEEPSEEK`, for the same DSA dependency dead-end plus 1404.2 GiB. **The two PERMANENT coverage gaps are now stated in the rows, not only in the spec:** the `noaux_tc` grouped router and the `q_lora` query branch each have NO e2e coverage on this hardware and are UNIT-GATED ONLY, because V2-Lite is `n_group=topk_group=1`/softmax with no `e_score_correction_bias` and `q_lora_rank=null`. **NAMED NEXT VEHICLE: GLM-4.7-Flash** (`Glm4MoeLiteForCausalLM`, 31.2B / 58.2 GiB, FITS GB10) — the only reachable checkpoint that closes BOTH gaps, and upstream a zero-override subclass of DeepSeek-V2's MLA attention. The full campaign DEVIATIONS LEDGER (upstream FA-2 empty-K/`unpadded_lse` defect worked around; scalar `MergeAttnStates` and `ConcatMlaNopeRope`; per-row-slice A-projections; `cuda_arch_tactics` deferral; DCP and fp8-prefill not ported; the split-KV occupancy deviation; the pre-existing process-wide `DevicePool` byte-size-class hazard) is now discoverable from the V2 row itself. **BLOCK CLOSURE: NOT YET.** Every row in this block is at its final honest state, but `MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm` is `ACTIVE`, not `DONE` — `DONE` needs vLLM-speed parity on EVERY axis, and the grid is 0.86-0.95x on throughput. Per the closing rule ("when EVERY row in an execution block is `DONE`, move the block plan/report to `.agents/completed/`") the plan/spec STAYS LIVE and NOTHING is archived. The claim remains open with exactly one open item: the named `gemvx` -> tensor-core dispatch lever. | `CLAIM-KERNEL-CPU-ELEM-GEMM-1` | `KERNEL-GEMM-CPU-ELEM` (work rows **E1-E4**) — the elementwise f32/f16/bf16 CPU GEMM: hoist the per-element dtype switch out of the K loop, replace the single serial f32 accumulator with 16 independent ones, add bit-exact per-arch SIMD tiers with a runtime probe, and M-block the `[N,K]` micro-kernel. Promoted to the #1 CPU lever by the CIQ G4 claim's own measurement (60 % of the bench file's weight bytes are `f16`, which no block encoding covers) | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a0080369412fcd2a7`; dgx CPU source/build `~/work/vllm.cpp-elemgemm`, CUDA gate build `~/work/vllm.cpp-elemgemm-cuda`; benchmark host dgx `~/work/bench-cpu-llama` (the x86 dev box is VOID for timing) | `main` (worktree), base `72f5db2` | Owned files: new `src/vt/cpu/cpu_matmul_elem.{h,cpp}`, `src/vt/cpu/cpu_ops.cpp` (`MatmulOneChunk` ONLY), new `tests/vt/test_ops_matmul_elem.cpp`, `CMakeLists.txt` + `tests/CMakeLists.txt` (one source and one test row), the new leaf spec `.agents/specs/cpu-elementwise-gemm.md`, the `KERNEL-GEMM-CPU-ELEM` row in `kernel-matrix.md`, the `QUANT-GGUF-CIQ-GEMM` row in `quantization-matrix.md`, roadmap, ledger, state, README §CPU, BENCHMARKS. MUST NOT touch a model forward, a loader, the quant GEMM, or any golden; MUST NOT trade bit-exactness for speed. | `DONE` | 2026-07-22 — **BIT-EXACT AND 3.1-3.4× FASTER.** Two defects removed from the elementwise chunk worker: the per-element `LoadF32` dtype switch is gone (the activation row is widened to f32 once per 16-row tile, the weight dtype resolves once per chunk into a typed micro-kernel), and the ONE serial f32 accumulator became 16 — the old 0.77-0.84 GFLOP/s/thread was a loop-carried FP-add latency chain, not an ISA limit. SIMD is ported from llama.cpp `vec.cpp:139,264` (`ggml_vec_dot_bf16`/`_f16`) with ONE recorded deviation: we vectorize across OUTPUT COLUMNS instead of along K, so every output keeps its sequential reduction and the whole change is **BYTE-IDENTICAL** to the historical kernel. Tiers: portable 16-accumulator (always built), AArch64 NEON (`vshll_n_u16`/`vcvt_f32_f16` widen, `vtrnq_f32` 4×4 transpose, MR=4), x86-64 SSE2 + probed F16C (MR=2). **CORRECTNESS:** new [test_ops_matmul_elem](../tests/vt/test_ops_matmul_elem.cpp) 5 cases / **654 assertions**, gate is `memcmp` not `Approx`, green on x86-64 AND dgx **aarch64** and under all three `VT_CPU_MATMUL_TIER` settings; the exhaustive case proves `vcvt_f32_f16`/`_mm_cvtph_ps` agree with `vt::F16ToF32` on **all 65,536 patterns including every inf/NaN** — no sNaN quieting, so no divergence to quantify and the NMSE ≤ 5e-4 fallback was not needed. E2E the BEFORE / AFTER / `VT_CPU_REF=1` token streams are one md5 `d235db12f2cd304007530286a1755c95` — the same md5 G4 recorded. **REGRESSIONS ALL UNCHANGED**, each STANDALONE on a clean CUDA `-Werror` build (**0 warnings**, production flags, one `flock $HOME/gpu.lock`, `git archive` transfer, goldens md5 `2965ef5772b556d3f3f86fedf4221b2f` identical before/after): 27B **235/235**, 35B **315/315**, Qwen3-Coder **6/6**, Qwen3-dense **16/16**, OPT **6/6**, DeepSeek-V2 **223/223 assertions (8/8 prompts)**, gguf **28/28**, plus `test_gguf_keep_quant`, `test_ops_quant_dot`, `test_ops_quant_traits`, `test_gguf_dequant`. ONE TRANSIENT RECORDED HONESTLY: the first DeepSeek-V2 pass returned the known co-scheduled-memory abort signature (95 of 223 assertions, 1 failure) while ANOTHER agent was running 27B gates on the same box; re-run STANDALONE under the flock it passes 223/223 — the same signature the CIQ G3 record documents, not a regression. Dev box full CPU ctest **157/157** (two co-scheduled HTTP-server flakes, both green standalone). **BENCHMARK (binding, idle dgx aarch64, one flock, SAME binary, 3 reps, medians):** prefill 21.67 -> **73.97 t/s (3.41×)**, decode 7.649 -> **23.79 t/s (3.11×)**, peak RSS unchanged 6.401 GiB; vs llama.cpp (pp128 173.28±1.75, tg32 24.52±0.45, RSS 2.798 GiB) **8.00× -> 2.34× behind on prefill and 3.21× -> 1.03× behind on decode — decode is AT PARITY within 3.1 %**, RSS still 2.29× worse. Op-level bf16 `[N,K]`: ref 18-24 -> portable 35-52 -> NEON 69-351 GFLOP/s (prefill shapes 14.4-14.5×), which puts the elementwise GEMM (347-351) level with the quant tier-0 (388-417). **RE-RANKS THE PLAN AGAIN, on a measured NEGATIVE:** E4's M-blocking gave 1.63× op-level on prefill and **0.0 % end-to-end**, so prefill is NO LONGER elementwise-GEMM-bound and the 95.37 % `kMatmul` attribution is STALE. Next step is a FRESH op-dispatch profile of the current binary before any further lever is started; RSS (loader **L5**) is now the largest single deficit at 2.29×; decode needs no further kernel work. Claim released | -| `CLAIM-MULTIMODAL-TRACK` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (this spike advanced it `INVENTORIED`→`SPIKE`; now `ACTIVE` — text path STRICT 32/32 gated per `CLAIM-GEMMA4-G1B`, mm towers pending), `MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation` (`INVENTORIED`→`SPIKE`). The two Qwen3.6 mm rows (27B `Qwen3_5ForConditionalGeneration` + 35B MoE) stay `PARTIAL`/narrative-only — annotated with the mm-completion plan, NOT claimed as work rows here (the mm implementation re-claims them at M3). | Claude Code (opus-4-8) | isolated worktree `/tmp/claude-1000/.../scratchpad/mm-tree` (SPIKE ONLY — no build, no GPU work, no download; one oracle-metadata + safetensors-header read on dgx) | detached HEAD, base `origin/main` `72f9fb1` | SPIKE ONLY, no implementation. Owns `.agents/specs/multimodal-track.md` plus the four `MODEL-MM-*` rows above. Covers: the multimodal seam map (input pipeline + `*ForConditionalGeneration` wrapper + engine/encoder-cache seams, vLLM `file:line`); per-target modality + oracle + GB10-fit + checkpoint gateability; the reuse-vs-new factoring; the M0–M5 W-plan. **HEADLINE:** our GATE models are ALREADY multimodal (`Qwen3_5ForConditionalGeneration`/`…Moe…`, image+video, subclass `Qwen3VLForConditionalGeneration` + `Qwen3_VisionTransformer`) brought up text-only, so the track COMPLETES them. Oracle 0.25.0 SHIPS the mm files (`qwen3_5.py`,`qwen3_vl.py`,`qwen2_5_vl.py`,`gemma4_mm.py`), so the mm path is oracle-constructible; but the cached NVFP4 gate checkpoints are TEXT-ONLY (no `visual.*`) so Qwen3.6 mm is CHECKPOINT-gated, not HW/oracle-blocked. First vehicle = `Qwen/Qwen3-VL-4B-Instruct` (same tower). Gemma-4 = image+video+AUDIO, ≥12B mm-wrapped HF-gated + PLE/YOCO/MoE backbone → SPIKE/BLOCKED-for-now; audio reachable ONLY via Gemma-4/gemma3n (Qwen3.6 has none), deferred M5. **NO SHARED-WORK COLLISION:** records-only spike; keeps the README/BENCHMARKS footprint to a single status line (per the docs-refresh + DFlash-spike coordination), does NOT touch `dflash-spec-decode.md`. | `SPIKE` | 2026-07-25 spike committed (spec + records only). No implementation, no build, no gate. Not pushed. NEXT (future, out of this claim): M0 fetch Qwen3-VL-4B + vision-inclusive Qwen3.6 checkpoint + oracle mm reference. **SUPERSEDED (annotation 2026-07-27):** the M0-M3 work this parent spike scoped has since LANDED under the DONE/ACTIVE child increment claims (`CLAIM-MULTIMODAL-M1`, `CLAIM-MULTIMODAL-M2A`, `CLAIM-MULTIMODAL-M2BC`, the M3 image/video + audio A0-A3 claims); Qwen3-VL-4B + Qwen3.6-27B image+video are e2e STRICT 32/32 and Voxtral audio is e2e 14/14. This row is retained for provenance; the two gemma4 mm rows stay `SPIKE` (owned per the row-owner column). | -| `CLAIM-GEMMA4-MULTIMODAL` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (now `ACTIVE` — text path STRICT 32/32 gated per `CLAIM-GEMMA4-G1B`; mm towers pending), `MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation` (stays `SPIKE`; ownership transferred here from `CLAIM-MULTIMODAL-TRACK` — that spike advanced them to `SPIKE`, this one owns the dedicated Gemma-4 mm + AUDIO spec + the sharpened oracle-blocked verdict). | Claude Code (opus-4-8) | isolated worktree `/tmp/claude-1000/.../scratchpad/g4mm-tree` (SPIKE ONLY — no build, no GPU, no download; oracle-metadata reads only) | detached HEAD, base `origin/main` `64a01af` | SPIKE ONLY, no implementation. Owns `.agents/specs/gemma4-multimodal.md` (new) + the two `MODEL-MM-gemma4-*` rows. **Coordination: does NOT touch `multimodal-track.md` or the Qwen `MODEL-MM-*` rows** (owned by the concurrent Qwen3.6-video agent). **DECISIVE FINDING:** Gemma-4 mm is oracle-BLOCKED — the vision/audio towers load via Transformers `AutoModel.from_config` (`gemma4_mm.py:1040,1056`) but the dgx oracle's transformers 5.13.1 has NO `gemma4` module (measured; `gemma3n` present) → mm path unconstructible, no gate. Architecture: SigLIP vision tower (REUSES M2a scaffold) + USM-Conformer audio tower (NEW) + PLE/YOCO/Gemma-4-MoE backbone (NEW, sweep-gemma §0.1); the Unified 12B variant is encoder-free. AUDIO is the genuinely-new modality (nothing built) — land it FIRST on the smallest oracle-runnable vehicle (`whisper-small`→`Voxtral-Mini-3B` on our LANDED Mistral backbone), Gemma-4 audio (G3) reuses it; Conformer tower delta proven on Granite-Speech-2b. | `SPIKE` | 2026-07-25 spike committed (spec + records only). No implementation, no build, no gate. Not pushed. NEXT (future, out of this claim): the AUDIO track A0 (fetch whisper-small + Voxtral-Mini-3B, oracle references); Gemma-4 stays blocked until the oracle Transformers carries `gemma4`. **SUPERSEDED (annotation 2026-07-27):** the AUDIO track this spike identified as the genuinely-new work has since LANDED (A0-A3, Voxtral-Mini-3B e2e 14/14 + Whisper encoder 203/203, DONE child claims). The Gemma-4 oracle block is now DISSOLVED (the pin advanced to transformers 5.14.1, which ships `gemma4`); Gemma-4 mm is reachable, implementation pending. This row is retained for provenance; the two gemma4 mm rows stay `SPIKE`. | -| `CLAIM-MULTIMODAL-M1` | `ENG-MM-INPUT-PIPELINE` (engine-matrix, `ACTIVE`). The two Qwen3.6 mm rows — 27B `Qwen3_5ForConditionalGeneration` + 35B `Qwen3_5MoeForConditionalGeneration` — stay `PARTIAL`/narrative-only (their narrative is advanced with the landed M0/M1 input pipeline; the tower re-claims them at M2/M3). Covers the new `src/vllm/multimodal/*` + the engine mm seam + `multimodal-track.md` (M0/M1 status). | Claude Code (opus-4-8) | isolated worktree `~/work/mm-m0m1` (CPU `build-cpu` + CPU ctest); dgx `~/work/mm-m0m1-cuda` (`git archive`, CUDA `build`, all GPU under ONE `flock $HOME/gpu.lock`, big-model gates STANDALONE one at a time) | branch `mm-m0m1`, base `origin/main` `39943fc` | Multimodal **M0+M1**: the M0 oracle reference (`scripts/mm/m0_oracle_capture.py` + committed fixtures) + the M1 mm INPUT pipeline (`src/vllm/multimodal/{inputs.h,hasher.*,qwen3vl_processor.*}`), the encoder-cache engine seam (`include/vllm/v1/core/encoder_cache_manager.h` + `src/vllm/v1/core/encoder_cache_manager.cpp`), the additive inert `mm_features` on `Request`/`EngineCoreRequest`, the `extra_keys` seam on `ChunkedTokenDatabase`, and the M0/M1 status in `multimodal-track.md`. Owns ONLY those NEW files + the four additive engine edits (`include/vllm/v1/request.h`, `src/vllm/v1/request.cpp`, `include/vllm/v1/engine/types.h`, `include/vllm/v1/kv_offload/lmcache/chunked_token_database.h` + `.cpp`) + tests `tests/vllm/multimodal/test_qwen3vl_processor.cpp`, `tests/vllm/v1/core/test_encoder_cache_manager.cpp` + 3 CMake lines. **NON-COLLISION:** does NOT build the vision tower / embed-merge (M2), does NOT touch the frontier-models spike files (`sweep-kimi-minimax-glm-latest.md`, `MODEL-TEXT-*`) or DFlash. NO vision forward. | `ACTIVE` | 2026-07-25 — **M0+M1 LANDED + PROCESSOR-PARITY GATE PASS.** Processor parity 23/23 BIT-identical vs the M0 vLLM-0.25.0 oracle (pixel_values bf16, grid_thw, mm-hash `ef6f5bea...`, placeholder expansion) — RED-first (wrong shift -> 1.2M mismatches); encoder-cache 32/32; CPU inertness green STANDALONE (request/engine_types/lmcache_codec/lmcache_key_agreement/openai_conformance); clean CPU `-Werror`; `check-device-leakage` OK. SACRED CUDA text-inertness STANDALONE under flock, cutlass-ON banner CONFIRMED: **27B 235/235, 35B 315/315, Coder 138/138 ALL PASS**; clean CUDA `-Werror` 0 warnings. `benchmark_binding=false`, no speed. Not pushed. NEXT (M2): `Qwen3_VisionTransformer` forward + `_merge_multimodal_embeddings` on Qwen3-VL-4B -> first image token-exact gate. | +| `CLAIM-MULTIMODAL-TRACK` | `MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation` (`INVENTORIED`→`SPIKE`). The two Qwen3.6 mm rows (27B `Qwen3_5ForConditionalGeneration` + 35B MoE) stay `PARTIAL`/narrative-only — annotated with the mm-completion plan, NOT claimed as work rows here (the mm implementation re-claims them at M3). The Gemma-4 mm model row was DROPPED from this claim 2026-08-06 by the live-state audit (`ACTIVE`→`READY`, no Git evidence for the claim); see the Closing and archival note. | Claude Code (opus-4-8) | isolated worktree `/tmp/claude-1000/.../scratchpad/mm-tree` (SPIKE ONLY — no build, no GPU work, no download; one oracle-metadata + safetensors-header read on dgx) | detached HEAD, base `origin/main` `72f9fb1` | SPIKE ONLY, no implementation. Owns `.agents/specs/multimodal-track.md` plus the four `MODEL-MM-*` rows above. Covers: the multimodal seam map (input pipeline + `*ForConditionalGeneration` wrapper + engine/encoder-cache seams, vLLM `file:line`); per-target modality + oracle + GB10-fit + checkpoint gateability; the reuse-vs-new factoring; the M0–M5 W-plan. **HEADLINE:** our GATE models are ALREADY multimodal (`Qwen3_5ForConditionalGeneration`/`…Moe…`, image+video, subclass `Qwen3VLForConditionalGeneration` + `Qwen3_VisionTransformer`) brought up text-only, so the track COMPLETES them. Oracle 0.25.0 SHIPS the mm files (`qwen3_5.py`,`qwen3_vl.py`,`qwen2_5_vl.py`,`gemma4_mm.py`), so the mm path is oracle-constructible; but the cached NVFP4 gate checkpoints are TEXT-ONLY (no `visual.*`) so Qwen3.6 mm is CHECKPOINT-gated, not HW/oracle-blocked. First vehicle = `Qwen/Qwen3-VL-4B-Instruct` (same tower). Gemma-4 = image+video+AUDIO, ≥12B mm-wrapped HF-gated + PLE/YOCO/MoE backbone → SPIKE/BLOCKED-for-now; audio reachable ONLY via Gemma-4/gemma3n (Qwen3.6 has none), deferred M5. **NO SHARED-WORK COLLISION:** records-only spike; keeps the README/BENCHMARKS footprint to a single status line (per the docs-refresh + DFlash-spike coordination), does NOT touch `dflash-spec-decode.md`. | `SPIKE` | 2026-07-25 spike committed (spec + records only). No implementation, no build, no gate. Not pushed. NEXT (future, out of this claim): M0 fetch Qwen3-VL-4B + vision-inclusive Qwen3.6 checkpoint + oracle mm reference. **SUPERSEDED (annotation 2026-07-27):** the M0-M3 work this parent spike scoped has since LANDED under the DONE/ACTIVE child increment claims (`CLAIM-MULTIMODAL-M1`, `CLAIM-MULTIMODAL-M2A`, `CLAIM-MULTIMODAL-M2BC`, the M3 image/video + audio A0-A3 claims); Qwen3-VL-4B + Qwen3.6-27B image+video are e2e STRICT 32/32 and Voxtral audio is e2e 14/14. This row is retained for provenance; the two gemma4 mm rows stay `SPIKE` (owned per the row-owner column). | +| `CLAIM-GEMMA4-MULTIMODAL` | `MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation` (stays `SPIKE`; ownership transferred here from `CLAIM-MULTIMODAL-TRACK` — that spike advanced them to `SPIKE`, this one owns the dedicated Gemma-4 mm + AUDIO spec + the sharpened oracle-blocked verdict). The Gemma-4 mm model row was DROPPED from this claim 2026-08-06 by the live-state audit (`ACTIVE`→`READY`, no Git evidence for the claim); see the Closing and archival note. | Claude Code (opus-4-8) | isolated worktree `/tmp/claude-1000/.../scratchpad/g4mm-tree` (SPIKE ONLY — no build, no GPU, no download; oracle-metadata reads only) | detached HEAD, base `origin/main` `64a01af` | SPIKE ONLY, no implementation. Owns `.agents/specs/gemma4-multimodal.md` (new) + the two `MODEL-MM-gemma4-*` rows. **Coordination: does NOT touch `multimodal-track.md` or the Qwen `MODEL-MM-*` rows** (owned by the concurrent Qwen3.6-video agent). **DECISIVE FINDING:** Gemma-4 mm is oracle-BLOCKED — the vision/audio towers load via Transformers `AutoModel.from_config` (`gemma4_mm.py:1040,1056`) but the dgx oracle's transformers 5.13.1 has NO `gemma4` module (measured; `gemma3n` present) → mm path unconstructible, no gate. Architecture: SigLIP vision tower (REUSES M2a scaffold) + USM-Conformer audio tower (NEW) + PLE/YOCO/Gemma-4-MoE backbone (NEW, sweep-gemma §0.1); the Unified 12B variant is encoder-free. AUDIO is the genuinely-new modality (nothing built) — land it FIRST on the smallest oracle-runnable vehicle (`whisper-small`→`Voxtral-Mini-3B` on our LANDED Mistral backbone), Gemma-4 audio (G3) reuses it; Conformer tower delta proven on Granite-Speech-2b. | `SPIKE` | 2026-07-25 spike committed (spec + records only). No implementation, no build, no gate. Not pushed. NEXT (future, out of this claim): the AUDIO track A0 (fetch whisper-small + Voxtral-Mini-3B, oracle references); Gemma-4 stays blocked until the oracle Transformers carries `gemma4`. **SUPERSEDED (annotation 2026-07-27):** the AUDIO track this spike identified as the genuinely-new work has since LANDED (A0-A3, Voxtral-Mini-3B e2e 14/14 + Whisper encoder 203/203, DONE child claims). The Gemma-4 oracle block is now DISSOLVED (the pin advanced to transformers 5.14.1, which ships `gemma4`); Gemma-4 mm is reachable, implementation pending. This row is retained for provenance; the two gemma4 mm rows stay `SPIKE`. | | `CLAIM-MULTIMODAL-M2A` | `ENG-MM-VISION-TOWER` (engine-matrix, `ACTIVE`). Owns the NEW vision-tower TU + 2 additive vt ops + the tower unit gate + tower reference/weight dump scripts + fixtures. The `Qwen3VLForConditionalGeneration` model-matrix row is narrative-advanced `INVENTORIED`→`PARTIAL` (tower faithful, e2e pending); the two Qwen3.6 mm rows stay `PARTIAL`/narrative-only (the tower attaches to them at M3). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-m2a` (branch `m2a-vision-tower`); dgx build+gate `~/m2a_build` (tar-transfer, CUDA `build`, ALL GPU under ONE `flock $HOME/gpu.lock`) | branch `m2a-vision-tower`, base `origin/main` `463633d` | Multimodal **M2a — vision TOWER proven faithful**: `src/vllm/model_executor/models/qwen3_vl_vision.{h,cpp}` (`Qwen3_VisionTransformer` forward: patch-embed matmul+bias, host pos-embed bilinear-interp+reorder, 24 ViT blocks [LayerNorm + partial-rotary vision RoPE via `vt::RopeFromCache` + non-causal `vt::Attention` + tanh-GELU MLP], patch merger [LN + exact-erf-GELU + 2 FCs], DeepStack 3 post-shuffle mergers at 5/11/17 → `[196,10240]`); 2 additive vt ops `GeluTanh`/`GeluErf` (`include/vt/ops.h`, `src/vt/ops.cpp`, `src/vt/cuda/cuda_layernorm.cu`, `src/vt/cpu/cpu_layernorm.cpp`); the tower gate `tests/vllm/multimodal/test_qwen3vl_tower.cpp` + fixtures `tests/vllm/multimodal/fixtures/qwen3vl_tower/`; dump scripts `scripts/mm/m2a_tower_{ref,weight}_dump.py`; 2 CMake lines. **NON-COLLISION:** additive-only — NO model runner / model TU / registry edit (text SACRED byte-identical BY CONSTRUCTION); does NOT build the MRoPE text backbone (M2b) or the merge/e2e image gate (M2c); does NOT touch M1's `src/vllm/multimodal/*`. | `ACTIVE` | 2026-07-25 — **M2a LANDED + TOWER FAITHFULNESS GATE PASS.** 4 RED-first unit gates vs the dumped vLLM-0.25.0 tower reference (patch-embed relL2 2.1e-3, block0 6.8e-3, merger 6.5e-2, DeepStack taps 1.2e-2/3.3e-2/4.4e-2, full tower 5.1e-2; NEW-kernel gates pos-embed 2.5e-3 + rope 1.9e-3 TIGHT) — bf16-depth envelope RCA'd (smooth ~0.25%/layer, no discontinuity); RED = vision-rope disabled → block0 0.149, tower 0.75, 6 fails. Build cutlass-ON+FA2 banner CONFIRMED; clean CUDA `-Werror` 0 warnings; compute-sanitizer 0 errors on the tower + new kernels. Text inertness BY CONSTRUCTION (`git diff --stat`: 128 insertions, 0 deletions, additive vt ops only). `benchmark_binding=false`, no speed. Not pushed. NEXT (M2b): MRoPE 3-D positions + DeepStack decoder injection on the Qwen3-VL text backbone; then (M2c) merge + e2e image token-exact gate. | | `CLAIM-MULTIMODAL-M2BC` | `ENG-MM-TEXT-BACKBONE` (engine-matrix, `ACTIVE`). Owns the NEW text-backbone helper TU + its unit gate + reference dump + fixtures. The `Qwen3VLForConditionalGeneration` model-matrix row stays `PARTIAL`/narrative-only (M2b/M2c numeric contracts unit-green; the e2e image gate is the remaining wire-up). The two Qwen3.6 mm rows stay `PARTIAL`/narrative-only. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/m2bc` (branch `m2bc-multimodal`, CPU `build-cpu` + CPU ctest); dgx used only for the vLLM-0.25.0 reference dump under `flock $HOME/gpu.lock` (CPU-only oracle) | branch `m2bc-multimodal`, base `origin/main` `0eed915` | Multimodal **M2b/M2c — text-backbone numeric contracts, unit-green**: `src/vllm/model_executor/models/qwen3_vl_text.{h,cpp}` (`Qwen3VLGetRopeIndex` MRoPE 3-D positions [3,T]; `Qwen3VLMergeMultimodal` masked-scatter; `Qwen3VLComputeDeepstack` [L,T,H] scatter); the unit gate `tests/vllm/multimodal/test_qwen3vl_text.cpp` (4 RED-first gates incl. the 3-section MRoPE application through the existing `vt::RopeFromCache` mrope path) + fixtures `tests/vllm/multimodal/fixtures/qwen3vl_text/`; dump `scripts/mm/m2b_text_ref_dump.py`; 1 CMake line + 1 lib source line. **NON-COLLISION:** additive-only — NO model runner / shared dense forward / registry edit (text SACRED byte-identical BY CONSTRUCTION); the e2e image forward (VL weight loader + forked MRoPE/DeepStack decode loop + greedy) is NOT yet built. | `ACTIVE` | 2026-07-25 — **M2b/M2c NUMERIC CONTRACTS UNIT-GREEN (CPU, no weights).** 4 RED-first gates vs the dumped vLLM-0.25.0 reference (`scripts/mm/m2b_text_ref_dump.py`) PASS 85/85: get_rope_index BIT-exact [3,204] (delta −182); 3-section MRoPE application (interleaved, section=[24,20,20]) via `vt::RopeFromCache` q rel-L2 1.5e-3 / k 1.5e-3 (bf16 band), RED guard interleaved-off diverges >5e-2; DeepStack scatter + embed-merge BIT-exact; clean CPU `-Werror`. Text inertness BY CONSTRUCTION (additive TU only, shared dense forward untouched). `benchmark_binding=false`, no speed. Not pushed. **REMAINING (M2c e2e):** VL weight loader (`language_model.*`+`visual.*`) + forked VL decode (inputs_embeds + MRoPE + DeepStack inject) + greedy loop → image token-exact gate vs vLLM 0.25.0 on Qwen3-VL-4B; risk = bf16-envelope tower (M2a rel-L2 ~5e-2) vs deterministic vLLM near-ties. | | `CLAIM-MULTIMODAL-M2C` | `ENG-MM-TEXT-BACKBONE` (engine-matrix, `ACTIVE`) — the forked VL decode. Narrative-advances the Qwen3-VL model-matrix row (`PARTIAL`→IMAGE e2e WORKING; not `DONE`, speed pending). Owns the NEW VL loader + forked-decode TU + the STRICT e2e image gate + the input-ids dump. M2 CLOSED. | Claude Code (opus-4-8) | dgx build+gate `~/work/mm-m0m1-cuda` (tar-transfer via `git archive`, reused Makefiles build cache, ALL GPU under ONE `flock $HOME/gpu.lock`) | branch `feat/mm-m2c-e2e`, base `origin/main` `1cd5710` | Multimodal **M2c — e2e IMAGE forward, STRICT gate PASS**: `src/vllm/model_executor/models/qwen3_vl.{h,cpp}` (VL weight loader `model.language_model.*`+`model.visual.*`; forked greedy VL decode = embed+`Qwen3VLMergeMultimodal` scatter + 3-section MRoPE via `vt::RopeFromCache` global cache + DeepStack inject at layers 0/1/2 + paged greedy); STRICT e2e gate `tests/vllm/multimodal/test_qwen3vl_e2e.cpp`; input-ids dump `scripts/mm/m2c_e2e_inputs.py` + fixture `input_ids_i32.bin`; 3 CMake lines. One M2a fix: `qwen3_vl_vision.cpp` `cap==nullptr` DeepStack concat (an explicit deferred-to-M2c stub — M2a capture output byte-identical). **NON-COLLISION:** additive VL path gated on mm input; NO shared dense forward / model runner / registry edit (text SACRED byte-identical BY CONSTRUCTION). | `ACTIVE` | 2026-07-25 — **M2c LANDED, STRICT e2e IMAGE gate PASS 32/32.** Full pipeline (image→C++ processor→M2a tower→merge→forked MRoPE/DeepStack greedy decode) == committed vLLM 0.25.0 golden token-for-token on Qwen3-VL-4B; no near-tie needed (STRICT passed exactly, first try after the stub fix). Clean CUDA `-Werror` 0 warnings. Text inertness: additive (`git diff --stat` shared forward untouched) + SACRED regressions re-run (27B/35B/Coder). `benchmark_binding=false`, speed pending. FULL SHA in commit; NOT pushed. NEXT (M3): Qwen3.6-27B image reusing this tower+backbone+loader on the GDN-hybrid backbone, then video. | | `CLAIM-MULTIMODAL-M3` | `ENG-MM-QWEN36-VL-FORWARD` (engine-matrix, `SPIKE` — the GDN-hybrid VL forward design grounded by M3-W0, implementation OWED as M3-b). The 27B mm model (`Qwen3_5ForConditionalGeneration`) stays `PARTIAL`/text-only, narrative-only (NOT claimed as a work row here — M3-b re-claims it); its narrative is advanced with the M3-W0 checkpoint verdict + the grounded GDN-hybrid VL forward design. Owns `.agents/specs/multimodal-track.md` (M3 section), `scripts/mm/m3_oracle_capture.py`, and the committed golden/input fixtures `tests/vllm/multimodal/fixtures/qwen3_5_27b/`. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/work-m3/tree` (branch `m3-qwen36-27b-image`); dgx `~/work/m3-golden` (oracle golden under `flock $HOME/gpu.lock`, ALONE, GMU 0.6, local-ai-worker verified Exited) | branch `m3-qwen36-27b-image`, base `origin/main` `3c8af64` | Multimodal **M3-W0** — resolve the GATING FACT (vision-inclusive checkpoint) + ground the GDN-hybrid VL forward design + capture the vLLM 0.25.0 oracle golden. **KEY FINDINGS:** checkpoint = **`Qwen/Qwen3.6-27B`** (51.7 GiB uniform bf16, 333 `visual.*` tensors, NOT gated; fits GB10 after reclaiming mine-only `~/work` trees, apex/darwin untouched); 27B vision config DIFFERS (depth 27/hidden 1152/out 5120/16 heads, **EMPTY deepstack** — no DeepStack, simpler than 4B); MRoPE `[11,11,10]`/rot 64/theta 1e7; backbone 48 GDN + 16 full-attn; **the bf16 GDN-hybrid LLM loader ALREADY EXISTS** (`LoadQwen3_5Dense` routes bf16 by `.weight_packed` absence) so M3-b needs only the `visual.*` loader + the forked forward (inputs_embeds + MRoPE + merge, gated ⇒ text byte-identical). **NON-COLLISION:** records + oracle scripts + fixtures only; NO shared-forward edit landed here (M3-b owns the `qwen3_5.cpp` forked forward + greedy driver + e2e gate). | `ACTIVE` | 2026-07-25 — **M3-W0 LANDED (checkpoint resolved + fetched + oracle golden captured + design grounded).** vLLM 0.25.0 CONSTRUCTS + LOADS + RUNS the 27B mm path (encoder cache profiled 1 image item; NOT oracle-blocked); golden = 214-token input (196 image toks @ off 4), 32-token greedy, **K=5 DETERMINISTIC ⇒ STRICT** (sha256 `ead4b484…`), GB10 held 54 GiB bf16 (GMU 0.6, no OOM). Fixtures `tests/vllm/multimodal/fixtures/qwen3_5_27b/`. NO e2e gate this session — M3-b (the forked GDN-hybrid VL forward + token-exact image gate + text-inertness) is the cleanly-scoped next brick, exactly as M2 landed as M2a/M2b/M2c. Not pushed. | | `CLAIM-MULTIMODAL-M3B` | `ENG-MM-QWEN36-VL-FORWARD` (engine-matrix, `SPIKE`→`ACTIVE`). The 27B mm model (`Qwen3_5ForConditionalGeneration`) narrative advances PARTIAL(text-only)→**IMAGE e2e working** (video + speed still pending; the model-matrix row stays `PARTIAL`, NOT claimed as a work row here). Owns EXACTLY: the forked GDN-hybrid VL forward + greedy driver `Qwen3_5VLGenerateGreedy` + `BuildMropeCosSinHost` + the `mrope_cos_sin` param on `DenseForwardLayers` (`src/vllm/model_executor/models/qwen3_5.cpp`); the vision-only loader `LoadQwen3VLVisionWeights` + 4B-loader refactor (`src/vllm/model_executor/models/qwen3_vl.{h,cpp}`); the `Qwen3_5VLGenerateGreedy` decl (`include/.../qwen3_5_dense.h`); the STRICT gate `tests/vllm/multimodal/test_qwen3_5_vl_e2e.cpp` + its CMake. Owns NO text forward algorithm change (mm path gated on mm input), NO scheduler/runner, NO other model. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/m3b-vl` (branch `m3b-qwen36-vl`), base `origin/main` `abde069`; dgx build+gate `dgx.casa:~/work/m3b-vl` under `flock $HOME/gpu.lock` (sole GPU owner, local-ai-worker stopped) | branch `m3b-qwen36-vl`, base `origin/main` `abde069` | Multimodal **M3-b** — build the GDN-hybrid VL image→text forward + RUN the STRICT image token-exact gate (complete our own gate model's image path). | `DONE` | 2026-07-25 — **M3-b LANDED: STRICT image token-exact 32/32 vs vLLM 0.25.0.** Full pipeline (M1 processor → M2a tower 27B config `[196,5120]` → embed+scatter into image_token(248056) rows, no deepstack → 3-section MRoPE `[11,11,10]` interleaved on the 16 full-attn layers → GDN-hybrid backbone → paged greedy) == golden `ead4b484…` (gate 54/54 assertions). Text-inertness re-run cutlass-ON+FA2: **27B 235/235, 35B 315/315, Coder 138/138** (the 3-point fork is byte-identical on text — `mrope_cos_sin==nullptr` on every text caller). Clean CUDA `-Werror` 0 warn (Release, arch 121a, cutlass NVFP4 + FA2 ENABLED banner); compute-sanitizer on the 27B VL forward. Weights via `LoadQwen3_5Dense(shards,cfg,&queue)` direct device load + host release (no unified-pool OOM). Generated text = the coherent image-conditioned "The user wants me to identify what is in the image.\n\n1. **Analyze the image:**…". Not pushed; FULL SHA reported. Video = M3c (owed); speed unmeasured. | -| `CLAIM-MULTIMODAL-M3C` | `ENG-MM-VIDEO-FORWARD` (engine-matrix, NEW `ACTIVE` — the video path). The `Qwen3VLForConditionalGeneration` model-matrix row narrative-advances IMAGE-only → **image + video preprocessing/wiring landed + unit-gated** (row stays `PARTIAL`/`🚧` — video e2e token-exact PENDING on tower fidelity, speed pending). The two Qwen3.6 mm rows stay `PARTIAL`/narrative-only (27B-video reuses this path, owed). Owns EXACTLY: the video additions to `src/vllm/multimodal/qwen3vl_processor.cpp` + `include/vllm/multimodal/{qwen3vl_processor.h,inputs.h}` (`ProcessVideo`/`VideoSmartResize`/`ComputeVideoTimestamps`/`BuildVideoRepl`/`VideoKwargs`); the per-frame windowed attention in `src/vllm/model_executor/models/qwen3_vl_vision.cpp`; `Qwen3VLGetRopeIndexVideo` in `qwen3_vl_text.{h,cpp}`; `Qwen3VLGenerateGreedyVideo` + the `VLGenerateCore` refactor in `qwen3_vl.{h,cpp}`; the two video tests + fixtures `tests/vllm/multimodal/fixtures/qwen3vl_video/` + 4 `scripts/mm/m3c_*.py` + CMake. NO text-path TU (text SACRED byte-identical BY CONSTRUCTION); the image driver is byte-identical (proven by the image gate). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/wt-m3c-video` (branch `m3c-video`), base `origin/main` `ed4bc11`; dgx build+gate `dgx.casa:~/work/m3b-vl` (reused build cache) under `flock $HOME/gpu.lock` (sole GPU owner, local-ai-worker Exited) | branch `m3c-video`, base `origin/main` `ed4bc11` | Multimodal **M3c** — extend the image pipeline to VIDEO (frame sampling + temporal grid + timestamp-interleaved expansion + per-frame windowed tower attention + temporal MRoPE) and RUN the video token-exact gate on Qwen3-VL-4B. | `DONE` | 2026-07-25 — **M3c LANDED: video preprocessing + wiring UNIT-GATED; video e2e 22/32, divergence RCA'd OUT of the new video path.** (Gate-form follow-on RESOLVED by `CLAIM-MULTIMODAL-TOWER-FIDELITY`: the divergence is a MEASURED genuine 0.125-nat bf16 near-tie ⇒ near-tie-robust gate PASS; the "PENDING on tower fidelity / tighten envelope" note below is SUPERSEDED — the tower already f32-accumulates everywhere, no kernel change.) Video-processor UNIT gate 41/41 (pixel_values_videos BIT-exact 0/393216, RED-first 195838 mismatch on the image-duplicate mapping); video MRoPE positions BIT-exact vs vLLM (delta −48); video tower faithful rel-L2 0.072 (within the bf16 envelope, image ~0.05); video e2e 22/32 (23-token exact prefix, token-24 bf16 near-tie flip) — every discrete video element bit-exact/faithful ⇒ residual is the shared-decode bf16 tower envelope (M2a ceiling), STRICT token-exact PENDING on tower fidelity, NOT loosened. NO REGRESSION: image e2e 4B STRICT 32/32, CPU units 23/41/85; text SACRED byte-identical BY CONSTRUCTION (zero text-path TU touched). Build cutlass+FA2, clean `-Werror` RC=0, arch 121a. Not pushed; FULL SHA reported. NEXT: tighten tower bf16 envelope to close video token-exact; then 27B-video + speed. | -| `CLAIM-MULTIMODAL-TOWER-FIDELITY` | `ENG-MM-VIDEO-FORWARD` (engine-matrix, `ACTIVE`). Resolves the M3c video e2e gate FORM by MEASUREMENT and closes it. Narrative-advances the `Qwen3VLForConditionalGeneration` model-matrix row IMAGE-only-STRICT + video-PENDING → **image STRICT + video NEAR-TIE-ROBUST, both correct (speed pending)** (row stays `PARTIAL`/`🚧`). Owns EXACTLY: `tests/vllm/multimodal/test_qwen3vl_video_e2e.cpp` (STRICT→near-tie-robust gate), the new `scripts/mm/m3c_video_neartie_gap.py`, and the two new fixture goldens `tests/vllm/multimodal/fixtures/qwen3vl_video/{our_ids_i32.bin,neartie_gap_mnats_i32.bin}`. NO src/kernel/shared-op edit (text SACRED byte-identical BY CONSTRUCTION); image driver unchanged (proven by the image gate). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/mm-tower-fidelity` (branch `mm-tower-fidelity`), base `origin/main` `648a5f1`; dgx measure+build+gate `dgx.casa:~/work/m3b-vl` (reused build cache) under `flock $HOME/gpu.lock` (sole GPU owner, local-ai-worker Exited) | branch `mm-tower-fidelity`, base `origin/main` `648a5f1` | Multimodal **M3c tower-fidelity/gate-form** — MEASURE whether the video token divergence is a genuine near-tie or a real fidelity gap (teacher-force vLLM 0.25.0 on OUR prefix), then close it the right way. | `DONE` | 2026-07-25 — **MEASURED: GENUINE bf16 near-tie ⇒ near-tie-robust gate PASS (NOT a fidelity gap, NO kernel change).** The prior spike RCA was WRONG (mislocated the flip at tok24; never teacher-forced). Teacher-forcing vLLM 0.25.0 on OUR exact sequence (`scripts/mm/m3c_video_neartie_gap.py`): the SOLE divergence is ONE near-tie at **tok22** (' colorful' 33866 vs vLLM ' static' 1099, **gap 0.125 nats**, our token vLLM's 2nd of 4 tokens tied within 0.25 nats), and EVERY downstream token (tok23-31) IS vLLM's teacher-forced argmax at gap **0.0000** (the 22/32-vs-greedy is the deterministic one-token shift from that single tie). vLLM is fully self-consistent on the golden (teacher-forced argmax == golden 32/32) ⇒ a clean STRICT target, so our divergence is REAL but a genuine tie, not vLLM jitter. Tower-accumulation analysis: our tower ALREADY accumulates f32 everywhere (cuBLASLt `CUBLAS_COMPUTE_32F` GEMMs + f32 online-softmax attention + f32 LayerNorm) = matches vLLM's cuBLAS/FlashAttention, so rel-L2 0.072 is the irreducible inter-op bf16 rounding envelope, NOT a fixable numeric choice ⇒ NO kernel work (unnecessary; the correct path per the DATA). Adopted the ratified near-tie gate (mirrors olmo2/qwen3-dense/glm4): anchor `our_ids_i32.bin` + `neartie_gap_mnats_i32.bin`, PASS iff all gaps ≤ 0.5 nats — max 0.125 << 0.5 ⇒ `test_qwen3vl_video_e2e` PASS (22/22 assertions). Image e2e 4B STRICT **32/32** (the deterministic strict-pass proof, unchanged). `git diff --stat`: only the video test + new script + 2 fixtures — text SACRED byte-identical BY CONSTRUCTION (zero src/kernel/shared-op ⇒ 27B/35B/Coder re-run NOT required; no kernel ⇒ compute-sanitizer N/A). `benchmark_binding=false`, no speed. Not pushed; FULL SHA reported. | | `CLAIM-AUDIO-PIPELINE` | `ENG-MM-AUDIO-PIPELINE` (engine-matrix, NEW `ACTIVE`). The genuinely-new AUDIO modality (audio-track A0+A1) on the modality-agnostic mm spine, on the smallest oracle-runnable vehicle `openai/whisper-small`. Owns EXACTLY the NEW `src/vllm/multimodal/audio_processor.{h,cpp}` + `include/vllm/multimodal/audio_processor.h`, the `AudioKwargs`+`audio_data` additions to `include/vllm/multimodal/inputs.h`, `MultiModalHasher::HashAudioF32` (`include/vllm/multimodal/hasher.h` + `src/vllm/multimodal/hasher.cpp` refactor), `tests/vllm/multimodal/test_audio_processor.cpp` + fixtures `tests/vllm/multimodal/fixtures/whisper_audio/`, `scripts/mm/a0_audio_ref.py`, 1 `CMakeLists.txt` source line + 2 `tests/CMakeLists.txt` lines, and `.agents/specs/audio-track.md` (NEW). **NON-COLLISION:** additive-only, gated on audio mm input (NO shared forward/kernel/runner/registry edit ⇒ text+image+video byte-identical BY CONSTRUCTION); does NOT touch `multimodal-track.md` / the Qwen3.6 mm model-matrix rows (Qwen3.6-video agent) / the Gemma-4 mm model-matrix rows + `gemma4-multimodal.md` (Gemma-4 agent). Does NOT build the audio ENCODER tower (A2) or the e2e audio→text (A3). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-audio-a0a1` (branch `audio-a0a1`, CPU `build-cpu` + CPU test); dgx used only for the A0 oracle reference capture (`~/venvs/vllm-oracle`, feature extraction is CPU, no GPU/flock) | branch `audio-a0a1`, base `origin/main` `435ba70` | Audio-track **A0+A1** — ground the whisper-small vehicle + capture the audio-processor oracle reference (A0), then build the C++ audio INPUT pipeline (WAV decode → identity resample → log-mel `input_features` → placeholder expansion → mm-hash) + the inert engine seam (A1), gated by feature-parity + inertness. NOT audio→text (that is A2 encoder tower + A3 e2e). | `ACTIVE` | 2026-07-25 — **A0+A1 LANDED: audio-processor parity gate PASS 77/77.** A0: `whisper-small` confirmed oracle-constructible on dgx (transformers 5.13.1); captured the WhisperFeatureExtractor log-mel `input_features` `[80,3000]` (real torch STFT path) + mel filterbank `[201,80]` + placeholder `[0]*1500` + mm-hash `2d0c7e4c…` for a deterministic PCM16-mono-16kHz clip (`scripts/mm/a0_audio_ref.py` + committed fixtures, content-hashed). A1: the C++ `WhisperAudioProcessor` — **log-mel rel-L2 1.96e-7** vs the oracle (stated 2e-4 band, torch.stft-FFT vs our DFT summation order; transformers' own torch/numpy claim 1e-5, we sit 2 orders tighter), WAV-decode + placeholder-ids + mm-hash BIT/BYTE-exact, **RED-first** (mel-perturb 2.6e-3 / hop-161 0.70 / no-normalize 9.27 all blow the band). Inertness (shared `hasher.cpp`/`inputs.h` re-run byte-identical): image 23/23 (hasher refactor inert), video 41/41, request 71/71, encoder-cache 32/32, text 85/85; clean CPU `-Werror` 0 warn; `check-device-leakage` unchanged (32==baseline); no CUDA kernel ⇒ compute-sanitizer N/A. All seven record checkers green by bare RC. `benchmark_binding=false`, speed pending. Not pushed; FULL SHA reported. NEXT (A2): the Whisper-class audio encoder tower proven faithful in isolation, then the USM Conformer delta (Granite-Speech-2b); then A3 e2e audio→text on Voxtral-Mini-3B (LANDED Mistral backbone). | | `CLAIM-MULTIMODAL-M3D` | `ENG-MM-QWEN36-VL-FORWARD` (engine-matrix, `ACTIVE`). Completes the Qwen video modalities on our own gate model: the 27B `Qwen3_5ForConditionalGeneration` row narrative-advances IMAGE-e2e-working → **IMAGE+VIDEO e2e working** (the model-matrix row stays `PARTIAL` — speed pending). Owns EXACTLY: the video driver `Qwen3_5VLGenerateGreedyVideo` + the `VLGenerateCoreGdn` refactor of the image driver (`src/vllm/model_executor/models/qwen3_5.cpp`); the video-driver decl (`include/.../qwen3_5_dense.h`); the STRICT gate `tests/vllm/multimodal/test_qwen3_5_vl_video_e2e.cpp` + its CMake + fixtures `tests/vllm/multimodal/fixtures/qwen3_5_27b_video/`; the oracle script `scripts/mm/m3d_video_oracle_capture.py` (reuses the M3c `m3c_video_neartie_gap.py` with `--model Qwen/Qwen3.6-27B`). REUSES (verified, NOT modified) the M3c `ProcessVideo`/`BuildVideoRepl`/`Qwen3VLGetRopeIndexVideo`/per-frame windowed tower attention. NO shared TEXT-forward algorithm change (the video driver is purely additive; the shared `DenseForwardLayers`/`DenseForwardBody`/etc. UNTOUCHED per `git diff --stat`), NO scheduler/runner, NO other model. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/wt-m3d-video` (branch `m3d-27b-video`), base `origin/main` `64a01af`; dgx build+gate `dgx.casa:~/work/m3d-video` under `flock $HOME/gpu.lock` (sole GPU owner, local-ai-worker Exited) | branch `m3d-27b-video`, base `origin/main` `64a01af` | Multimodal **M3d** — run VIDEO through the 27B GDN-hybrid VL forward (reuse the landed M3c video path) + RUN the STRICT video token-exact gate (complete our own gate model's video path). | `DONE` | 2026-07-25 — **M3d LANDED: STRICT video→text token-exact 32/32 vs vLLM 0.25.0 — Qwen3.6-27B video works e2e; the Qwen video modalities (image+video) are COMPLETE.** Full pipeline (M3c `ProcessVideo` → M2a tower 27B config per-frame windowed attn `[64,5120]` → embed+scatter into video_token(248057) rows [no deepstack] → temporal MRoPE `[11,11,10]` interleaved on the 16 full-attn layers via `Qwen3VLGetRopeIndexVideo` → GDN-hybrid backbone → paged greedy) == golden (gate 27/27 assertions). Oracle `scripts/mm/m3d_video_oracle_capture.py` on the M3c synthetic clip (raw sha `8a111599…`, grid `[4,8,8]`, 64 video tokens) K=5 DETERMINISTIC (first_divergence=None) ⇒ STRICT golden; teacher-forced near-tie gaps 0.0000 nats at every position (0 divergent positions). Generated text: coherent video-conditioned "The user wants me to describe the video…I see a sequence of 6 images…". **Inertness:** 27B IMAGE e2e re-run STRICT **32/32** (54/54, the driver refactor is byte-identical on image); text SACRED **27B 235/235, 35B 315/315, Coder 138/138** byte-identical BY CONSTRUCTION (`git diff --stat`: the qwen3_5.cpp change is confined to the VL-only driver region; the shared text forward is untouched; the video path is gated on mm input ⇒ `mrope_cos_sin==nullptr` on every text caller). Clean CUDA `-Werror` 0 warn (Release, arch 121a, cutlass NVFP4+FP8+Marlin+FA2 ENABLED banner); compute-sanitizer memcheck 0 on the 27B video forward. Not pushed; FULL SHA reported. SPEED unmeasured (`benchmark_binding=false`). | -| `CLAIM-AUDIO-ENCODER` | `ENG-MM-AUDIO-ENCODER` (engine-matrix, NEW `ACTIVE`). The Whisper-class AUDIO ENCODER TOWER (audio-track A2) — the encoder half of audio understanding, proven faithful in ISOLATION on `openai/whisper-small`, consuming the A1 log-mel `input_features` `[80,3000]` → encoder hidden states `[1500,768]`. Owns EXACTLY the NEW `include/vllm/model_executor/models/whisper_audio.h` + `src/vllm/model_executor/models/whisper_audio.cpp`, `tests/vllm/multimodal/test_whisper_audio.cpp` + committed golden fixtures `tests/vllm/multimodal/fixtures/whisper_audio/enc_*`, `scripts/mm/a2_audio_encoder_{ref,weight}_dump.py`, 1 `CMakeLists.txt` source line + 2 `tests/CMakeLists.txt` lines, and its section of `.agents/specs/audio-track.md` (§0b, A2→DONE; A3 kept OWED). **NON-COLLISION:** additive-only (NO shared forward/kernel/runner/registry/other-model TU edit ⇒ text+image+video+audio-pipeline byte-identical BY CONSTRUCTION); does NOT touch `multimodal-track.md` / the Qwen3.6 mm model-matrix rows (Qwen3.6-video agent) / the Gemma-4 mm model-matrix rows + `gemma4-multimodal.md` (Gemma-4 agent); does NOT touch the A0/A1 `audio_processor.*`/`hasher.*`/`inputs.h`. Does NOT build the projector/merge or the e2e audio→text (A3), nor the USM Conformer tower (A2-follow, Granite-Speech-2b). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm-a2-audio-enc` (branch `feat/audio-a2-encoder`, CPU `build-cpu`); dgx `~/work/a2-audio` (`git archive`, CUDA `build-cuda` cutlass-ON, the A2 reference/weight DUMP on CPU in `~/venvs/vllm-oracle` NO flock, the parity GATE under ONE `flock $HOME/gpu.lock` FOREGROUND, sibling 27B NOT co-resident) | branch `feat/audio-a2-encoder`, base `origin/main` `adcac8e` | Audio-track **A2** — the Whisper-class encoder tower (2×Conv1d im2col+`vt::MatmulBT` frontend + fixed sinusoidal pos + 12 pre-norm bidirectional encoder blocks + final `layer_norm`), proven faithful in isolation vs a dumped bf16 `WhisperEncoder` reference (mirror M2a, bf16-envelope tol). NOT audio→text (A3). | `ACTIVE` | 2026-07-25 — **A2 LANDED: encoder-tower fidelity gate PASS 203/203.** C++ `WhisperAudioEncoderForward` mirrors transformers `WhisperEncoder` (`modeling_whisper.py:641-721,400-430,298-368,54` @ 5.13.1; cross-checked vLLM `whisper.py:458,353,322,473-476`): conv frontend as im2col+`vt::MatmulBT` (no new CUDA kernel), GELU-erf, k_proj no-bias, full bidirectional `vt::Attention(causal=false)`. Reference dumped in bf16 on CPU (`scripts/mm/a2_audio_encoder_ref.py`, post_pos recon maxdev 0.0); embed_positions committed as a golden constant. **GATE (GPU under flock, cutlass-ON banner CONFIRMED, sibling 27B NOT co-resident):** post_conv rel-L2 **4.7e-3**, post_pos **2.8e-3**, block0 **6.6e-3**, encoder-output **3.0e-2** (bf16-depth envelope ~0.28%/layer over 12 layers, matches M2a; bands post_conv/post_pos<8e-3, block0<1.5e-2, final<5e-2 = measured×1.6–2.3). **RED-first (revert-experiment, GPU under flock):** wrong conv-stride → post_conv 0.34 (72×, FAIL); missing sinusoid → post_pos 0.86 (FAIL); skipped final-LN → 4.22 (140×, FAIL); honest non-discriminators recorded (GELU-tanh≈erf in-envelope, single conv-weight aggregate-insensitive). **INERT:** `git diff --stat` vs `adcac8e` = 7 additive entries + committed fixtures, NO shared TU ⇒ text/image/video/audio-pipeline byte-identical BY CONSTRUCTION; `check-device-leakage` unchanged; im2col+existing GEMM ⇒ no new kernel, no compute-sanitizer. Clean CUDA + CPU `-Werror` 0 warn; all seven record checkers green by bare RC. `benchmark_binding=false`, speed pending. Not pushed; FULL SHA reported. NEXT (A3): projector (RMSNorm+Linear) + masked-scatter merge into the LANDED Mistral decoder on Voxtral-Mini-3B → e2e audio→text token-exact vs vLLM 0.25.0 (gate form by measurement). | -| `CLAIM-AUDIO-E2E` | `ENG-MM-AUDIO-E2E` (engine-matrix, NEW `ACTIVE`) + `MODEL-MM-voxtral-voxtral-for-conditional-generation` (model-matrix, `INVENTORIED`→`ACTIVE`). Audio-track **A3** — the FIRST e2e AUDIO→TEXT understanding, on `mistralai/Voxtral-Mini-3B-2507` (Whisper-large-v3-class encoder + AudioLanguageAdapter projector + the LANDED Mistral/Llama text decoder). Owns EXACTLY the NEW `include/vllm/model_executor/models/voxtral.h` + `src/vllm/model_executor/models/voxtral.cpp`, `tests/vllm/multimodal/test_voxtral_e2e.cpp` + committed fixtures `tests/vllm/multimodal/fixtures/voxtral_audio/`, `scripts/mm/a3_voxtral_{oracle_capture,neartie_gate,textonly,wcheck,debug_dump}.py`, 1 `CMakeLists.txt` source line + `tests/CMakeLists.txt` test wiring, and its §1 (A3) of `.agents/specs/audio-track.md`, plus this row + the Voxtral model-matrix row + the `ENG-MM-AUDIO-E2E` engine row + README/BENCHMARKS/state/ledger. **NON-COLLISION:** additive-only (NO shared forward/kernel/runner/registry/other-model TU edit ⇒ text/image/video/audio-pipeline byte-identical BY CONSTRUCTION); does NOT touch the Qwen/Gemma-4 mm rows or `multimodal-track.md`/`gemma4-multimodal.md`; reuses (does NOT edit) `whisper_audio.*` (A2), `audio_processor.*` (A1), `qwen3.*`/`dense_attn_block.h` (Mistral decoder), `qwen3_vl_text.*` (merge). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-a3` (branch `a3-voxtral-e2e`); dgx build+gate `~/vllm-cpp-a3-build` (CUDA cutlass-ON `-DCMAKE_CUDA_ARCHITECTURES=121a`, oracle capture in `~/venvs/vllm-oracle`, e2e gate under ONE `flock $HOME/gpu.lock` FOREGROUND, sibling 27B NOT co-resident) | branch `a3-voxtral-e2e`, base `origin/main` `c4f4dfcc` | Audio-track **A3** — encoder-at-Voxtral-config + downsample-concat + AudioLanguageAdapter projector + masked-scatter merge into the LANDED Mistral decoder → forked greedy; audio→text gate form BY MEASUREMENT. | `ACTIVE` | 2026-07-25 — **A3 LANDED: e2e audio→text gate PASS 14/14.** Voxtral-Mini-3B downloadable (NOT HF-gated) + oracle-runnable (vLLM 0.25.0 `load_format=mistral`, K=5 DETERMINISTIC greedy → STRICT bar). Full C++ pipeline (A1 log-mel [128,3000] rel-L2 7.7e-7 → A2 encoder at Voxtral config → downsample4 → projector [375,3072] → merge at token-24 → Mistral greedy). STRICT prefix 33/48 vs vLLM greedy; decoder proven token-exact (vLLM ref-audio→48/48); bit-exact infeasible (encoder different bf16 GEMM/attn kernels, 8.7% = A2 envelope over 32 layers) ⇒ ratified near-tie-robust gate PASS (worst teacher-forced gap 0.0 nats, sole branch a 4-way bf16 tie at -2.069 nats; `scripts/mm/a3_voxtral_neartie_gate.py`). **KEY FIX:** mistral-consolidated q/k need vLLM's Meta→NeoX rope PERMUTE (verified `permute(wq)==vLLM q_proj`); RED text-only 1/22→22/22. **INERT:** additive (`git diff --stat` = 2 modified lines); Mistral 541/541 + A1 77/77 + A2 203/203 byte-identical; `check-device-leakage` unchanged; no new CUDA kernel ⇒ no compute-sanitizer. Clean CUDA `-Werror` 0-warn, cutlass-ON banner CONFIRMED, all seven checkers green by bare RC. `benchmark_binding=false`, speed pending. NOT pushed; FULL SHA reported. NEXT: A2-follow USM-Conformer (Granite-Speech-2b) for the Gemma-4 audio family; A3 speed grid vs vLLM. | +| `CLAIM-AUDIO-E2E` | `ENG-MM-AUDIO-E2E` (engine-matrix, NEW `ACTIVE`). The Voxtral model row was DROPPED from this claim 2026-08-06 by the live-state audit (`ACTIVE`→`READY`, no Git evidence for the claim); see the Closing and archival note. Audio-track **A3** — the FIRST e2e AUDIO→TEXT understanding, on `mistralai/Voxtral-Mini-3B-2507` (Whisper-large-v3-class encoder + AudioLanguageAdapter projector + the LANDED Mistral/Llama text decoder). Owns EXACTLY the NEW `include/vllm/model_executor/models/voxtral.h` + `src/vllm/model_executor/models/voxtral.cpp`, `tests/vllm/multimodal/test_voxtral_e2e.cpp` + committed fixtures `tests/vllm/multimodal/fixtures/voxtral_audio/`, `scripts/mm/a3_voxtral_{oracle_capture,neartie_gate,textonly,wcheck,debug_dump}.py`, 1 `CMakeLists.txt` source line + `tests/CMakeLists.txt` test wiring, and its §1 (A3) of `.agents/specs/audio-track.md`, plus this row + the Voxtral model-matrix row + the `ENG-MM-AUDIO-E2E` engine row + README/BENCHMARKS/state/ledger. **NON-COLLISION:** additive-only (NO shared forward/kernel/runner/registry/other-model TU edit ⇒ text/image/video/audio-pipeline byte-identical BY CONSTRUCTION); does NOT touch the Qwen/Gemma-4 mm rows or `multimodal-track.md`/`gemma4-multimodal.md`; reuses (does NOT edit) `whisper_audio.*` (A2), `audio_processor.*` (A1), `qwen3.*`/`dense_attn_block.h` (Mistral decoder), `qwen3_vl_text.*` (merge). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp-a3` (branch `a3-voxtral-e2e`); dgx build+gate `~/vllm-cpp-a3-build` (CUDA cutlass-ON `-DCMAKE_CUDA_ARCHITECTURES=121a`, oracle capture in `~/venvs/vllm-oracle`, e2e gate under ONE `flock $HOME/gpu.lock` FOREGROUND, sibling 27B NOT co-resident) | branch `a3-voxtral-e2e`, base `origin/main` `c4f4dfcc` | Audio-track **A3** — encoder-at-Voxtral-config + downsample-concat + AudioLanguageAdapter projector + masked-scatter merge into the LANDED Mistral decoder → forked greedy; audio→text gate form BY MEASUREMENT. | `ACTIVE` | 2026-07-25 — **A3 LANDED: e2e audio→text gate PASS 14/14.** Voxtral-Mini-3B downloadable (NOT HF-gated) + oracle-runnable (vLLM 0.25.0 `load_format=mistral`, K=5 DETERMINISTIC greedy → STRICT bar). Full C++ pipeline (A1 log-mel [128,3000] rel-L2 7.7e-7 → A2 encoder at Voxtral config → downsample4 → projector [375,3072] → merge at token-24 → Mistral greedy). STRICT prefix 33/48 vs vLLM greedy; decoder proven token-exact (vLLM ref-audio→48/48); bit-exact infeasible (encoder different bf16 GEMM/attn kernels, 8.7% = A2 envelope over 32 layers) ⇒ ratified near-tie-robust gate PASS (worst teacher-forced gap 0.0 nats, sole branch a 4-way bf16 tie at -2.069 nats; `scripts/mm/a3_voxtral_neartie_gate.py`). **KEY FIX:** mistral-consolidated q/k need vLLM's Meta→NeoX rope PERMUTE (verified `permute(wq)==vLLM q_proj`); RED text-only 1/22→22/22. **INERT:** additive (`git diff --stat` = 2 modified lines); Mistral 541/541 + A1 77/77 + A2 203/203 byte-identical; `check-device-leakage` unchanged; no new CUDA kernel ⇒ no compute-sanitizer. Clean CUDA `-Werror` 0-warn, cutlass-ON banner CONFIRMED, all seven checkers green by bare RC. `benchmark_binding=false`, speed pending. NOT pushed; FULL SHA reported. NEXT: A2-follow USM-Conformer (Granite-Speech-2b) for the Gemma-4 audio family; A3 speed grid vs vLLM. | | `CLAIM-MULTIMODAL-SPEED-TOWER` | `ENG-MM-QWEN36-VL-FORWARD` (engine-matrix, `ACTIVE`) + `ENG-MM-VISION-TOWER` (narrative). Executes mm speed lever #1 (the vision encoder tower). Owns EXACTLY: the vision tower forward + resident-weights path `src/vllm/model_executor/models/qwen3_vl_vision.{cpp,h}`; the NEW additive attention op `AttentionDenseFast` (`src/vt/cuda/cuda_ops.cu` `AttentionWarpKernel`/`LaunchAttentionWarp`/registration; `include/vt/ops.h` enum+decl; `src/vt/ops.cpp` wrapper; `src/vt/cpu/cpu_ops.cpp` CPU registration reusing the existing reference); the bench `tests/vllm/multimodal/bench_qwen3_5_vl_tower.cpp` + its CMake; the records. **NON-COLLISION:** purely additive — the existing `kAttention`/`AttentionKernel` and every text/audio kernel UNCHANGED (`git diff`); text never calls `kAttentionDenseFast` ⇒ text/audio byte-identical BY CONSTRUCTION (`test_ops_attention` 37239/37239, 27B text SACRED 235/235). Does NOT touch scheduler/runner/registry/other-model TUs. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm-tower-speed` (branch `spike/mm-tower-speed`); dgx build+profile+gate `dgx.casa:~/work/mm-tower-speed` (CUDA cutlass 4.5.0 `-DCMAKE_CUDA_ARCHITECTURES=121a` + FA2 + Triton, ALL GPU under `flock $HOME/gpu.lock` sole owner) | branch `spike/mm-tower-speed`, base `origin/main` `27bc3054` | Multimodal SPEED **tower lever #1** — profile the tower (W0), port the dominant divergent kernel (W1), keep image e2e STRICT 32/32. | `DONE` | 2026-07-26 — **TOWER LEVER CLOSED + BEATEN.** W0 (nsys `cuda_gpu_kern_sum`): 98.9% of the tower forward = the naive `vt::cuda::AttentionKernel` (56 ms/block over 784 patches; NOT QKV/FA2-routing — the `cuda_api_sum` cudaFree-93% was a sync-artifact). W1: warp-scoped online-softmax op `AttentionDenseFast` (no `__syncthreads`, `__shfl_xor` head_dim reduce, register accumulator; separate op ⇒ text byte-identical) + one-time resident-weight load ⇒ **per-image tower 2114→148 ms (14.3×), 0.59× vs vLLM's ~250 ms eager encode = FASTER.** RED-line HELD: 27B image STRICT 32/32, 27B video STRICT 32/32, 4B DeepStack image STRICT 32/32, `test_ops_attention` 37239/37239, 27B text SACRED 235/235; clean `-Werror` 0 warn (cutlass+FA2 banner), compute-sanitizer memcheck 0. `benchmark_binding=false`; DONE-bar still unmet (batched/graphed mm serving c2+ + audio our-side). NOT pushed; FULL SHA reported. | | ~~`CLAIM-ROADMAP-D4APC`~~ **RELEASED 2026-07-27 — W2 + W3 both DONE; `KV-PREFIX-CACHE`→DONE (stamped with the closing commit) and `ROAD-V1-D4-APC`→DONE (headline).** | `KV-PREFIX-CACHE` (engine-matrix, `PARTIAL`→`ACTIVE`; the `generate_block_hash_extra_keys` extra-key sub-item → DONE) + `ROAD-V1-D4-APC` (roadmap `SPIKE`→`PARTIAL`). ROAD-V1-D4-APC **W2 extra_keys** (mm/LoRA/`cache_salt` block-hash) + **W3** cache-ON e2e assessment. Owns EXACTLY: `include/vllm/v1/request.h`, `include/vllm/v1/engine/types.h`, `src/vllm/v1/request.cpp`, `src/vllm/v1/core/kv_cache_utils.cpp`, `tests/vllm/v1/test_kv_cache_utils.cpp`, `tests/vllm/v1/test_kv_cache_manager.cpp`, **(W3) `tests/parity/test_qwen3_apc_e2e.cpp`, `tests/CMakeLists.txt` (one additive block), `scripts/qwen3-apc-oracle-capture.py`, `scripts/qwen3-apc-neartie-gap.py`, `tests/parity/goldens/qwen3_apc_4b/`**, and the records (engine-matrix `KV-PREFIX-CACHE` row, roadmap D4-APC row, `specs/prefix-prompt-caching-parity.md` W2/feature rows, `specs/roadmap-v1-completion.md` D4-APC lines, this claim row, ledger, state, README, BENCHMARKS). **NON-COLLISION:** does NOT touch the C8 sibling's serving/metrics/streaming-parser files or C8 rows; additive extra-key path — text requests (no mm/lora/salt) hash byte-identical BY CONSTRUCTION (the 253-assertion pinned goldens in `test_kv_cache_utils.cpp` unchanged), so APC-off / SACRED greedy gates are unaffected. | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a278a93e1db71a20f`; dgx CPU build+gate `dgx.casa:~/scratch_apc_w2` (`-DVLLM_CPP_CUDA=OFF` Release CPU build, CPU doctest — W2 is a device-neutral behavioral port, NO GPU/flock needed) | branch `worktree-agent-a278a93e1db71a20f`, base `origin/main` `05a2e0d0` | ROAD-V1-D4-APC W2 `extra_keys` (unblocks MM/LoRA cache consumers) + W3 cache-ON gate. | `DONE` | 2026-07-27 — **W2 DONE (CPU-gated on dgx GB10, NOT pushed).** `generate_block_hash_extra_keys` ported 1:1 (`kv_cache_utils.py:451-591`): `GenMmExtraHashKeys` over `Request.mm_features` + LoRA name + `cache_salt`, order lora->mm->salt (prompt_embeds deferred — no prompt-embeds path). `cache_salt`+`lora_name` on `Request`/`EngineCoreRequest`; `FromEngineCoreRequest` installs the hasher AFTER the fields (fixed a latent ordering bug: mm_features were assigned after the ctor's first hash). **RED-first PROVEN on dgx:** restoring the stub → `test_kv_cache_utils` 5 cases FAIL + `test_kv_cache_manager` no-false-share FAIL (`n1==48` — tenant-B request false-hits tenant-A's 48 cached tokens); with extra_keys `n1==0`. GREEN: utils 29/29 (253 assertions), manager 10/10 (74). Clean CPU build RC=0. **INERT:** text-path goldens byte-identical; no kernel touched (no compute-sanitizer owed). **W3 DONE 2026-07-27 (`CLAIM-ROADMAP-D4APC-W3`, dgx GB10, NOT pushed) — the FIRST-EVER cache-ON model gate.** Vehicle `Qwen/Qwen3-4B` (dense, full-attention, APC-default-ON — the vehicle the prior RI note missed). `test_qwen3_apc_e2e` 2/2, 84/84 asserts: APC-ON hits 2240/2777 (rate 0.807) / OFF 0; APC-ON==APC-OFF token-exact 5/6 (1 diff = a vLLM-confirmed 0.125-nat near-tie, RCA'd = attention-kernel-path flip, not a cache bug); == vLLM-APC-ON teacher-forced (OFF 6/6 gap 0.0 nats, ON 6/6 gap ≤0.125, 0 outside top-20); TTFT 70.1→39.9 ms = 1.76×. **NO engine code changed** (gate-only over the already-shipped default-ON path; binary byte-identical ⇒ SACRED unaffected); 4B SACRED 16/16 no-regression. CUDA `-Werror` clean; no kernel ⇒ no compute-sanitizer surface. Oracle vLLM 0.25.0 (0.26 venv broken — editable source disk-reclaimed; 4B byte-stable across the pin). `KV-PREFIX-CACHE`→`DONE`, `ROAD-V1-D4-APC`→`DONE` (headline; W4-W9 tails non-blocking in own rows). NOT pushed; FULL SHA reported. | @@ -1579,17 +1574,11 @@ items a-runner/b stay with the async/GDN `runner.cpp` owners. | `CLAIM-MM-SPEED-AUDIO-ENC-RESIDENT` | `ENG-MM-AUDIO-E2E` (engine-matrix, stays `ACTIVE`/`PARTIAL` — audio TTFT/encoder still NOT at parity; carries the note in-place); Voxtral model-matrix row + feature-matrix mm row stay `PARTIAL`. Implements §14.5 residual lever #2 (the MEDIUM, byte-exact host-data-movement half). Owns EXACTLY: the device-resident weight handles (`mutable std::shared_ptr` on `WhisperEncoderLayerWeights` + `WhisperAudioEncoderWeights`) in `include/vllm/model_executor/models/whisper_audio.h` + the `ResidentBf16` helper / `VT_WHISPER_ENC_REMARSHAL` A-B knob / weight-site rewiring in `src/vllm/model_executor/models/whisper_audio.cpp`; the records (`specs/multimodal-speed.md` §15 + headline pointer, this row, ledger, state, Voxtral model-matrix row, feature-matrix mm row, roadmap MM row, `docs/ENVIRONMENT.md` env row, docs/STATUS.md audio line, docs/BENCHMARKS.md). **NON-COLLISION:** pure host DATA-MOVEMENT (no new kernel, no op-registry/`cuda_ops.cu`/`ops.*`/`cpu_ops.cpp` edit); touches ONLY `whisper_audio.{h,cpp}` + `docs/ENVIRONMENT.md` + records; does NOT touch `voxtral.cpp`/`qwen3_5.cpp`/`cuda_paged_attn.cu`/the tower/scheduler/runner/any serving-parser file; README/Metal NOT touched (concurrent session owns them). Byte-identical output by construction (moves weights device-resident, no math change). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a11fada6333aa0108`; dgx build+gate+profile `dgx.casa:~/vllmcpp-whisper-resid` (`git archive` of the code tree, CUDA cutlass 4.5.0 `-DVLLM_CPP_CUTLASS_DIR=$HOME/cutlass-4.5.0 -DVLLM_CPP_TRITON=ON -DCMAKE_CUDA_ARCHITECTURES=121a`, "CUTLASS found"+FA2-ENABLED banners CONFIRMED, `-Werror` 0-warn); ALL GPU under `flock $HOME/gpu.lock` (concurrent CPU-build agent present, box GPU idle), rep0 dropped | branch `worktree-agent-a11fada6333aa0108`, base local `main` `0e2c667a` (confirmed via `git rev-parse HEAD`) | Multimodal SPEED **audio ENCODER TTFT lever #2** — profile-CONFIRM the ~0.75 s host chunk, then make the encoder weights device-resident (mirror the Qwen `d_dev` seam; upload+convert ONCE, reuse); the conv host round-trip removal is DEFERRED (needs a device im2col kernel). Verify BYTE-EXACT on GPU (16/16 + goldens unchanged + proof-of-run + RED + sanitizer) + A/B + honest verdict vs vLLM 43 ms. | `DONE` | 2026-07-28 — **§14.5 lever #2 LANDED: device-resident one-time encoder weights, byte-exact, real 1.89× encoder-forward host win, encoder still NOT at parity.** PROFILE-CONFIRMED the ~0.75 s: it is DOMINATED by per-call weight marshalling (host f32→bf16 `ToBf16` + `Backend::Copy` H2D of 487 weight tensors / ~635 M f32 every forward), NOT the conv round-trip (small; its removal needs a device im2col kernel — DEFERRED). Mirrored the Qwen decoder residency (`qwen3_5_weights.h` `d_dev`; `qwen3_5.cpp` `ResidentWeight:797`): each weight f32→bf16-converted + uploaded ONCE into a `mutable shared_ptr` (Backend-Free deleter), reused across forwards. **BYTE-EXACT (RED line HELD):** `test_voxtral_e2e` **16/16** (strict prefix 18/48, teacher-force PASS, seq 48/48) — IDENTICAL to §14; goldens md5 UNCHANGED (`voxtral_golden.json 8ab87b7e…`, `voxtral_neartie.json 937b9ad3…`, before==after). **Proof-of-run + RED (`VT_WHISPER_ENC_REMARSHAL`):** nsys `memcpy HtoD` resident **740 ops / 9.4 GB** vs re-marshal **1714 ops / 11.9 GB** (−974 ops = 487 weights × 2 saved re-uploads, −2.5 GB); RED forces per-call re-upload → the +648 ms returns. **compute-sanitizer memcheck 0 errors** (16/16 under sanitizer; encoder path not graphed ⇒ capture-safety N/A). **A/B (same-binary `VT_ENC_REPS`, `flock`, 6 reps rep0 dropped, steady-state):** encoder forward **~1377 → ~729 ms (−648 ms, 1.89×, NON-OVERLAPPING)** — the 1377 ms re-marshal arm reproduces §14's pre-lever ~1375 ms EXACTLY. Trajectory §13→§14→§15: **1834 → 1375 → 729 ms.** **HONEST — NOT closed:** ~729 ms vs vLLM ~43 ms (~17×, was ~32×); the residual is now GPU-compute-bound (scalar warp-per-query attention 617 ms/32L + conv GEMMs) — the LARGE gap-closer remains tensor-core MMA hd-64 non-causal FA2 (§14.5 lever #1, dedicated slot). vLLM not re-measured (OOM-reboot risk of a big oracle alongside the active tree; residual ~17× regardless). Clean CUDA `-Werror` 0-warn (banners CONFIRMED) + CPU `-Werror` 0-warn (`whisper_audio.cpp`). Audio TTFT/encoder stays speed-pending/`PARTIAL`. `benchmark_binding=false`. NOT pushed; FULL SHA reported. | | `CLAIM-MM-SPEED-QWEN-IMAGE` | `ENG-MM-QWEN36-VL-FORWARD` (engine-matrix, stays `ACTIVE`/`PARTIAL` — vision-forward speed axis now BEATS vLLM but the umbrella row stays PARTIAL for batched c2+/serving; §16 note added in-place, no state change, no Total recompute); `KERNEL-ATTN-DENSE-FLASH` (kernel-matrix, stays `ACTIVE` — scope EXTENDED to the vision tower, no new kernel row ⇒ KERNEL count unchanged). GPU CAMPAIGN #2 (2/3): close the Qwen image/video mm-forward SPEED. Owns EXACTLY: the vision-tower attention op-select (default `AttentionDenseFast`→`AttentionDenseFlash` + `VT_QWEN3VL_ATTN_WARP`/`VT_QWEN3VL_ATTN_EAGER` A/B knobs) in `src/vllm/model_executor/models/qwen3_vl_vision.cpp`; the warp-vs-flash A/B arm + bit-identity assert in `tests/vllm/multimodal/bench_qwen3_5_vl_tower.cpp`; the records (`specs/multimodal-speed.md` §16 + headline pointer, this row, engine-matrix `ENG-MM-QWEN36-VL-FORWARD` §16 note + claim-list, kernel-matrix `KERNEL-ATTN-DENSE-FLASH` scope+claim, model-matrix Qwen3-VL/27B speed note, feature-matrix mm row, roadmap MM row, ledger, state, `docs/ENVIRONMENT.md` (2 A/B env rows), `docs/STATUS.md`, `docs/BENCHMARKS.md`). **NON-COLLISION:** pure op-SELECT of an EXISTING additive op (no new kernel, no `cuda_ops.cu`/`ops.*`/`cpu_ops.cpp` edit); `kAttention`/`kAttentionDenseFast`/`kAttentionDenseFlash` are separate ops ⇒ text/audio/other-model + the whisper encoder byte-identical by construction; touches ONLY `qwen3_vl_vision.cpp` + the tower bench + records; does NOT touch `qwen3_5.cpp`/`voxtral.cpp`/`whisper_audio.cpp`/`cuda_paged_attn.cu`/scheduler/runner/any serving-parser file; README/Metal NOT touched (concurrent session owns them). Byte-identical tower output by construction (flash per-warp math copied verbatim from warp; only K/V source changes). | Claude Code (opus-4-8) | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a0304cec284f452e8`; dgx build+gate+profile `dgx.casa:~/vllmcpp-mmspeed` (`git archive` of the working tree over the reused §14/§15 tree, CUDA `build-cuda` cutlass 4.5.0 `-DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUTLASS_DIR=$HOME/cutlass-4.5.0`, "CUTLASS found"+FA2-ENABLED banners CONFIRMED, `-Werror` 0-warn exit 0); ALL GPU under `flock $HOME/gpu.lock` sole owner (`nvidia-smi` idle, `local-ai-worker` absent), rep0 dropped | branch `main` (worktree-isolated), base local `main` `0a07ac76` (confirmed via `git rev-parse HEAD`) | Multimodal SPEED **Qwen image/video vision-forward** — ATTRIBUTE the tower forward (nsys), extend the §14 flash kernel to the tower attention (byte-exact), verify STRICT 32/32 + goldens unchanged + proof-of-run + RED + sanitizer + A/B, honest verdict vs vLLM eager encode. | `DONE` | 2026-07-28 — **ATTRIBUTION REFUTED the assumed big lever; the tower ALREADY BEATS vLLM; byte-exact 1.04× landed.** nsys (`cuda_gpu_kern_sum`, 27B tower, 784 patches): the post-§7 148 ms forward is **~85% the dense attention** (`AttentionWarpKernel` **4.66 ms/block × 27 = ~126 ms**), GEMMs ~10%, glue <5%. Routed the tower attention (head_dim 72, non-causal) to the §14 flash-tiled `vt::AttentionDenseFlash` (head_dim-generic; per-warp online-softmax copied VERBATIM from the warp kernel, only K/V from shared-mem tiles ⇒ BIT-IDENTICAL). **BYTE-EXACT (RED line HELD):** 27B image e2e STRICT **32/32** (54/54), 4B image **32/32** (46/46), 27B video **32/32** (gap 0 nats, 27/27), `test_ops_attention` 37239/37239; bench flash-vs-warp tower **0/1,003,520 mismatches**; goldens md5 UNCHANGED (`qwen3_5_27b 3bc5f231…`, `qwen3vl_text b7221f22…`, `qwen3_5_27b_video bf14a962…`, `qwen3vl_video 09b2fce3…`, before==after). **Proof-of-run:** nsys default 4B e2e = `AttentionDenseFlashKernel` **24 inst** (= 24 vision blocks), ZERO `AttentionWarpKernel` on mm path. **RED:** corrupt flash V-accum → 30/46 FAIL → restore → 46/46. **compute-sanitizer memcheck 0 errors.** **A/B (same-binary, `flock`, rep0 dropped, 27B tower):** warp **148.3 → flash 142.3 ms = 1.04×** — **REFUTES** the assumed lever: at t=784 (single window) the vision attention is serial-latency-bound (dependent 784-key online-softmax chain; L2 already serves the redundant reads), NOT K/V-bandwidth-bound like audio (§14, t=1500, 1.82×), so flash recovers only ~6 ms. **HONEST HEADLINE: the tower ALREADY BEATS vLLM — 142 ms vs vLLM 0.25.0 ~250 ms eager encode = 0.57×** (carried-forward §7 denominator; not re-measured — OOM-reboot risk, well under it). Lever lands FREE + byte-exact + unifies the codebase (tower + audio encoder now both on `AttentionDenseFlash`). Clean CUDA `-Werror` 0-warn (banners CONFIRMED). Image/video mm-forward = correctness-DONE + speed-BEATS-vLLM; residual (NOT needed for parity): tensor-core MMA hd-72 non-causal attention (§14.5 lever #1) + batched c2+/serving. `benchmark_binding=false`. NOT pushed; FULL SHA reported. | -| `CLAIM-GEMMA4-G1` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (G1 backbone landed here; row now `ACTIVE` — text path STRICT 32/32 gated by `CLAIM-GEMMA4-G1B`) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a98e29bc69ca350d2` (CPU build `build` `-DVLLM_CPP_CUDA=OFF` Release + `-Werror` clean; dgx used ONLY read-only for the E4B config + safetensors HEADER via HTTP range, NO checkpoint download, NO GPU, NO `gpu.lock`) | branch `worktree-agent-a98e29bc69ca350d2`, base `main` `da5191e3` (confirmed `git rev-parse HEAD`) | MODEL-GEMMA4 G1: the Gemma-4 TEXT backbone (`Gemma4ForConditionalGeneration` language_model stack, `unsloth/gemma-4-E4B-it`) — registry stub + weight loader + backbone forward. Owns ONLY: NEW `include/vllm/model_executor/models/gemma4.h` + `src/vllm/model_executor/models/{gemma4,gemma4_weights,gemma4_registry}.cpp`, `CMakeLists.txt` (3 src lines), and the record surfaces (this claim, model-matrix `MODEL-MM-gemma4-*`/`MODEL-TEXT-gemma4-*` rows, `.agents/specs/gemma4-multimodal.md` G1 §, feature-matrix, roadmap_v1, kernel-matrix, parity-ledger, state, docs/STATUS, docs/BENCHMARKS). **NON-COLLISION:** NEW additive files only (mirrors OLMo-2/gemma3 registration seam); ZERO edits to any existing model forward TU or the runner/KV shared paths; does NOT touch the C8 parser `gemma4.cpp` tool-parser (that is `src/vllm/entrypoints/openai/tool_parsers/gemma4.cpp`, a different file) or README/Metal. | `SPIKE (G1 PARTIAL — backbone landed, e2e gate runner-blocked)` | 2026-07-28 — **REGISTRY + LOADER + BACKBONE FORWARD LANDED + COMPILE-CLEAN, e2e STRICT gate BLOCKED on runner topology (foreground, NOT pushed).** Grounded 1:1 in `gemma4.py` (`Gemma4Attention` :374-552, `Gemma4DecoderLayer` :555-767, `Gemma4Model` PLE/embed :845-928, forward :1289-1370, load_weights :1625-1728) + `gemma4_rope.py` (proportional RoPE) + `layernorm.py::RMSNorm` (PLAIN `x·w`, NOT the Gemma `(1+w)` — the load-bearing divergence from gemma2/3). E4B config: enable_moe_block=false / attention_k_eq_v=false / use_double_wide_mlp=false ⇒ the MoE/k_eq_v/double-MLP primitives are OUT of scope (they stay the ≥12B follow-on). Landed primitives: PLE (`embed_tokens_per_layer` + `per_layer_model_projection` + per-layer gate/proj/norm, gate reuses `vt::GeluAndMul` for `gelu(gate)*ple` — no new op), YOCO KV-sharing (shared layers read the target layer's cache in-forward), heterogeneous head_dim (256 sliding / 512 full), proportional partial-RoPE (custom host cos/sin cache) + standard sliding rope, GeGLU, per-layer scalar, final logit soft-cap 30, tied lm_head, weight-less V-norm (ones-weight). **Loader VERIFIED** against the real E4B safetensors HEADER (2130 tensors; all 336 `model.language_model.*` names+shapes matched; mm towers audio_tower/vision_tower/embed_audio/embed_vision skipped per `gemma4.py:1716-1723`). Build: CPU `-Werror` 0-warn on all 3 TUs + full `libvllm.a` link (SACRED inertness — the whole existing model set still builds). **HONEST e2e BLOCKER (named, not faked):** the strict 32/32 gate vs `tests/parity/goldens/gemma4_e4b_text/gen_manifest.json` cannot RUN because the runner allocates ONE uniform KV head_dim per non-GDN layer (`runner.cpp:600-646`); Gemma-4's per-layer 256/512 heads need a shared-path change to `attn_kv_` construction (per-layer/per-group head_dim). The forward's `VT_CHECK(kv.head_size==Dh)` makes this an explicit failure, not a silent wrong answer. G-next = that runner heterogeneous-KV topology + a full CUDA gate; no push. | -| `CLAIM-GEMMA4-G1B` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (SPIKE → **ACTIVE** — text path STRICT 32/32 gated; mm towers G2/G3 + speed pending; row owned here) + `ENG-RUNNER-MODELSHAPE` (engine-matrix, stays `ACTIVE` — per-layer-KV extension added in-place, no state change) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a7d31fe6f2d19a381` (CPU `build-cpu` `-DVLLM_CPP_CUDA=OFF` Release `-Werror` clean; dgx CUDA build `~/gemma4g1b` sm_121a for the STRICT gate under `flock ~/gpu.lock`) | branch `worktree-agent-a7d31fe6f2d19a381`, base `main` `2e874490` (confirmed `git rev-parse HEAD`) | MODEL-GEMMA4 G1b: the runner heterogeneous per-layer KV head_dim change unblocking the strict gate. Owns: `include/vllm/v1/kv_cache_interface.h` (`KVCacheConfig::per_layer_attn_specs`), `src/vllm/v1/worker/gpu/runner.cpp` (per-layer alloc+view), `src/vllm/model_executor/models/gemma4_registry.cpp` (publishes per-layer specs + text_config reads), `gemma4.cpp`/`gemma4_weights.cpp` (text_config reads + stale-note fix), `src/vllm/transformers_utils/hf_config.cpp` (nested-rope loads), `src/vllm/tokenizer/tokenizer.cpp` (Gemma metaspace-via-normalizer), NEW `tests/parity/test_gemma4_paged_engine.cpp` + 1 CMake block, `tests/vllm/test_hf_config.cpp` (nested-rope subcase flipped) + the record surfaces. **NON-COLLISION:** the KVCacheConfig field is OPTIONAL + empty-defaulted ⇒ byte-neutral for every existing model (the `ENG-RUNNER-MODELSHAPE` shared path is extended additively, not rewritten); does NOT touch README/Metal or any other model forward. | `PARTIAL (text path STRICT 32/32; mm towers G2/G3 + speed pending)` | 2026-07-28 — **TEXT PATH STRICT 32/32 TOKEN-EXACT vs vLLM 0.25.0 (foreground, NOT pushed).** Runner per-layer KV head_dim (`per_layer_attn_specs`) + 3 additive loader gaps fixed en route to the first-ever Gemma-4 forward (nested `rope_parameters`; Gemma `Replace(" "→"▁")` metaspace normalizer; reading Gemma-4 scalars from `raw["text_config"]` — the G1 latent bug that silently treated all layers as sliding). Gate `test_gemma4_paged_engine` on the dgx CUDA build under `flock` = 32/32 exact ids. BYTE-NEUTRAL: full CPU runner/KV/scheduler/tokenizer/hf_config suite green + OLMo-2 SACRED GPU re-gate 16/16 UNCHANGED on the final binary. Residuals: YOCO cache dedup (G1c, memory-only), G2 vision (reuses M2a) / G3 audio towers, per-axis SPEED. dgx: re-fetched E4B (16 GB, disk-guarded), pruned after; local-ai-worker restored. | -| `CLAIM-GEMMA4-G2` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (stays **ACTIVE** — text path already STRICT 32/32; this claim adds the IMAGE oracle + SigLIP/NaFlex port map; image NOT yet engine-gated) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a232a21e6580e48ca` (records + golden fixture only — NO C++ vision code; dgx used under `flock /tmp/gpu` for the E4B image golden + CPU vision-ref dump) | branch `main` worktree, base `main` `bfc8cf5d` (confirmed `git rev-parse HEAD`) | MODEL-GEMMA4 G2: IMAGE oracle capture + staged vision references + the corrected SigLIP/NaFlex port map. Owns ONLY: NEW `scripts/mm/g2_gemma4_image_oracle_capture.py` + `scripts/mm/g2_vision_ref_dump.py`, NEW `tests/parity/goldens/gemma4_e4b_image/` fixture, `.agents/specs/gemma4-multimodal.md` §G2, and the record surfaces (this claim, model-matrix mm row, roadmap, feature-matrix, parity-ledger, state, docs/STATUS, docs/BENCHMARKS). **NON-COLLISION:** additive golden + scripts + records only; ZERO edits to any C++/build TU; does NOT touch README/Metal. | `PARTIAL (image oracle + port map LANDED; C++ tower residual)` | 2026-07-28 — **IMAGE→text ORACLE GOLDEN STRICT + SigLIP/NaFlex PORT MAP LANDED (honest partial, foreground, NOT pushed).** `unsloth/gemma-4-E4B-it` image golden K=5 DETERMINISTIC ⇒ **STRICT** (18 tokens → coherent gradient-image description, 256 soft tokens, prompt ids 274; `tests/parity/goldens/gemma4_e4b_image/gen_manifest.json`) + 4 staged vision refs (proc/patch-embed/encoder/pooled/projected, sha-verified — M2a unit-gate targets). ★ Port-map CORRECTION: the tower is a custom **NaFlex SigLIP2 WITH multidim vision-RoPE + q/k/v-norm + Gemma2 sandwich norms + learned-2D one_hot pos-embed + √hidden avg-pool-by-position pooler** — spec §0.1 "no vision-RoPE, simpler than Qwen3-VL" REFUTED; also NEW = the Gemma-4 NaFlex image processor (not covered by `qwen3vl_processor`). **RESIDUAL (named, unbuilt):** the C++ SigLIP/NaFlex tower forward + Gemma-4 image processor + projector/merge — no C++ vision code this pass, no token-exact claim; follow-on `G2-impl` is turnkey against the committed refs. dgx: re-fetched E4B (16 GB, disk-guarded, `flock`), model+tree pruned after, worker restored (`--restart=always`). | -| `CLAIM-GEMMA4-G2-IMPL` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (stays **ACTIVE** — text path STRICT 32/32; this claim adds the C++ NaFlex SigLIP2 vision TOWER, per-stage gates PASS; image e2e still residual) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-aaeedee89d17acc5c` (dgx GB10 sm_121a CUDA `flock ~/gpu.lock` for the staged gate + memcheck + text-inertness re-gate; E4B safetensors re-fetched ~15 GiB, build tree pruned) | base `main` `c497668d` (confirmed `git rev-parse HEAD`) | MODEL-GEMMA4 G2-impl: the C++ Gemma-4 NaFlex SigLIP2 vision tower forward + per-stage unit gate. Owns ONLY: NEW `include/vllm/model_executor/models/gemma4_vision.h` + `src/vllm/model_executor/models/gemma4_vision.cpp` + `tests/vllm/multimodal/test_gemma4_vision_tower.cpp` + `scripts/mm/g2_vision_weight_dump.py`, `CMakeLists.txt` (1 src line) + `tests/CMakeLists.txt` (1 test block), and the record surfaces (this claim, model-matrix mm row, `.agents/specs/gemma4-multimodal.md` §G2-impl, kernel-matrix, feature-matrix, roadmap_v1, parity-ledger, state, docs/STATUS, docs/BENCHMARKS). **NON-COLLISION:** NEW additive standalone TU (not referenced by the registry/runner ⇒ text path byte-identical); ZERO edits to any existing model forward TU or shared path; does NOT touch README/Metal. | `ACTIVE (vision tower per-stage gates PASS; image e2e residual)` | 2026-07-28 — **C++ NaFlex SigLIP2 vision TOWER LANDED + all 4 per-stage gates PASS (foreground, NOT pushed).** `test_gemma4_vision_tower` (dgx CUDA `flock`, 220/220): patch-embed rel-L2 2.15e-3, encoder 3.14e-2, pooled 1.36e-2, projected 1.85e-2; `n_valid=2304, n_soft=256`; compute-sanitizer memcheck **0 errors**. ★ Port-map correction: E4B `use_clipped_linears=True` with FINITE trained QAT clamps on the 7 encoder linears (G2 assumed plain Linear) — implemented (q/k/v share in-clamp, gate/up share both). Multidim vision-RoPE via 2× `vt::RopeFromCache` sharing one cache; attn scale 1.0; **ZERO new vt op**. **Inertness:** text `test_gemma4_paged_engine` STRICT 32/32 UNCHANGED on the linking binary; Qwen3-VL mm additive-by-construction; `-Werror` 0-warn both new TUs (one pre-existing UNRELATED GCC-13 `-Warray-bounds` false positive in `voxtral.cpp` suppressed per-file on the dgx tree ONLY, not committed). **RESIDUAL (named):** image→text e2e (the C++ NaFlex image processor + engine mm-plumbing: SupportsMultiModal, hasher/encoder-cache, masked-scatter merge at ``, decode fork) is the M2c-equivalent follow-on; the tower+projector (merge INPUT) are proven. Speed pending. dgx worker left `--restart=always` running. Not pushed; FULL SHA reported. | -| `CLAIM-GEMMA4-G3` | `MODEL-MM-gemma4-mm-gemma4-for-conditional-generation` (stays **ACTIVE** — text path STRICT 32/32; this claim adds the C++ USM-Conformer AUDIO TOWER + audio projector, per-stage gates PASS; audio e2e still residual) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a43b3924681bbae30` (dev-box CPU `build` `-DVLLM_CPP_CUDA=OFF` Release `-Werror` clean + host-f32 gate — the USM-Conformer tower is device-neutral pure C++; dgx used ONLY read-only CPU (`CUDA_VISIBLE_DEVICES=""`, NO GPU, NO `gpu.lock`) for the oracle per-stage tower dump + the E4B audio-weight dump — E4B was ALREADY cached, NO fetch, disk left at 52 GiB) | base `main` `308c312a` (confirmed `git rev-parse HEAD`) | MODEL-GEMMA4 G3: the C++ Gemma-4 USM-Conformer audio tower forward (`Gemma4AudioModel`) + the audio `Gemma4MultimodalEmbedder` projector + per-stage unit gate. Owns ONLY: NEW `include/vllm/model_executor/models/gemma4_audio.h` + `src/vllm/model_executor/models/gemma4_audio.cpp` + `tests/vllm/multimodal/test_gemma4_audio_tower.cpp` + `scripts/mm/g3_audio_tower_ref.py` + `tests/parity/goldens/gemma4_e4b_audio/` (golden), `CMakeLists.txt` (1 src line) + `tests/CMakeLists.txt` (1 test block), and the record surfaces (this claim, model-matrix mm row, `.agents/specs/gemma4-multimodal.md` §G3, kernel-matrix, feature-matrix, roadmap_v1, parity-ledger, state, docs/STATUS, docs/BENCHMARKS). **NON-COLLISION:** NEW additive standalone TU (not referenced by the registry/runner ⇒ text/vision paths byte-identical); ZERO edits to any existing model forward TU or shared path; ZERO new `vt::` op / kernel (host f32); does NOT touch README/Metal. | `ACTIVE (audio tower per-stage gates PASS; audio e2e residual)` | 2026-07-28 — **C++ USM-Conformer AUDIO TOWER LANDED + all 7 per-stage gates PASS (foreground, NOT pushed).** `test_gemma4_audio_tower` (dev-box host f32, 1256/1256): subsample rel-L2 5.4e-7, posemb 8.9e-8, block0 4.2e-7, block_mid 3.8e-7, block_last 4.4e-6, output_proj 5.9e-6, projected 6.3e-6 — f32-EXACT (residual = f64-vs-f32 accumulation order). Ported 1:1 from `modeling_gemma4.py` @ 5.13.1: 2×Conv2d subsample (k3s2p1 + LayerNorm-no-bias + ReLU + mask[::2]) → input_proj → rel-pos-enc (inv_timescales, sin/cos) → 12 Conformer layers (ff1 half-step 0.5 → norm_pre_attn → CHUNKED-LOCAL attn [chunk 12, past window 12, Transformer-XL rel-shift + softcap 50 + per_dim_scale softplus] → norm_post_attn → light-conv [GLU + depthwise causal conv k5] → ff2 → norm_out) → output_proj → embed_audio (RMSNorm-noweight + Linear→2560). ★ E4B `use_clipped_linears=True` with FINITE trained QAT clamps on all clippable linears — implemented. **RED-first proven:** the initial run used the wrong sliding window (`kj∈[qi-12,qi]`, 13 keys) → block_mid 2.8e-2 / block_last 0.22 / projected 0.31 RED; the fix to the source `sliding_window_mask_function` semantics (`dist=qi-kj ∈ [0,12)`, 12 keys) → all ~1e-6 GREEN. **Inertness (by construction):** NEW standalone TU not referenced by registry/runner ⇒ text `test_gemma4_paged_engine` STRICT 32/32 + the G2 vision gates byte-identical; full `libvllm` + all tests re-link clean; `-Werror` 0-warn on both new TUs (CPU). **RESIDUAL (named, honest):** audio→text e2e (the Gemma-4 audio feature-extractor / A1 mel frontend + engine mm-plumbing: SupportsMultiModal, hasher/encoder-cache, masked-scatter merge at `