mudler · localai-bot · Aug 6, 2026 · Aug 6, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -22,7 +22,7 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | **e2e RUNS** (bf16-resident §13): 13/13·656. Token gate **NEAR-TIE 106/128** | device GDN/MLA islands; 1.59 tok/s; default OFF |
 | 35B fresh grid | **BOUND** @`1ea26427`: 0.93-1.03x, c16 0.93x. INTAKE + Option A both NEGATIVE | Lever left: prefill glue (#61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |
-| MXFP4 parity | c1 1.020, c2-c8 0.962-0.969. #75: occupancy IDENTICAL 8.33%; matched vLLM reg+instr, STILL +10us = ptxas gap | c8: flash ptxas ~40%; try ptxas-lineage vendor |
+| MXFP4 parity | c1 1.020, c2-c8 0.962-0.969. **#82 CLOSED: ptxas-lineage REFUTED (A/B ties our+vLLM PTX all ptxas/JIT; +10us=engine context, not codegen)** | TERMINAL: at parity |
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): async mirror on classic dense Qwen3; SACRED 184/184 | Residual: sibling scope one-liner |
 | CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Op-dispatch profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn**. **G5 not next** | Parakeet encoder; attn dtype hoist |
 | Supported-models list (`row/DOCS-SUPPORTED-MODELS-MATRIX`) | **DRAFT PR**: FEATURES per-arch table CI-bound to registry (30 archs) | Reviewer merge |

diff --git a/.agents/benchmark-record.md b/.agents/benchmark-record.md
@@ -19,6 +19,33 @@ from relative link targets repointed for this file's location.
 
 # Benchmarks
 
+## QUANT-CT-MXFP4-FLASH-PTXAS — the ptxas-lineage hypothesis is REFUTED three ways: vLLM's fa2 wheel ships NO sm_12x cubin (only CUDA-13.0 PTX-ISA-9.0 compute_80 PTX, driver-JIT'd on GB10), an "old CUDA 12.x ptxas" cannot even target sm_121a, and a same-params cuModule A/B shows our-PTX and vLLM's-OWN-PTX schedule the c8 decode kernel identically (~144 us) across driver-JIT / ptxas 13.0 / ptxas 13.2 — the +10 us/call engine gap is ENGINE CONTEXT, not flash codegen (2026-08-06, `row/QUANT-CT-MXFP4-FLASH-PTXAS`, base `362a3c99`, GB10 sm_121a, PR #82)
+
+#75 attributed the residual +10 us/call c8 flash gap (ours ~167 vs vLLM ~157) to "vLLM's wheel `ptxas` SASS-scheduling quality (older CUDA 12.x lineage)" and OWED obtaining that ptxas and A/B'ing it. This row did. **The lineage hypothesis dies at the premise, then again at the measurement.**
+
+**W1(a) — the wheel has no old lineage and no runnable GB10 cubin (`cuobjdump` `_vllm_fa2_C.abi3.so`, vLLM 0.25.0-stage).** 52 ELF cubins ALL `sm_80`; 52 PTX ALL `.target sm_80 .version 9.0`. PTX ISA 9.0 = **CUDA 13.0**, the SAME major lineage as our nvcc 13.0 — not an older 12.x. There is **NO sm_90/100/120/121 cubin**. On GB10 (sm_121a) the sm_80 SASS cannot execute, so the flash SASS that actually RUNS is **driver-JIT'd from the compute_80 PTX by the box CUDA-13.0 driver (580.159.03)** — produced by the DRIVER's JIT, never by a wheel `ptxas`. #75's mental model (a wheel `ptxas` baked vLLM's fast sm_121a cubin) is factually wrong; vLLM ships portable PTX and relies on the same driver JIT our own build can invoke. #75's own Build C already used that exact assembler (it compiled `code=compute_80` → driver-JIT) and matched vLLM's reg+instr — the assembler was never a variable we lacked.
+
+**W1(b) — an "old CUDA 12.x ptxas" cannot target the arch.** Box `ptxas`: 12.8 (triton), 13.0 (toolkit, our build default), 13.2 (nvidia/cu13 wheel). `ptxas 12.8` tops out at **sm_120a — no sm_121a** (first appears in 13.0), and it cannot read PTX ISA 9.0. So the mission's literal recipe (get the old ptxas, assemble our PTX for sm_121a) is **impossible by construction**: sm_121a SASS only exists from ptxas 13.0+.
+
+**W1(c) — same-params cuModule A/B (THE ARBITER), locked idle box, CUDA-event medians (100× back-to-back × 15 reps).** A standalone harness builds `Flash_fwd_params` for the exact c8 GQA-swap decode (b=8, kv_heads=8, ngroups=4, d=128, seqlen_k=1024, num_splits=3 → grid (1,3,64), block 128, dyn-smem 81 920 B) and times the SAME decode kernel (`flash_fwd_splitkv_kernel<…128,64,128,4…,0,0,0,0,1,0,1,0>`, mangled-identical in our build and vLLM PTX #30) loaded from our compute_80+fast-math PTX and from vLLM's extracted PTX #30, each via {driver-JIT, ptxas 13.0, ptxas 13.2}. All six cubins REG=241 (matched vLLM). Result:
+
+| decode-kernel arm (single split-launch) | module us/call | module/native |
+|---|---|---|
+| our compute_80+fm — driver-JIT | 144.96 | 0.998 |
+| our compute_80+fm — ptxas 13.0 | 149.07 | 1.004 |
+| our compute_80+fm — ptxas 13.2 | 150.78 | 1.009 |
+| vLLM PTX #30 — driver-JIT | 144.55 | 1.007 |
+| vLLM PTX #30 — ptxas 13.0 | 135.95 | 0.969 |
+| vLLM PTX #30 — ptxas 13.2 | 140.54 | 1.013 |
+
+Native sm_121a anchors drifted 138.7–149.4 across the sequential arms (box clock drift), so the module/native RATIO is the drift-normalized read: all six ∈ [0.969, 1.013], i.e. **within ±1.3% — a tie.** No ptxas lineage (13.0/13.2/driver-JIT) moves our kernel, and vLLM's OWN PTX through the same assemblers runs the same as ours. The single 0.969 excursion (vLLM PTX+ptxas13.0) is contradicted by its two sister vLLM arms (driver-JIT 1.007, ptxas13.2 1.013) and by our-PTX+ptxas13.0 (1.004): neither ptxas 13.0 nor vLLM's PTX is systematically faster — it is the low tail of box drift.
+
+**THE ARBITER VERDICT: NO.** No old-ptxas cubin hits ~157 (none can be built; and the driver-JIT that vLLM actually uses was already our #75 Build C). The flash decode kernel's CODEGEN is at PARITY across every reachable toolchain AND against vLLM's own PTX. **The corrected mechanism:** in ISOLATION every variant is ~144–151 us, whereas in the real engine #75 measured ours 167 / vLLM 157 — so the +10 us/call gap is NOT in the flash kernel's SASS at all; it is ENGINE CONTEXT (the L2/TLB/clock state the neighbour kernels leave for the flash launch — exactly #69's "the flash kernel's cost moves with its NEIGHBOURS' L2 footprint"). This RETIRES #75's "ptxas SASS-scheduling quality" attribution. (Caveat, stated: the microbench is L2-warm — KV re-read back-to-back — so its ~144 us absolute sits below the engine's DRAM-cold 157–167; this does not affect the codegen arbitration because #75 showed the dominant stalls are smem-scoreboard/CTA-barrier, regime-independent, and they tie here across all toolchains.)
+
+**W2/W3 — none owed.** NO vendor (nothing beat the driver JIT we already use). Binding UNCHANGED: c1 1.020 / c2 0.962 / c4 0.966 / c8 0.969. THE PARITY VERDICT: MXFP4 stays BELOW-FLOOR at c2–c8, TERMINAL, with the corrected residual map — flash-kernel codegen at PARITY (this row); the c8 ~0.969 residual = flash ENGINE-CONTEXT adjacency (glue/marlin-neighbour L2/orchestration, a different lever than the kernel) + portable-fusion glue ~18% (numerics-delicate, sub-parity) + marlin/host remainder. No default flip owed. **hd256 (27B/35B) cross-model projection:** structurally identical — vLLM's fa2 wheel ships the hd256 splitkv kernels as sm_80-only PTX driver-JIT'd on GB10 too (no old-ptxas lineage, same driver JIT), so NO hd256 flash-ptxas vendor is owed either; the 27B/35B flash-decode codegen is at the same parity and their residuals are likewise context/glue, not ptxas.
+
+Evidence: `dgx:~/mxfp4-ptxas/{exp.sh,exp.log,bench_flash_ptxas.cu,*.cubin,our_c80fm.ptx,vllm.ptx}`; wheel `_vllm_fa2_C.abi3.so` cuobjdump (52×sm_80, PTX 9.0). The CMakeLists NOTE + this entry record the closed levers so they are not re-tried.
+
 ## QUANT-CT-MXFP4-FLASH-OCCUPANCY — the owed ours-vs-vLLM flash decode ncu diff: occupancy is IDENTICAL (8.33%, smem-limited); the flash gap is a COMPILER-CODEGEN difference (vLLM's wheel SASS runs 13% fewer instructions on the byte-identical kernel), NOT occupancy/L2/arch/fast-math — no lever exists on our stack (2026-08-06, `row/QUANT-CT-MXFP4-FLASH-OCCUPANCY`, base `f7a1e322`, GB10 sm_121a, PR #75)
 
 #69 closed the compile lens (flash source byte-identical to vLLM's `2c839c33`; `-use_fast_math` REJECTED, +21us/call) and OWED the ours-vs-vLLM ncu diff (vLLM side lost to a box OOM-reboot). This row runs that diff to a MEASURED verdict on an idle box and re-frames the #69 premise. **Vehicle** `Yi30/Qwen3-8B-MXFP4` (dense W4A16 Marlin), oracle arm `VLLM_DISABLED_KERNELS=FlashInferMxFp4LinearKernel`.

diff --git a/.agents/specs/mxfp4-flash-ptxas-2026-08-06.md b/.agents/specs/mxfp4-flash-ptxas-2026-08-06.md
@@ -0,0 +1,76 @@
+# QUANT-CT-MXFP4-FLASH-PTXAS — arbitrate the ptxas lineage behind vLLM's faster flash decode SASS
+
+<!-- spec-status: CLOSED -->
+Row: `QUANT-CT-MXFP4-FLASH-PTXAS` (helper, `row/QUANT-CT-MXFP4-FLASH-PTXAS`).
+Base: `origin/main` `362a3c99`. Vehicle: `Yi30/Qwen3-8B-MXFP4` (dense
+`Qwen3ForCausalLM`, W4A16 Marlin keep-quant); oracle arm
+`VLLM_DISABLED_KERNELS=FlashInferMxFp4LinearKernel`, vLLM 0.25.0-stage. GB10 sm_121a.
+
+## Why this row exists
+
+#75 closed the codegen lens to a MEASURED verdict: same `flash_fwd_splitkv` source
+(vendored from vLLM's pinned `2c839c33`), same grid (1×3×64), same 8.33% occupancy
+(both smem-limited to 1 CTA/SM by 81.92 KB), same L2 ~1%, and — with
+`compute_80 + -use_fast_math` (Build C) — matched vLLM's register (241) AND
+instruction (17,008 vs 17,020) counts EXACTLY. Yet Build C is ~167 us/call, vLLM
+is ~157 us/call. #75 attributed the residual to "vLLM's wheel `ptxas`
+SASS-scheduling quality (older CUDA 12.x lineage)". This row arbitrates that
+attribution to a MEASURED yes/no: get the wheel's exact ptxas lineage, re-assemble
+our flash PTX with it, and A/B the kernel.
+
+## W1 — arbitrate the ptxas lineage
+(a) Identify the wheel's toolkit lineage: `cuobjdump` the fa2 `.so`.
+(b) Obtain that ptxas (venv `nvidia-cuda-nvcc-cu12` wheels or a scratch pip download).
+(c) Compile our flash decode TU to PTX (compute_80 + fast-math per #75 Build C),
+assemble with the candidate ptxas lineages → cubin, load via `cuModule` in a
+microbench with the c8 decode params, and A/B (CUDA-event medians + ncu).
+THE ARBITER: does a different-lineage-ptxas cubin hit ~156-157 us? NO ⇒ refutation,
+close flash as measured-irreducible-for-us, STOP.
+
+## W2 — vendor (only on a YES)
+Mirror the GDN Triton-AOT precedent: commit the cubin + exact regen recipe under the
+vendored-kernels tree; a load path routing the sm_121a decode flash launch through
+the vendored cubin, gated `VT_FA2_VENDORED_CUBIN`, default per parity-enablers ONLY
+IF the full battery is green (near-tie razor for the fast-math numerics; #44 smoke;
+SACRED 0.6B/4B + 32B strict; async; memcheck; eager+graphed; both GQA ratios).
+hd256 (27B/35B) cross-model projection measured, not flipped, as a follow-up.
+
+## W3 — binding + verdict
+c1..c8 ×3 production defaults vs 1.020/0.962/0.966/0.969. THE PARITY VERDICT:
+≥1.0 every axis ⇒ MXFP4 parity DONE; short ⇒ honest terminal map with the fresh
+decomposition (flash-after-arbitration + glue ~18% + marlin/host remainder).
+
+## Gates
+Byte-exact razor: #44 MXFP4-8B smoke 3/3 + coherent. If fast-math shifts the
+reduction order: SACRED 0.6B/4B distributional + 32B strict, async, memcheck,
+eager+graphed. Box safety: BOTH flock locks, free -g ≥ 90, worker STOPPED, tmux +
+done-markers, sequential arms, single-load steady-state, disk floor 15G.
+
+## CLOSED — verdict (2026-08-06, THE ARBITER = NO)
+
+The ptxas-lineage hypothesis is REFUTED three ways. **(a) No old lineage exists:**
+`cuobjdump` of vLLM 0.25.0's `_vllm_fa2_C.abi3.so` = 52 `sm_80` cubins + 52
+`.target sm_80 .version 9.0` PTX (ISA 9.0 = **CUDA 13.0**, our own major), NO
+sm_12x cubin. On GB10 the sm_80 SASS cannot run, so vLLM's flash SASS is
+**driver-JIT'd from compute_80 PTX by the box CUDA-13.0 driver** — the SAME
+assembler #75's Build C (`code=compute_80`) already used. There is no separate
+"wheel ptxas" that baked a fast sm_121a cubin. **(b) Impossible by construction:**
+the only sub-13 ptxas on the box (12.8) tops out at sm_120a and cannot target
+sm_121a or read PTX 9.0 — an old-CUDA-12.x sm_121a cubin cannot be built at all.
+**(c) Measured tie:** a same-params cuModule A/B (harness `bench_flash_ptxas.cu`,
+c8 GQA-swap decode, grid (1,3,64), dyn-smem 81 920 B) times the byte-identical
+decode kernel from our compute_80+fast-math PTX and from vLLM's PTX #30, each via
+driver-JIT / ptxas 13.0 / ptxas 13.2 (all REG=241). Module/native ratios ∈
+[0.969, 1.013] — a ±1.3% tie; the lone 0.969 (vLLM-PTX+ptxas13.0) is contradicted
+by both sister vLLM arms (1.007, 1.013) and our-PTX+ptxas13.0 (1.004), i.e. box
+drift, not a lever. **Corrected mechanism:** the flash kernel codegen is at PARITY
+across every reachable toolchain AND vs vLLM's own PTX (~144–151 us in isolation);
+the +10 us/call the engine shows (ours 167 / vLLM 157, #75) is ENGINE CONTEXT
+(neighbour-kernel L2/orchestration, per #69), not the kernel's SASS. This RETIRES
+#75's "ptxas SASS-quality" attribution. **W2/W3 none owed:** no vendor (nothing
+beat the driver JIT we already use); binding UNCHANGED c1 1.020 / c2 0.962 / c4
+0.966 / c8 0.969; MXFP4 terminal below-floor at c2–c8 with the corrected map
+(flash codegen at parity; residual = context/glue/marlin-host). hd256 (27B/35B)
+projection: structurally identical (sm_80-only PTX driver-JIT'd), NO hd256 vendor
+owed. No functional code shipped; CMakeLists NOTE + benchmark-record (#82) capture
+the closed levers. No byte-exact razor / SACRED battery owed (nothing shipped).
diff --git a/.agents/state.md b/.agents/state.md
@@ -39742,3 +39742,51 @@ partition/supported_tasks guard mirroring upstream (community files strip the re
 and the encoder vision tower (W3) for clean image/video-conditioned ref2va/fl2va (ref2va with
 a synthetic reference + text-only encoder still grids). dgx assets: `~/h3fp4/ckpt/MiniMax-H3-
 FL2VA-Q3_K_M.gguf`, `~/h3fp4/fl2va_t2va_20/`. Box left clean.
+
+## 2026-08-07T00:45 - QUANT-CT-MXFP4-FLASH-PTXAS: the ptxas-lineage arbiter = NO; #75's "wheel ptxas SASS-quality" attribution RETIRED - vLLM's flash SASS is driver-JIT'd from CUDA-13.0 PTX, codegen ties across all ptxas AND vs vLLM's own PTX, the +10us gap is ENGINE CONTEXT (row/QUANT-CT-MXFP4-FLASH-PTXAS, helper, PR #82)
+<!-- state: 2026-08-07T00:45 -->
+
+The last MXFP4 flash lever. #75 owed obtaining vLLM's wheel `ptxas` (hypothesized
+"older CUDA 12.x lineage") and A/B'ing it against our flash PTX. Done. **Arbiter =
+NO, refuted three ways.**
+
+- **(a) No old lineage / no runnable GB10 cubin.** `cuobjdump` of vLLM 0.25.0's
+  `_vllm_fa2_C.abi3.so` = 52 `sm_80` cubins + 52 `.target sm_80 .version 9.0` PTX.
+  PTX ISA 9.0 = CUDA 13.0 (our own major), NOT 12.x; NO sm_12x cubin. On GB10 the
+  sm_80 SASS can't run, so vLLM's flash SASS is **driver-JIT'd from compute_80 PTX
+  by the box CUDA-13.0 driver (580.159.03)** - the SAME assembler #75's Build C
+  (`code=compute_80`) already used. The "wheel ptxas baked a fast sm_121a cubin"
+  premise is factually wrong.
+- **(b) Impossible by construction.** Box ptxas = 12.8 (triton), 13.0 (toolkit,
+  build default), 13.2 (nvidia/cu13). ptxas 12.8 tops out at sm_120a - it CANNOT
+  target sm_121a (first in 13.0) nor read PTX 9.0. So an old-12.x sm_121a cubin
+  cannot be built at all.
+- **(c) Measured tie.** Standalone cuModule harness (`dgx:~/mxfp4-ptxas/`,
+  `bench_flash_ptxas.cu`) times the byte-identical c8 GQA-swap decode kernel
+  (grid (1,3,64), dyn-smem 81 920 B, all cubins REG=241) from our compute_80+
+  fast-math PTX and vLLM's PTX #30, each via driver-JIT / ptxas 13.0 / ptxas 13.2.
+  Module/native ratios ∈ [0.969, 1.013] - a ±1.3% tie (native anchors drifted
+  138.7-149.4 across the sequential arms; the lone 0.969 is contradicted by both
+  sister vLLM arms 1.007/1.013 and our-PTX+ptxas13.0 1.004 = box drift, not a
+  lever).
+
+**Corrected mechanism:** the flash decode kernel codegen is at PARITY across every
+reachable toolchain AND vs vLLM's own PTX (~144-151 us in isolation); the +10
+us/call the engine showed (ours 167 / vLLM 157, #75) is ENGINE CONTEXT
+(neighbour-kernel L2/orchestration, exactly #69's finding), NOT the kernel's SASS.
+This RETIRES #75's "ptxas SASS-scheduling quality" attribution. (Microbench is
+L2-warm so its absolute sits below the engine's DRAM-cold number; irrelevant to the
+codegen arbitration since the dominant stalls are smem-scoreboard/CTA-barrier per
+#75, regime-independent, and they tie here.)
+
+**Verdict / follow-up.** W2/W3 none owed: NO vendor (nothing beat the driver JIT we
+already use), binding UNCHANGED c1 1.020 / c2 0.962 / c4 0.966 / c8 0.969, MXFP4
+TERMINAL below-floor at c2-c8 with the corrected map (flash codegen at parity;
+residual = flash engine-context adjacency + portable-fusion glue ~18% +
+marlin/host). No default flip owed. hd256 (27B/35B) cross-model projection:
+structurally identical (sm_80-only PTX driver-JIT'd), NO hd256 vendor owed; their
+flash-decode residuals are likewise context/glue, not ptxas. No functional code
+shipped; CMakeLists NOTE + benchmark-record (#82) + spec CLOSED capture the closed
+levers. Box: experiment left artifacts under `~/mxfp4-ptxas/`; box OOM-rebooted
+mid-confirmatory-run (another agent's 25 GiB vLLM on the 119 GiB unified pool, not
+this <1 GiB microbench) - run-1 data is on disk and decisive.
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -1430,9 +1430,14 @@ if(VLLM_CPP_CUDA)
     # instruction-bound, and the residual is vLLM's wheel-`ptxas` SASS-SCHEDULING
     # quality, not reachable from our nvcc 13.0. (native+fast-math REGRESSES to
     # 189.8 us, #68; compute_80 PTX driver-JIT alone is neutral, #75 Build B.) So
-    # we keep the plain native sm_121a compile and pass no -use_fast_math. The
-    # residual is an irreducible-for-us ptxas SASS-quality gap on a byte-identical
-    # kernel; see .agents/benchmark-record.md (#75).
+    # we keep the plain native sm_121a compile and pass no -use_fast_math. #82
+    # (FLASH-PTXAS) then ARBITRATED the "ptxas lineage" claim to a NO and CORRECTED
+    # it: the wheel ships NO sm_12x cubin — only CUDA-13.0 (PTX ISA 9.0) sm_80 PTX
+    # driver-JIT'd on GB10 — so there is no old-CUDA-12.x ptxas in vLLM's path (and
+    # ptxas 12.8 cannot even target sm_121a); a same-params cuModule A/B ties our
+    # PTX and vLLM's OWN PTX at ~144 us across driver-JIT / ptxas 13.0 / 13.2, so the
+    # +10 us/call gap is ENGINE CONTEXT (neighbour L2/orchestration), not this
+    # kernel's codegen. See .agents/benchmark-record.md (#75, #82).
     set_source_files_properties(
       src/vt/cuda/cuda_flash_attn_fa2.cu
       ${_FA2_KERNEL_SRCS} PROPERTIES