spike(mxfp4-flash): arbitrate the ptxas lineage behind vLLM's faster flash decode (QUANT-CT-MXFP4-FLASH-PTXAS) - #82
Closed
localai-bot wants to merge 2 commits into
Closed
Conversation
…lineage The last MXFP4 flash lever. #75 matched vLLM's reg(241)+instr(17k) with compute_80+fast-math yet stays ~167 vs ~157 us/call, attributing the residual to vLLM's wheel-ptxas SASS-scheduling lineage. This row arbitrates that to a MEASURED yes/no: identify the wheel's ptxas lineage, re-assemble our flash PTX with it, A/B the c8 decode kernel. YES -> vendor the cubin (GDN Triton-AOT precedent). NO -> close flash as measured-irreducible with a corrected mechanism. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
…ineage REFUTED, gap is engine context not codegen #75 owed obtaining vLLM's wheel ptxas (hypothesized older CUDA 12.x lineage) and A/B'ing it. Done. The lineage hypothesis dies three ways: (a) cuobjdump of vLLM 0.25.0 _vllm_fa2_C.abi3.so = 52 sm_80 cubins + 52 PTX .version 9.0 (= CUDA 13.0, our own major), NO sm_12x cubin. On GB10 the sm_80 SASS can't run, so vLLM's flash SASS is DRIVER-JIT'd from compute_80 PTX by the box CUDA-13.0 driver — the same assembler #75's Build C already used. There is no older-CUDA-12.x wheel ptxas anywhere in vLLM's path. (b) ptxas 12.8 (the only sub-13 ptxas on the box) tops out at sm_120a and cannot target sm_121a or read PTX 9.0 — an old-12.x sm_121a cubin is impossible. (c) same-params cuModule A/B: the byte-identical c8 decode kernel from our compute_80+fast-math PTX and vLLM's own PTX #30, via driver-JIT / ptxas 13.0 / ptxas 13.2 (all REG=241), ties at ~144us (module/native ratios 0.969-1.013, a +-1.3% tie; the lone 0.969 is box-drift, contradicted by its sisters). Corrected mechanism: flash-kernel CODEGEN is at PARITY across every reachable toolchain AND vs vLLM's own PTX; the +10us/call the engine showed (167 vs 157) is ENGINE CONTEXT (neighbour L2/orchestration, per #69), NOT ptxas. Retires #75's "ptxas SASS-scheduling quality" attribution. No vendor (nothing beat the driver JIT we already use). Binding UNCHANGED c1 1.020 / c2 0.962 / c4 0.966 / c8 0.969; MXFP4 TERMINAL below-floor at c2-c8. hd256 (27B/35B) projection: same structure (sm_80-only PTX driver-JIT'd), no hd256 vendor owed. No functional code shipped; CMakeLists NOTE + benchmark-record (#82) + spec CLOSED record the closed levers. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
localai-bot
marked this pull request as ready for review
August 6, 2026 23:14
Collaborator
Author
|
Landed on main as the squash (mudler authorship). MXFP4 terminal is now complete to the assembler-lineage level; the engine-context adjacency term is recorded as the residual class. 🤖 Generated with Claude Code |
mudler
added a commit
that referenced
this pull request
Aug 6, 2026
…RITY; MXFP4 terminal complete (#82) row/QUANT-CT-MXFP4-FLASH-PTXAS squash, docs-only. THE ARBITER: NO. (1) vLLM's wheel ships ONLY sm_80 cubins + sm_80 PTX at ISA 9.0 = CUDA 13.0 — the SAME lineage as our nvcc; on GB10 its flash SASS is driver-JIT'd from that PTX, never made by a "wheel ptxas". (2) A CUDA 12.x sm_121a cubin is impossible — 12.x ptxas cannot target sm_121a nor read ISA 9.0. (3) The six-arm cuModule A/B (our PTX and vLLM's own PTX, each under driver-JIT/ptxas-13.0/13.2; identical mangled kernel, grid, 81920B smem, REG=241) ties at [0.969, 1.013] — +/-1.3% box drift. Retires #75's "ptxas SASS-scheduling" attribution: in isolation every arm is ~144-151us, so the engine's +10us/call is ENGINE-CONTEXT adjacency (neighbour L2/orchestration), not kernel SASS. hd256 (27B/35B) is structurally identical — no vendor owed anywhere. MXFP4 TERMINAL, now complete to the assembler-lineage level: c1 1.020 PASS + mem 2.63x; c2-c8 0.962-0.969 with the final residual map = flash engine-context adjacency + portable glue (~18%, numerics- delicate) + marlin/host remainder — all sub-parity levers. No default flip; binding unchanged. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
main moved 17 commits under this branch (#82, #83, #84 plus the public-doc gate restoration), so every earlier green result was re-run on the merged tree rather than carried over. Three record conflicts, all resolved BY KEY rather than by taking a side: * docs/STATUS.md — main had COMPACTED this paragraph and lowered the size ratchet to 284062. Taking our side wholesale would have silently reverted their compaction and blown the gate, so the resolution keeps THEIR paragraph and swaps in only our Vulkan clause, then trims it to fit. * docs/BENCHMARKS.md — a keyed table. Kept THEIR rows (they updated MiniMax-H3 and MXFP4 and added H3-RENDER-CLOSE) and appended only our Vulkan row. * .agents/state.md — append-only log, unioned theirs then ours. Re-verified on the merged tree: clean -Werror Vulkan-ON build 0 warnings, e2e opt-125m on Vulkan STRICT token-exact 6/6 prompts / 96/96 tokens with 0 provider declines, test_vulkan_backend 10/10 (480), test_backend_cross_device 11/11 (123), clean CPU-only build 0 warnings, CPU ctest 336/337 with the one failure (test_engine_core_proc) passing serially in 0.01s — the known starve-under-`-j` pattern on a box running several suites at once, not a regression.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QUANT-CT-MXFP4-FLASH-PTXAS — arbiter = NO (ptxas-lineage REFUTED)
Verdict: the last MXFP4 flash lever is closed. #75's "vLLM's wheel
ptxasSASS-scheduling quality (older CUDA 12.x lineage)" attribution is REFUTED three
ways, and the +10 us/call c8 flash gap is re-attributed to ENGINE CONTEXT, not the
flash kernel's codegen.
cuobjdumpof vLLM 0.25.0_vllm_fa2_C.abi3.so= 52sm_80cubins + 52.target sm_80 .version 9.0PTX (ISA 9.0 = CUDA 13.0, ourown major), NO sm_12x cubin. On GB10 the sm_80 SASS can't run, so vLLM's flash
SASS is driver-JIT'd from compute_80 PTX by the box CUDA-13.0 driver — the
same assembler QUANT-CT-MXFP4-FLASH-OCCUPANCY: the owed ours-vs-vLLM flash decode ncu diff + occupancy/L2/codegen lever #75's Build C (
code=compute_80) already used.at sm_120a — it cannot target sm_121a or read PTX 9.0.
decode kernel from our compute_80+fast-math PTX and vLLM's own PTX fix(metal): scope MLX AppleClang warning suppression #30, via
driver-JIT / ptxas 13.0 / ptxas 13.2 (all REG=241): module/native ratios
0.969–1.013 (±1.3% tie; the lone 0.969 is box drift, contradicted by its sisters).
In ISOLATION every variant is ~144–151 us; the engine's 167 (ours) vs 157 (vLLM)
is ENGINE CONTEXT (neighbour L2/orchestration, per #69). Binding UNCHANGED: c1
1.020 / c2 0.962 / c4 0.966 / c8 0.969 — MXFP4 TERMINAL below-floor at c2–c8.
hd256 (27B/35B) projection: same structure, no vendor owed.
No functional code shipped (docs-only refutation): benchmark-record #82 +
CMakeLists NOTE + spec CLOSED + STATUS/BENCHMARKS/NOW/state. Note: the repo's
pre-existing
check-fusion-consistencyred (minimax_h3_video_vae_device, presenton
main) is unrelated to this PR.FOLLOWING_AGENTS_PROTOCOL