Vulkan: a model runs end to end, token-exact — 16 native kernels, variant pipeline, two CI gates - #80
Merged
Merged
Conversation
Task-by-task plan for sub-project VK-A1 of .agents/specs/vulkan-full-support.md. Plan only: no source, no shader, no build. The decision the plan implements: keep the committed-SPIR-V route so the build stays hermetic on every box, but change the VARIANT mechanism from GLSL #defines to SPIR-V specialization constants. llama.cpp needs 242 string_to_spv call sites largely because a #define variant is a whole new module; a specialization constant is one module specialized at pipeline creation, so artifact count tracks shader FILES rather than the dtype x quant x coopmat-tier cross product. Seven tasks: pin glslang and prove the committed SPIR-V reproduces; a CI staleness gate with a mutation proof; SpecId metadata in the generated table; specialization plumbed through Dispatch with the cache keyed by it; the workgroup size collapsed from three copies to one constant; SPIR-V words moved out of the header; the feature-matrix/backend-matrix drift repaired. Two preconditions are recorded because they block execution rather than the plan: disk is at 99% (4.8 GB free), which makes any build unreliable, and neither box has a GLSL compiler, so no .comp can change until Task 1. One deliberate scope narrowing against the spec's wording is called out in the self-review: the vt::arch_tactics generalization is left to VK-C, since a tactic registry with exactly one tactic would be speculative. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…oduces VK-A1 Task 1. Neither box had a GLSL compiler, so no .comp file could be changed without invalidating the committed SPIR-V, and no CI gate could check it. Two measured corrections to the recorded pin. The header claims "Glslang Version: 11:16.4.0", but 16.4.0 ships NO release assets -- only 16.5.0 and main-tot do, and Ubuntu packages 15.1.0 -- so the recorded version is not fetchable and could never have backed a CI job. And the committed SPIR-V reproduces BYTE-FOR-BYTE under 16.5.0: `gen-vulkan-spirv.py --check` passes, exit 0. That second result is stronger than the check was written to obtain. --check compares SPIR-V bytes and ignores only the version comment, so a byte-identical result under a DIFFERENT, newer compiler proves both that the committed artifact is genuinely what it claims to be, and that the emitted SPIR-V did not move across a glslang minor bump. The freshness gate can therefore pin the download URL instead of asserting a version string, which would have been brittle for no gain. Nothing is linked against glslang; it is a build-time tool. The plan is updated in the same change to record the corrected pin and why. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
The committed-artifact route trades a build-time shader toolchain for the obligation to regenerate by hand, and nothing enforced that obligation: --check existed but no job ran it, so a .comp edit without a regenerate shipped silently. That is the one way the hermetic-build trade can go wrong, and it goes wrong quietly. The glslang DOWNLOAD URL is the pin. An exact version-string assertion was deliberately not used: the committed SPIR-V is byte-identical under both the 16.4.0 that produced it and the 16.5.0 pinned here, so gating on the string would be brittle without buying anything, and a future glslang codegen change will surface as STALE either way. Mutation-proved rather than assumed: rewriting relu's select as an equivalent expression (v > 0.0 ? v : 0.0 -> v < 0.0 ? 0.0 : v) turns the gate RED (exit 1), and reverting turns it green. A comment-only edit would not have proved it, since -g0 can emit identical SPIR-V. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
The Vulkan unit gate was RED on main and nothing could see it. test_vulkan_backend asserted that unimplemented ops throw. Since accelerator-seam row S5 (af0b21b) they do not: a unified-memory device that misses GetOp lazily installs the CPU kernel as a priority -1000 provider, and Vulkan is eligible (GB10 integrated and llvmpipe both report unified). The Metal sibling was updated for this as Metal work continued (test_metal_backend.cpp:215-231); Vulkan was not, because VLLM_CPP_VULKAN=ON appeared NOWHERE in ci.yml -- the backend was built on no machine, so its whole suite ran nowhere and the assertion rotted from the moment S5 landed. MEASURED on this tree before repairing it: of 87 CPU-registered ops, 8 are NATIVE on Vulkan, 79 are served by the portable reference tier, and ZERO throw. So the record's "vt::GetOp throws" and "8 of 83" are both wrong, and every op a model needs already resolves on Vulkan -- on the host. (Op resolution is not an end-to-end claim: get_attn_backend_priority() is still deliberately EMPTY, a separate gate at the platform seam.) The assertion is rewritten in the Metal sibling's exact form, so the two backends fail the same way: the op resolves, and the provider that served it is checked BY NAME against kReferenceProviderName, so a host kernel can never masquerade as a native Vulkan one. The new build-test-vulkan leg runs GPU-free on llvmpipe and executes both the backend gate and the cross-device numerics gate against the CPU oracle. Verified locally: 8/8 (87 assertions) and 6/6 (73 assertions). FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…rkgroup size Two halves: the metadata the variant mechanism needs, and a MEASURED negative result about the constant everyone would reach for first. The generator now parses OpDecorate SpecId out of each emitted module and records the sorted IDs beside the blob. The host will pass specialization values BY ID, and Vulkan SILENTLY IGNORES a map entry whose ID the module does not declare, so without this table a host/shader drift is wrong numbers rather than a clean failure. Parsing the module is the only source of truth: glslang exposes no side-channel listing, and a hand-maintained list beside the shaders is exactly the duplicate that drifts. Covered by unit tests including the two ways a naive scanner breaks (a non-SpecId decoration, and stepping over unrelated opcodes by word count). The workgroup size is NOT that constant, and the attempt is recorded rather than silently abandoned. VT_TG is written down three times (vt_common.glsl, each .comp's local_size_x, kWorkgroupSize on the host, which derives the workgroup COUNT from it) -- precisely the shape a specialization constant should collapse. It cannot at this target: layout(local_size_x_id = 0) makes glslang emit `ExecutionMode LocalSize 1 1 1` plus the legacy BuiltIn WorkgroupSize vector, because the modern LocalSizeId mode needs SPIR-V 1.2 + VK_KHR_maintenance4 (core in Vulkan 1.3) and this backend targets vulkan1.1 deliberately. MEASURED on llvmpipe: the literal LocalSize 1 wins, each workgroup runs ONE thread against a ceil(n/128) dispatch, and cross-device NMSE went from ~1e-14 to ~0.99 on kAdd. An A/B isolated it -- keeping the constant and reverting only local_size_x_id is fully green -- so specialization constants work; the launch geometry is what cannot use them here. Half-converting is worse than not converting: a host-settable VT_TG whose value the actual workgroup size does not follow would silently corrupt. So VT_TG stays a #define with the measurement written next to it, and the mechanism is left for axes that need not agree with launch geometry (dtype, quant format, coopmat tier). The gates now assert ZERO declared constants as a fact, so the first shader that takes one flips them loudly. SPIR-V words are byte-identical to before (109,436 bytes); only the table gained columns. Vulkan gate 9/9 (101 assertions), cross-device 6/6 (73). FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
… first axis The variant mechanism, now exercised by a real shader instead of a synthetic one. Dispatch takes specialization values, the pipeline cache is keyed by name PLUS those values, and the count is checked against the module's declared SpecIds -- Vulkan silently ignores a map entry whose constantID the module does not declare, so an unchecked drift is wrong numbers rather than an error, the same class as the existing binding-layout check. The VkSpecializationInfo and its map entries are named locals that outlive vkCreateComputePipelines, not temporaries whose address escapes; that use-after-free class has bitten this project twice already under CUDA-graph capture. vt_cast is the first axis. Its source and destination dtype were push constants re-tested per element inside VT_LOAD/VT_STORE; they are now specialization constants 0 and 1, so the driver folds them at pipeline creation and eliminates the branch that cannot be taken. ONE committed module still serves every (src, dst) pair -- which is the whole argument against llama.cpp's spelling, where the same axis is a GLSL #define and therefore a separate SPIR-V module per combination. The module got SMALLER, 109,436 -> 109,216 bytes, because the dead dtype branches are gone. Gated on the property that matters: the cache must GROW BY TWO across two dtype pairs. Results alone prove nothing here, since the shader defaults are f32->f32 and a specialization that silently did nothing would also look right. Numerics are checked in the BIT-EXACT tier, not the NMSE tier -- the f32->bf16->f32 round trip must equal the CPU codec exactly -- and cross-device is unchanged at 6/6, 73 assertions. Clean full -Werror build with Vulkan ON: exit 0, 0 warnings. CMakeLists is untouched, so VLLM_CPP_VULKAN AUTO still resolves OFF and the CUDA gate build is unaffected by construction. Vulkan gate 10/10, 405 assertions. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
… in the header At 7 modules the generated header was 3,487 lines and two TUs included it. At the target shader surface it is megabytes of constexpr array initializer that every includer re-parses, and every regeneration is a huge header diff. The words now live in a generated vulkan_spirv.cpp; the header carries the struct plus extern declarations and is 55 lines. Adding shaders now costs ONE TU's compile time instead of all of them. Doing this at 7 modules is cheap; at 100 it is not. Consumers move from sizeof/sizeof to the generated kSpirvModuleCount, since an extern array has no bound at the use site, and --check now covers both files. FIXES A LATENT BUG INTRODUCED BY THE SPECIALIZATION WORK. The descriptor pool was sized maxSets = one per MODULE, which was right when a module meant a pipeline. It no longer does: vt_cast alone reaches one pipeline per (src, dst) dtype pair, each allocating its own descriptor set, so the pool would have been exhausted by the Nth specialization and failed inside vkAllocateDescriptorSets -- a bare VkResult a long way from its cause. The pool is now sized with explicit specialization headroom and that allocation names the pipeline and the likely reason. The split is asserted, not assumed: a test fails if the word arrays reappear in the header, which would otherwise be invisible until compile times grew. Clean -Werror build with Vulkan ON: exit 0, 0 warnings. Vulkan gate 10/10 (405 assertions), cross-device 6/6 (73), generator suite 9/9, --check green on both generated files. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
Repairs the BACKEND-VULKAN drift this campaign was scoped to fix, and corrects two claims the campaign spec itself made three commits ago. DRIFT: feature-matrix carried INVENTORIED / "runtime absent" against backend-matrix's ACTIVE for the same row. backend-matrix was right. CORRECTION, which matters more. The spec stated -- from the backend matrix -- that Vulkan's unimplemented ops make vt::GetOp THROW, and put the surface at "8 of 83". Both are false. Accelerator-seam row S5 (af0b21b) gave unified-memory devices the portable reference tier, and Vulkan is eligible. Re-counted at RUNTIME on a Vulkan-ON build: of 87 CPU-registered ops, 8 are NATIVE, 79 are served by the reference tier (the CPU kernel against shared memory), and ZERO throw. Every op a model needs already resolves, so the campaign's real content is moving those 79 off the host -- a performance project, not a make-it-run one, and GetReferenceTierHits() is the progress metric that must reach 0. HONEST LIMIT, stated wherever the numbers are: op resolution is not an end-to-end claim. get_attn_backend_priority() is still deliberately EMPTY, which is a separate gate at the platform seam, and no model has been run on Vulkan. Also records the measured negative result so nobody retries it: the workgroup size cannot be a specialization constant at the vulkan1.1 target, because local_size_x_id emits ExecutionMode LocalSize 1 1 1 and computes ~1/128 of each tensor (NMSE 0.99 on kAdd). No speed number measured, claimed or owed. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
First VK-B brick: 8 native kernels -> 10, and the first op the reference tier was carrying that a model actually spends its time in. NUMERIC CONTRACT ported from cpu_ops.cpp MatmulChunked (:187-260): one invocation per OUTPUT ELEMENT with the whole K reduction on it, sequential f32 accumulation, rounded once on store. The CPU kernel deliberately never splits a K reduction across threads, so keeping one invocation per element makes the two backends share an accumulation ORDER rather than merely a tolerance. DELIBERATELY NOT TILED. This is the portable correctness tier: no shared-memory blocking, no cooperative matrix. llama.cpp's mul_mm.comp (scalar + coopmat1 by define) and mul_mm_cm2.comp (NV coopmat2) are the performance port and belong to VK-C with its tactic selection -- which needs exactly this as a known-correct same-device A/B reference rather than a rewrite to compare against. ONE MODULE SERVES 54 VARIANTS: three dtypes each for a, b and out, times the two orientations, as specialization constants. Folding the orientation at pipeline creation also removes a per-ELEMENT branch from the inner K loop. llama.cpp spells the same axis as #defines and emits a module per combination; this is the mechanism VK-A1 built, now paying for itself at 54:1. Gated against the CPU oracle at ragged shapes (M=13, K=37, N=9 -- none a multiple of the workgroup size, so a kernel that only handled whole tiles would fail rather than pass on a friendly shape), both orientations separately since MatmulBT is a different indexing path and not a transpose of the same code. Three gates moved and are updated as executable facts rather than loosened: the registered/unregistered op lists, the module count and name table, and the reference-tier probe, which now uses kPagedAttention because kMatmul is no longer a fallback and would not exercise the tier. Clean -Werror build 0 warnings. Vulkan gate 10/10 (417 assertions), cross-device 7/7 (80), generator 9/9, --check green. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
The two ends of the model: token ids in, token ids out. 10 native kernels now. EmbeddingKernel (cpu_ops.cpp:661-672) is one output ELEMENT per invocation, with the id width (i32 vs i64, which vt::Embedding accepts both of) as a specialization constant rather than a per-element branch. Only the low 32 bits of an i64 id are read: an id is a vocabulary index that the host already range-checked, so it is far below 2^31, and reconstructing the full value would need VK_KHR_shader_int64 at a 1.1 floor for nothing. The CPU kernel's VT_CHECK on the range has no shader equivalent -- a shader cannot throw and a per-element bounds branch would cost the gather -- but vt::Embedding validates on the host before dispatch, so the error is already caught by the time this runs. GreedyArgmaxKernel (cpu_sample.cpp:40-56) is ONE INVOCATION PER ROW, and that is a contract decision rather than laziness. The CPU scan uses a strict `>` so the FIRST occurrence of the maximum wins; greedy decoding is compared token-for-token against a vLLM golden, so a tie-indifferent tree reduction returns a different token and fails the gate. A parallel reduction has to carry the index and break ties toward the lower one at every merge, which is a separate change with its own gate. Decode rows are few; the vocabulary scan is the slow axis and is left. Both are gated EXACTLY, not by NMSE -- a gather moves bytes and an argmax picks an index. The embedding case uses REPEATED ids (13 twice) so a kernel that consumed ids positionally fails, and covers i32 and i64 separately since they are different index paths. The argmax case plants a DELIBERATE TIE in row 0 at columns 2 and 5 and requires 2, which is what a `>=` or a tie-indifferent reduction would get wrong; the oracle is asserted to honour it first, so the test cannot pass by both sides being wrong together. Binder gains AddU32Only for operands bound through a single view -- integer ids and f32-by-contract logits -- because a descriptor a shader does not declare must not be written. Clean -Werror build 0 warnings. Vulkan gate 10/10 (434 assertions), cross-device 8/8 (88), generator 9/9. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
Re-measured with the same runtime probe rather than re-grepped, so the number is comparable to the VK-A1 baseline: native 8 -> 12, reference-tier 79 -> 75, still zero throwing. The four that moved are kMatmul, kMatmulBT, kEmbedding and kGreedyArgmax. Carries the same honest limits everywhere the counts appear: paged attention, the KV cache ops, the whole RoPE family, quant, MoE, GDN/MLA and every sampler beyond greedy argmax are still the CPU kernel running on shared memory; get_attn_backend_priority() is still EMPTY; no model runs end to end on Vulkan; no speed number is measured, claimed or owed. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
13 native kernels now. This is the ONE kernel in the backend with no Vulkan port source: grep over all 132 .comp + 26 .glsl at pin 237ad9b96 finds ZERO block_table/paged matches, because ggml attention takes a contiguous KV buffer plus a mask. So the online-softmax skeleton follows flash_attn.comp's shape while the block-table indirection, GQA mapping and windowing are ported from our own cpu_paged_attn.cpp PagedAttentionKernel (:52-171), which is also the oracle. ONLINE SOFTMAX RATHER THAN THE CPU'S THREE PASSES, and the reason is structural, not preference: the CPU kernel materialises a `probs` array of one float per key in the window (:121,131), which is unbounded -- thousands at long context -- and a shader has no such allocation. The passes fuse into the standard running max/denominator recurrence, same mathematical result with a different rounding order, which is why this op sits in the NMSE tier and claims no bit-exactness. ONE WORKGROUP per (query token, query head), lanes splitting the head dimension, each key's q.k dot a cooperative vt_tg_sum. No broadcast of the score is needed: vt_tg_sum returns smem[0] to every lane and its leading barrier makes back-to-back calls safe, so all lanes derive the score from the same dot with the same instructions. An earlier draft added a lane-0 broadcast and two extra barriers for this; removing them is both simpler and one less barrier hazard. fp8 KV cache DECLINES rather than throws. Those pages are 1-byte and must be dequantised as Dequant(fp8) * k_scale|v_scale before the f32 softmax (cpu_paged_attn.cpp:79-93), which this shader does not do -- and throwing would REMOVE a capability the portable reference tier already provides. It forwards through GetOpFallback, which op_provider.h:94-100 documents as exactly the place for a per-call refusal, since GetOp has no shape or dtype to inspect. Gated against the CPU oracle in THREE configurations, because they are different branches and a single causal case would leave two unexercised: causal, causal+softcap (cap * tanh(s/cap)), and a sliding window. The block table is deliberately NON-IDENTITY (page j at cache block kBlocks-1-j) so a kernel that ignored it and indexed the cache linearly fails, and 37 tokens over block size 4 leaves the last page partly occupied so a whole-page walk reads past the sequence. Native 12 -> 13, reference-tier 75 -> 74. Clean -Werror build 0 warnings. Vulkan gate 10/10 (447 assertions), cross-device 9/9 (103), generator 10/10. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…s box Refreshes the measured counts (native 12 -> 13, reference tier 75 -> 74) after paged attention landed, re-measured with the same runtime probe. Also records the answer to "can we test Vulkan on AMD here": NO, and the reason is worth keeping. The dev box is a KVM virtual machine and the AMD is the CPU (Ryzen 9 9950X3D), not a GPU -- display is QEMU virtual VGA 1234:1111, /dev/dri has no renderD* node at all, and Vulkan enumerates only llvmpipe. The useful corollary is that RADV (radeon_icd.json) is already installed, so an AMD GPU passed into this VM would enumerate with zero code changes and VK-I's discrete-GPU staging path -- still dead code, since GB10 and llvmpipe are both unified -- would become testable. The blocker is hardware passthrough, not software. Noted but not acted on: environment.md describes the dev box as having an RTX 5070 Ti, which this VM does not see. Left for the owner rather than guessed at. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
14 native kernels. This completes the attention block's device side: paged attention reads the cache, this writes it. Ported from cpu_cache.cpp ReshapeAndCacheKernel (:33-72), which is two memcpys per token and CONVERTS NOTHING. So the dtype here selects only the storage WIDTH to copy at -- 32-bit through the uint32 view, 16-bit through the uint16 view, never mixing -- and the gate is BIT-EXACTNESS rather than NMSE. The host refuses a source/cache dtype mismatch rather than silently converting. SLOT -1 IS NOT AN ERROR and the gate proves the kernel knows it. Upstream pads the slot mapping and marks padded tokens negative (:60); those tokens are SKIPPED, leaving the page as it was. The mapping is i64, so the shader reads both halves and tests the HIGH word for the sign -- reading it as unsigned would turn -1 into 0xFFFFFFFF and index astronomically out of range. The gate seeds the cache with RANDOM contents rather than zeros, so "the padded token left the page intact" cannot pass vacuously, and uses scattered out-of-order slots (20, -1, 3, 11, -1, 0, 23, 7, 15) so a kernel that assumed slot == token index fails. Native 13 -> 14, reference tier 74 -> 73. Clean -Werror build 0 warnings. Vulkan gate 10/10 (454 assertions), cross-device 10/10 (109), generator 11/11. NEXT IS RoPE, AND IT NEEDS A DECISION FIRST. The CPU kernel computes the angle in DOUBLE (cpu_ops.cpp:701-705: std::pow for the frequency, std::cos/std::sin of pos * freq). In f32 that loses precision badly at long context, where pos is in the thousands and the angle is large. Shipping an f32 transcription would be subtly wrong in exactly the regime that matters, so it is not done here. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…efore more work Refreshes the measured counts after the KV-cache write landed (native 13 -> 14, reference tier 74 -> 73), and records the reason the next brick stopped at a decision rather than a commit. RoPE's CPU kernel computes the angle in DOUBLE (cpu_ops.cpp:701-705). An f32 transcription loses precision badly at long context, where pos is in the thousands and the angle is large -- subtly wrong in exactly the regime that matters, and it would still pass a short-sequence gate, which is the dangerous combination. Two ways out are on record: require shaderFloat64 and transcribe faithfully, or implement kRopeFromCache (the apply, pure f32 multiply-add) natively and leave kRopeCosSinCache (the once-per-model table build, where the double math lives) on the reference tier. The second matches vLLM's own structure and costs no device feature; it is the recommendation. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…LLM oracle
opt-125m, greedy, on llvmpipe:
the engine selected device type 3 (VULKAN)
BACKEND PROOF -- all 9 OPT ops dispatched on device type 3 with 0 declines
(kPagedAttention selections=1152)
STRICT correctness gate: 6/6 prompts token-exact (96/96 tokens)
vs the vLLM 0.25.0 oracle
The strong form of the claim, not "it produced plausible text". The gate first
verifies vLLM's own greedy is deterministic here (0 multi-valued cells over K=5
runs) so it is the STRICT bar rather than the distributional one; the engine picks
Vulkan itself through CurrentPlatform() with no test-side override; and ZERO
declines is the load-bearing half, because a provider can be selected and then
decline INSIDE its kernel and forward to the CPU tier -- 1152 kPagedAttention
selections with no declines means attention really ran on Vulkan rather than the
host fallback wearing its name.
Three pieces got it there.
RoPE, SPLIT THE WAY vLLM SPLITS IT. RotaryEmbedding builds cos_sin_cache once in
__init__ and the forward only applies it, which is why ops.rotary_embedding()
takes the cache rather than a base and a scaling factor
(rotary_embedding/base.py:160-252, common.py:145-185 @ e24d1b24fe96). So the table
build stays on the portable tier -- where the double-precision pow/cos/sin lives
-- and the per-token apply is native. Faithfulness and numerics agreed here: an
f32 transcription of the angle construction would be wrong at long context and
would still pass a short-sequence gate. mrope DECLINES through the provider seam
rather than throwing, like fp8 KV. Gated on both NeoX and GPT-J pairings, with
positions deliberately NOT 0..n-1 so a kernel using the token index instead of the
position fails.
kQkvSplit mirrors QKVParallelLinear's qkv.split([q_size, kv_size, kv_size]) --
three INDEPENDENT widths, because under GQA k and v are narrower than q.
FLASH_ATTN registered for kVULKAN on exactly Metal's footing. The real
precondition is not the device name but that our kPagedAttention and
kReshapeAndCache read and write the same NHD layout get_kv_cache_shape allocates,
which they do since both are ports of the CPU pair. MLA still returns EMPTY
DELIBERATELY: kMlaDecodeAttention / kMlaPrefillAttention / kConcatAndCacheMla have
no Vulkan kernel, and naming a backend there would route an MLA model into one
that cannot serve it.
Native 14 -> 16 (measured with the runtime probe, not counted by hand). Clean -Werror Vulkan-ON build 0 warnings; Vulkan gate 10/10 (480
assertions), cross-device 11/11 (123), test_opt_load 1/1 (958), generator 13/13.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-opus-5 [Claude Code]
…kernels opt-125m greedy: engine selects VULKAN on its own, all 9 OPT ops dispatch natively with 0 provider declines (kPagedAttention selections=1152), STRICT token-exact 6/6 prompts / 96/96 tokens vs the vLLM 0.25.0 oracle. Recorded with its limits attached everywhere the claim appears. It is a CORRECTNESS proof measured on llvmpipe, the software rasterizer, because no Vulkan GPU is reachable from this VM -- no speed number is measured, claimed or owed. 16 of 87 ops are native; the other 71 still run on the portable CPU tier. Quant, MoE, MLA and linear attention have no Vulkan kernels at all, and MLA is refused at the platform seam rather than mis-routed. The 0-declines figure is the part worth keeping: a provider can be selected and then decline INSIDE its kernel and forward to the CPU tier, so selections without declines is what separates "attention ran on Vulkan" from "a host fallback wearing its name". FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
localai-bot
changed the title
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…the MP4
Row `SERVE-VIDEOS-OAI`, branch `row/SERVE-VIDEOS-OAI`. An unmodified OpenAI
(Sora) client now works against our `/v1/videos`, ADDITIVELY over the
vLLM-Omni-derived fields we already take.
SPLIT, deliberately. The original change was 1241 non-exempt lines, over the
900 check-pr-size.py cap. It is split along its real seam rather than trimmed to
fit: this row is the request/response WIRE SHAPE (no generation code, no VAE, no
examples/server change), and REFERENCE CONDITIONING (`input_reference` -> fl2va
plus the two `metadata` ref2va modalities) is a stacked follow-up row. No cap and
no ratchet was raised.
Request aliases. `model`, `size` ("1280x720") and `seconds` land on the existing
native members. `seconds` is taken as a number OR a numeric string, because
OpenAI's schema types it as a string enum ("4"/"8"/"12") and a literal client
would otherwise be rejected on a type. PRECEDENCE is defined and gated: the
NATIVE field WINS (`width`/`height` over `size`, `duration` over `seconds`),
which is what guarantees every body that parsed before means exactly what it
meant before, applied PER-AXIS so an explicit `width` alone still lets `size`
supply the height. Both spellings are VALIDATED either way, so a malformed
`size` is a 400 even when explicit `width`/`height` override it.
`model` warns, never rejects: a Sora client cannot know the local model's name,
so refusing would defeat the compatibility and ignoring would hide a real
mismatch. The requested name and the divergence ride the job for its whole life.
GET /v1/videos/{id}/content returns the finished MP4. Without it a caller can
start and poll a job but never FETCH the result over HTTP. Unknown id -> 404;
queued/running -> 409 naming the status (a pending job must never answer with
bytes: a partially muxed file reaches the client as a valid-looking truncated
MP4); failed -> 500 with the failure; a vanished output -> 500, not a 200 with
zero bytes.
GATE (CPU, foreground): test_video_api 11/11 (125 assertions),
test_openai_api_server 40/40 (509), `server` builds clean. Additivity is gated
over a REAL socket: with no VideoRunner all four routes are absent (a bare 404,
no ErrorResponse envelope), with one they serve and the unknown-id 404 is ours.
TWO OUT-OF-ROW CI REPAIRS, carried here because they block EVERY merge, not just
this one. Both are `agent-record` job failures on main, both reproduced on an
unrelated PR, and neither is repaired by weakening a checker.
(1) check-fusion-consistency has failed since run 31129401136 because it flags
minimax_h3_video_vae_device.cpp. Repaired the way AGENTS.md names, with a
conscious allowlist entry carrying the verified reason: w1 already ships merged
and is already one MatmulBT, so nothing is unmerged; the seam is unusable because
the VAE block is f32 end to end where the method is kBF16, and every VAE Linear
carries a rank-1 bias the bias-free method has no slot for.
(2) check-role-discipline failed on every feature PR. CI checks out
refs/pull/N/merge, a SYNTHETIC merge GitHub builds whose entire message is
"Merge <head> into <base>": it names neither the row branch nor the PR, and it
NEVER lands on main, so a gate about MAIN's history was run on a commit that is
not main's history. Reproduced on PR #80 (Vulkan) to prove it is not this row's
doing. The fix reads the SECOND parent, which is the PR head: a merge of a branch
whose own commits name the row IS arrival through a row PR, one hop away. Gated
as a non-weakening: a new test asserts a merge naming no row ANYWHERE, and a
plain local `Merge branch 'wip'`, both STILL fail. Suite 40/40.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Opus 5 (1M context)
main moved 17 commits under this branch (#82, #83, #84 plus the public-doc gate restoration), so every earlier green result was re-run on the merged tree rather than carried over. Three record conflicts, all resolved BY KEY rather than by taking a side: * docs/STATUS.md — main had COMPACTED this paragraph and lowered the size ratchet to 284062. Taking our side wholesale would have silently reverted their compaction and blown the gate, so the resolution keeps THEIR paragraph and swaps in only our Vulkan clause, then trims it to fit. * docs/BENCHMARKS.md — a keyed table. Kept THEIR rows (they updated MiniMax-H3 and MXFP4 and added H3-RENDER-CLOSE) and appended only our Vulkan row. * .agents/state.md — append-only log, unioned theirs then ours. Re-verified on the merged tree: clean -Werror Vulkan-ON build 0 warnings, e2e opt-125m on Vulkan STRICT token-exact 6/6 prompts / 96/96 tokens with 0 provider declines, test_vulkan_backend 10/10 (480), test_backend_cross_device 11/11 (123), clean CPU-only build 0 warnings, CPU ctest 336/337 with the one failure (test_engine_core_proc) passing serially in 0.01s — the known starve-under-`-j` pattern on a box running several suites at once, not a regression.
mudler
force-pushed
the
row/BACKEND-VULKAN-A1
branch
from
August 7, 2026 00:38
1833103 to
461ce6b
Compare
localai-bot
marked this pull request as ready for review
August 7, 2026 00:40
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…the MP4
Row `SERVE-VIDEOS-OAI`, branch `row/SERVE-VIDEOS-OAI`. An unmodified OpenAI
(Sora) client now works against our `/v1/videos`, ADDITIVELY over the
vLLM-Omni-derived fields we already take.
SPLIT, deliberately. The original change was 1241 non-exempt lines, over the
900 check-pr-size.py cap. It is split along its real seam rather than trimmed to
fit: this row is the request/response WIRE SHAPE (no generation code, no VAE, no
examples/server change), and REFERENCE CONDITIONING (`input_reference` -> fl2va
plus the two `metadata` ref2va modalities) is a stacked follow-up row. No cap and
no ratchet was raised.
Request aliases. `model`, `size` ("1280x720") and `seconds` land on the existing
native members. `seconds` is taken as a number OR a numeric string, because
OpenAI's schema types it as a string enum ("4"/"8"/"12") and a literal client
would otherwise be rejected on a type. PRECEDENCE is defined and gated: the
NATIVE field WINS (`width`/`height` over `size`, `duration` over `seconds`),
which is what guarantees every body that parsed before means exactly what it
meant before, applied PER-AXIS so an explicit `width` alone still lets `size`
supply the height. Both spellings are VALIDATED either way, so a malformed
`size` is a 400 even when explicit `width`/`height` override it.
`model` warns, never rejects: a Sora client cannot know the local model's name,
so refusing would defeat the compatibility and ignoring would hide a real
mismatch. The requested name and the divergence ride the job for its whole life.
GET /v1/videos/{id}/content returns the finished MP4. Without it a caller can
start and poll a job but never FETCH the result over HTTP. Unknown id -> 404;
queued/running -> 409 naming the status (a pending job must never answer with
bytes: a partially muxed file reaches the client as a valid-looking truncated
MP4); failed -> 500 with the failure; a vanished output -> 500, not a 200 with
zero bytes.
GATE (CPU, foreground): test_video_api 11/11 (125 assertions),
test_openai_api_server 40/40 (509), `server` builds clean. Additivity is gated
over a REAL socket: with no VideoRunner all four routes are absent (a bare 404,
no ErrorResponse envelope), with one they serve and the unknown-id 404 is ours.
TWO OUT-OF-ROW CI REPAIRS, carried here because they block EVERY merge, not just
this one. Both are `agent-record` job failures on main, both reproduced on an
unrelated PR, and neither is repaired by weakening a checker.
(1) check-fusion-consistency has failed since run 31129401136 because it flags
minimax_h3_video_vae_device.cpp. Repaired the way AGENTS.md names, with a
conscious allowlist entry carrying the verified reason: w1 already ships merged
and is already one MatmulBT, so nothing is unmerged; the seam is unusable because
the VAE block is f32 end to end where the method is kBF16, and every VAE Linear
carries a rank-1 bias the bias-free method has no slot for.
(2) check-role-discipline failed on every feature PR. CI checks out
refs/pull/N/merge, a SYNTHETIC merge GitHub builds whose entire message is
"Merge <head> into <base>": it names neither the row branch nor the PR, and it
NEVER lands on main, so a gate about MAIN's history was run on a commit that is
not main's history. Reproduced on PR #80 (Vulkan) to prove it is not this row's
doing. The fix reads the SECOND parent, which is the PR head: a merge of a branch
whose own commits name the row IS arrival through a row PR, one hop away. Gated
as a non-weakening: a new test asserts a merge naming no row ANYWHERE, and a
plain local `Merge branch 'wip'`, both STILL fail. Suite 40/40.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Opus 5 (1M context)
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…the MP4
Row `SERVE-VIDEOS-OAI`, branch `row/SERVE-VIDEOS-OAI`. An unmodified OpenAI
(Sora) client now works against our `/v1/videos`, ADDITIVELY over the
vLLM-Omni-derived fields we already take.
SPLIT, deliberately. The original change was 1241 non-exempt lines, over the
900 check-pr-size.py cap. It is split along its real seam rather than trimmed to
fit: this row is the request/response WIRE SHAPE (no generation code, no VAE, no
examples/server change), and REFERENCE CONDITIONING (`input_reference` -> fl2va
plus the two `metadata` ref2va modalities) is a stacked follow-up row. No cap and
no ratchet was raised.
Request aliases. `model`, `size` ("1280x720") and `seconds` land on the existing
native members. `seconds` is taken as a number OR a numeric string, because
OpenAI's schema types it as a string enum ("4"/"8"/"12") and a literal client
would otherwise be rejected on a type. PRECEDENCE is defined and gated: the
NATIVE field WINS (`width`/`height` over `size`, `duration` over `seconds`),
which is what guarantees every body that parsed before means exactly what it
meant before, applied PER-AXIS so an explicit `width` alone still lets `size`
supply the height. Both spellings are VALIDATED either way, so a malformed
`size` is a 400 even when explicit `width`/`height` override it.
`model` warns, never rejects: a Sora client cannot know the local model's name,
so refusing would defeat the compatibility and ignoring would hide a real
mismatch. The requested name and the divergence ride the job for its whole life.
GET /v1/videos/{id}/content returns the finished MP4. Without it a caller can
start and poll a job but never FETCH the result over HTTP. Unknown id -> 404;
queued/running -> 409 naming the status (a pending job must never answer with
bytes: a partially muxed file reaches the client as a valid-looking truncated
MP4); failed -> 500 with the failure; a vanished output -> 500, not a 200 with
zero bytes.
GATE (CPU, foreground): test_video_api 11/11 (125 assertions),
test_openai_api_server 40/40 (509), `server` builds clean. Additivity is gated
over a REAL socket: with no VideoRunner all four routes are absent (a bare 404,
no ErrorResponse envelope), with one they serve and the unknown-id 404 is ours.
TWO OUT-OF-ROW CI REPAIRS, carried here because they block EVERY merge, not just
this one. Both are `agent-record` job failures on main, both reproduced on an
unrelated PR, and neither is repaired by weakening a checker.
(1) check-fusion-consistency has failed since run 31129401136 because it flags
minimax_h3_video_vae_device.cpp. Repaired the way AGENTS.md names, with a
conscious allowlist entry carrying the verified reason: w1 already ships merged
and is already one MatmulBT, so nothing is unmerged; the seam is unusable because
the VAE block is f32 end to end where the method is kBF16, and every VAE Linear
carries a rank-1 bias the bias-free method has no slot for.
(2) check-role-discipline failed on every feature PR. CI checks out
refs/pull/N/merge, a SYNTHETIC merge GitHub builds whose entire message is
"Merge <head> into <base>": it names neither the row branch nor the PR, and it
NEVER lands on main, so a gate about MAIN's history was run on a commit that is
not main's history. Reproduced on PR #80 (Vulkan) to prove it is not this row's
doing. The fix reads the SECOND parent, which is the PR head: a merge of a branch
whose own commits name the row IS arrival through a row PR, one hop away. Gated
as a non-weakening: a new test asserts a merge naming no row ANYWHERE, and a
plain local `Merge branch 'wip'`, both STILL fail. Suite 40/40.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Opus 5 (1M context)
localai-bot
added a commit
that referenced
this pull request
Aug 7, 2026
…the MP4 (#71) Row `SERVE-VIDEOS-OAI`, branch `row/SERVE-VIDEOS-OAI`. An unmodified OpenAI (Sora) client now works against our `/v1/videos`, ADDITIVELY over the vLLM-Omni-derived fields we already take. SPLIT, deliberately. The original change was 1241 non-exempt lines, over the 900 check-pr-size.py cap. It is split along its real seam rather than trimmed to fit: this row is the request/response WIRE SHAPE (no generation code, no VAE, no examples/server change), and REFERENCE CONDITIONING (`input_reference` -> fl2va plus the two `metadata` ref2va modalities) is a stacked follow-up row. No cap and no ratchet was raised. Request aliases. `model`, `size` ("1280x720") and `seconds` land on the existing native members. `seconds` is taken as a number OR a numeric string, because OpenAI's schema types it as a string enum ("4"/"8"/"12") and a literal client would otherwise be rejected on a type. PRECEDENCE is defined and gated: the NATIVE field WINS (`width`/`height` over `size`, `duration` over `seconds`), which is what guarantees every body that parsed before means exactly what it meant before, applied PER-AXIS so an explicit `width` alone still lets `size` supply the height. Both spellings are VALIDATED either way, so a malformed `size` is a 400 even when explicit `width`/`height` override it. `model` warns, never rejects: a Sora client cannot know the local model's name, so refusing would defeat the compatibility and ignoring would hide a real mismatch. The requested name and the divergence ride the job for its whole life. GET /v1/videos/{id}/content returns the finished MP4. Without it a caller can start and poll a job but never FETCH the result over HTTP. Unknown id -> 404; queued/running -> 409 naming the status (a pending job must never answer with bytes: a partially muxed file reaches the client as a valid-looking truncated MP4); failed -> 500 with the failure; a vanished output -> 500, not a 200 with zero bytes. GATE (CPU, foreground): test_video_api 11/11 (125 assertions), test_openai_api_server 40/40 (509), `server` builds clean. Additivity is gated over a REAL socket: with no VideoRunner all four routes are absent (a bare 404, no ErrorResponse envelope), with one they serve and the unknown-id 404 is ours. TWO OUT-OF-ROW CI REPAIRS, carried here because they block EVERY merge, not just this one. Both are `agent-record` job failures on main, both reproduced on an unrelated PR, and neither is repaired by weakening a checker. (1) check-fusion-consistency has failed since run 31129401136 because it flags minimax_h3_video_vae_device.cpp. Repaired the way AGENTS.md names, with a conscious allowlist entry carrying the verified reason: w1 already ships merged and is already one MatmulBT, so nothing is unmerged; the seam is unusable because the VAE block is f32 end to end where the method is kBF16, and every VAE Linear carries a rank-1 bias the bias-free method has no slot for. (2) check-role-discipline failed on every feature PR. CI checks out refs/pull/N/merge, a SYNTHETIC merge GitHub builds whose entire message is "Merge <head> into <base>": it names neither the row branch nor the PR, and it NEVER lands on main, so a gate about MAIN's history was run on a commit that is not main's history. Reproduced on PR #80 (Vulkan) to prove it is not this row's doing. The fix reads the SECOND parent, which is the PR head: a merge of a branch whose own commits name the row IS arrival through a row PR, one hop away. Gated as a non-weakening: a new test asserts a merge naming no row ANYWHERE, and a plain local `Merge branch 'wip'`, both STILL fail. Suite 40/40. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Opus 5 (1M context) Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
jefby
pushed a commit
to jefby/vllm.cpp
that referenced
this pull request
Aug 7, 2026
…was vacuous Self-reported violation and its prevention. PR mudler#80 (Vulkan, merged 5397e91) landed EIGHT commits that touched src/ and tests/ without updating docs/STATUS.md and docs/BENCHMARKS.md in the same commit, reddening documentation-checkpoint for that push range: 9579f94, ba5ea0c, 196ea46, e32c5ed, 3bfa1f1, 34a3efe, 2c86f79, f4738bb. Each deferred its doc update into a following record(...) commit. WHY IT PASSED LOCALLY, which is the part worth fixing. check-doc-checkpoint.py --staged inspects the STAGED paths, so it is VACUOUS when nothing is staged -- and nothing is staged after `git commit`, which is exactly when agent-preflight.sh runs it. The gate reported OK on every one of those commits while checking literally nothing. CI is diff-scoped over the pushed range and checks each commit independently, so the failure only surfaced on main, where a diff-scoped range can never be re-covered by a later run. Preflight now runs --base origin/main --head HEAD whenever the branch is ahead. That check is deliberately OUTSIDE the --staged block: my first attempt nested it inside, which reproduces the identical hole one level up, since --staged is precisely the flag you are not passing when the range is unchecked. Verified two ways -- it prints "ok doc-checkpoint range" here, and pointed at the mudler#80 range it raises 20 errors naming every offending commit. workflow.md carries the same instruction with the reason. NOT REPAIRED BY REWRITING HISTORY: main had already moved (mudler#86 landed on top) and other sessions branch from it, so force-pushing to regroup eight commits' files would cost more than the defect. Substance was never wrong -- STATUS, BENCHMARKS and FEATURES on main all describe the shipped state with the llvmpipe-only and no-speed-number caveats intact. What was violated is the per-COMMIT granularity that keeps a bisect landing on a commit whose docs match its code. Rule restated where it is enforced: a feature commit carries its OWN STATUS/BENCHMARKS update; when the numbers are not yet known the honest line is pending/void with the reason, which is what the gate's own message asks for. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A model runs end to end on Vulkan, token-exact against the vLLM oracle.
This is the strong form of the claim:
CurrentPlatform(); no test-side override.kPagedAttentionselections with zero declines is what separates "attention ran on Vulkan" from "a host fallback wearing its name".Measured on llvmpipe, the software rasterizer — no Vulkan GPU is reachable from this VM (it's a KVM guest; the AMD here is the CPU). This is a correctness proof, not a performance one. No speed number is measured, claimed or owed.
What this PR contains
Implements
VK-A1in full plus the first bricks ofVK-Bfrom.agents/specs/vulkan-full-support.md.Infrastructure
16.4.0ships no release assets, so it could never have backed a CI gate; committed SPIR-V reproduces byte-for-byte under 16.5.0, proving both provenance and stability across a minor bump.vulkan-spirv-freshness— mutation-proved red-then-green.build-test-vulkan— GPU-free on llvmpipe.VLLM_CPP_VULKAN=ONpreviously appeared nowhere inci.yml, which is why the Vulkan suite had rotted RED since accelerator-seam rowS5.#define, so every dtype × quant × coopmat combination is a separate SPIR-V module (242string_to_spvcall sites at pin237ad9b96). One specialized module serves the whole axis instead —vt_matmulalone covers 54 variants, and its module got smaller because the dead branches fold away..cpp: 3,487 → 55 lines.Kernels (8 → 16 native): dense GEMM both orientations, block-paged attention, KV-cache write, QKV split, rotary apply, embedding, greedy argmax.
Platform seam:
FLASH_ATTNregistered forkVULKANon exactly Metal's footing. MLA still returns EMPTY deliberately —kMlaDecodeAttention/kMlaPrefillAttention/kConcatAndCacheMlahave no Vulkan kernel, and naming a backend there would route an MLA model into one that cannot serve it.Measured findings worth reading
The baseline in the spec was wrong.
vt::GetOpdoes not throw for unimplemented Vulkan ops and hasn't sinceS5gave unified-memory devices a portable CPU reference tier. Re-counted at runtime: of 87 CPU-registered ops, 16 are native on Vulkan, 71 are served by the tier, 0 throw. So the campaign's real content is moving ops off the host — a performance project, not a make-it-run one.vt::GetReferenceTierHits()must reach 0.The workgroup size cannot be a specialization constant at
vulkan1.1.local_size_x_idmakes glslang emitExecutionMode LocalSize 1 1 1plus the legacyBuiltIn WorkgroupSizevector —LocalSizeIdneeds SPIR-V 1.2 +VK_KHR_maintenance4(core in Vulkan 1.3). Measured: every workgroup ran one thread against aceil(n/128)dispatch, NMSE1e-14→ 0.99 onkAdd. An A/B isolated it to launch geometry, not specialization. Recorded beside the#defineso nobody retries it.RoPE is split the way vLLM splits it.
RotaryEmbeddingbuildscos_sin_cacheonce in__init__and the forward only applies it (rotary_embedding/base.py:160-252). The table build stays on the portable tier — where the double-precisionpow/cos/sinlives — and the per-token apply is native. Faithfulness and numerics agreed: an f32 transcription of the angle would be wrong at long context and would still pass a short-sequence gate.Paged attention has no Vulkan port source anywhere. Zero
block_table/pagedmatches across all 132.comp+ 26.glslat the pin. The online-softmax skeleton followsflash_attn.comp; the block-table indirection and windowing come from our CPU kernel. It cannot mirror the CPU's three passes — that needs aprobsarray of one float per key in the window, unbounded at long context — so it sits in the NMSE tier and claims no bit-exactness. Deliberate tier choice, not a widened tolerance.Per-call refusals, not regressions. fp8 KV cache and mrope forward through
GetOpFallbackrather than throwing — throwing would remove a capability the reference tier already provides.Gates
Clean
-Werrorbuild with Vulkan ON: 0 warnings.test_opt_paged_engine(e2e, Vulkan)test_vulkan_backendtest_backend_cross_devicetest_opt_loadgen-vulkan-spirv.py --checkCPU-only build (
AUTO=OFF) rebuilt clean and its full ctest re-run, sincev1/attention/backend.cppis shared.Pre-existing red, not from this branch:
check-fusion-consistencyfails onminimax_h3_video_vae_deviceGEMM merge drift, verified at075b9f21^.Still not native
Quant, MoE, MLA, linear attention (GDN/KDA), the rotary table build, and every sampler beyond greedy argmax — 71 ops on the portable tier. Those are
VK-CthroughVK-H.🤖 Generated with Claude Code