VK-C: cooperative-matrix GEMM tactic, verified executing on GB10

[localai-bot]

VK-C: the cooperative-matrix (tensor-core) GEMM tactic, verified executing on GB10.

NVIDIA GB10:  coopmat bf16xbf16->f32 16x16x16 SUBGROUP: YES, subgroup size 32
              bf16 GEMM tactic: COOPMAT
              bf16 GEMM NMSE vs the f32 oracle: 0.000000
              test_vulkan_backend 12/12, 491 assertions
llvmpipe:     coopmat: no  ->  tactic: scalar, coopmat module NEVER BUILT

The gate that almost wasn't

The pre-existing cross-device GEMM case uses f32 operands. The tactic requires bf16. So it passed on GB10 while exercising the coopmat path not at all — green over an unexercised path, which reads exactly like success.

Numbers can never separate a working tactic from a silent fallback here, because the scalar kernel is equally correct. So VulkanContext::PipelineExistsFor(module) is added, and the load-bearing assertion is the tactic, not the NMSE:

• coopmat where the device reports the configuration,
• scalar where it does not,
• and on llvmpipe the coopmat module asserted never built — selecting it there would fail at pipeline creation.

Same failure shape the op-provider decline counters exist for.

NMSE 0.0 is expected, not suspicious. The inputs are exactly representable in bf16, so the products are exact and f32 accumulation of 32 terms has nothing to round — which isolates the tile ordering as the thing under test rather than input narrowing. M=20, N=12 are ragged on purpose so the bounds-checked store runs.

Shape dictated by hardware, not chosen

Probed on GB10 first: 11 configurations, all SUBGROUP scope — f16→f16 and f16→f32 at 16x16x16 / 16x8x16 / 16x8x8, u8/s8→u32/s32 at 16x16x32 / 16x8x32, and 16x16x16 bf16/bf16/f32/f32, which is our model dtype with our f32-accumulate contract. Vulkan matches configurations exactly — there is no nearest-fit — so the shader is written to that one tuple and the host predicate asks for it and nothing else.

Structure ported from llama.cpp mul_mm_cm2.comp @ 237ad9b96; per-element semantics from our own cpu_ops.cpp MatmulChunked, the oracle for both tactics.

Selection: four hard requirements

Each failure falls back to the always-correct scalar kernel.

requirement	why it is not a heuristic
exact configuration reported	Vulkan has no nearest-fit match
subgroup size 32	the shader's workgroup is a literal 32
both operands bf16	every reported configuration takes bf16/f16/int8 — f32 can never use this path
K % 16 == 0	a ragged K tail cannot be masked inside a coopmat load; truncating silently drops dot-product terms

Ragged M and N are fine — the store is bounds-checked.

Two traps avoided because they were already on record

local_size_x_id is unusable at vulkan1.1 — it emits ExecutionMode LocalSize 1 1 1 and silently runs one thread per workgroup (measured earlier in this campaign). The workgroup size is a literal 32, so the SPIR-V is compiled for subgroup 32 and the host refuses the tactic on any device reporting otherwise.

The result is always spilled through shared memory. A direct coopMatStore would have to target the uint32 view this backend binds — storing a coopmat<float> through a uint pointer — which glslang accepts without complaint, and no CI-reachable device can execute a coopmat shader, so it would never have surfaced. A direct-store fast path is a legitimate later refinement, but it is a speed change and belongs behind a hardware A/B.

Conditional extension enablement

Naming an unsupported extension in VkDeviceCreateInfo makes vkCreateDevice fail outright, so an unconditional request would take the entire Vulkan backend down on llvmpipe. Enablement is gated on the probe, and the loader gains an optional entry-point tier — its existing X-macro VT_CHECKs every name, which is right for core functions and fatal for extension ones.

Split for reviewability

check-pr-size flagged the original 969 lines against a 900 cap, so this PR was split at a natural seam: this one is the tactic, and the A/B harness plus its measured numbers follow in a separate PR. That keeps each reviewable on its own terms rather than trimming arbitrarily.

The speed result exists and is good — 11.1x-32.9x on NVIDIA Thor — but it belongs with the harness that produced it.

What is NOT claimed

No speed number. The tactic is correct and it engages; whether it beats the scalar kernel is unmeasured, and that A/B on GB10 is what VK-C still owes.

Numerics also do not match the scalar tactic bit-for-bit and do not claim to — the scalar kernel walks K sequentially per output element to match the CPU's order; a cooperative matrix accumulates a tile at a time in a hardware-defined one. Both sit in the NMSE tier.

Gates

Clean -Werror Vulkan-ON builds on both boxes, 0 warnings. Local (llvmpipe): test_vulkan_backend 12/12, test_backend_cross_device 11/11, generator 14/14. dgx (GB10): test_vulkan_backend 12/12 (491), test_backend_cross_device 11/11.

Pre-existing red, not from this branch: check-fusion-consistency on minimax_h3_video_vae_device, failing on main since before this work.

🤖 Generated with Claude Code