perf(vulkan): barrier only on a PROVEN dependency, and the llama.cpp gate is already MET

.agents/NOW.md

@@ -26,7 +26,7 @@ Work: exact-chunks on main `1ce0d662b`; sm_120 measured at `3d2581551`.
 | CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn**. **G5 not next** | Parakeet encoder; attn dtype hoist |
 | Supported-models list | **LANDED**: FEATURES arch table CI-bound (33 archs) | — |
 | `/v1/videos` OpenAI shape | **MERGED** (#71): Sora `model`/`size`/`seconds` + `GET /{id}/content` | `row/SERVE-VIDEOS-REFS` PR open: reference conditioning |
-| Vulkan 27B decode | **4.285 vs 4.35**; host ~3 ms/tok, pipeline -1.41 ms (5/5), spin rejected | NOT met: 900 barriers/tok |
+| Vulkan 27B decode | **MET: 4.36 vs llama.cpp 4.35** (7 legs, main). Barriers -19.8%/tok, GPU -1.09 ms, e2e 8/12, OFF | Re-measure before flipping `VT_VULKAN_SMART_BARRIERS` |
 | `BACKEND-ROCM` | **(b) fix in; #140 gfx1201 hipBLAS + Gemma-4 MoE landed (contributor, authorship-preserved); W0 green 4 archs** | compile + M2 ([spec](specs/rocm-unified-memory-b.md)) |
 | TP spike #287 (PR #143) | **TP-W1 LANDED**: rank-group table + TP handle (6/6); DSR leak FIXED (unblocks #127/#154/#155) | TP-W2 (linears + loader) |
 | Release | **ACTIVE; W5 19/19+10/10; contract 30/30** | #141; artifacts pending |

.agents/coordination.md

@@ -118,6 +118,22 @@ without the selected contention proof for their entire run are discarded.
 
 ## Active claims
 
+**Vulkan dependency-driven barriers (`BACKEND-VULKAN`, 2026-08-09, `CLAIM-VULKAN-FULL-1`).** Claude Code (opus-5), isolated worktree
+`/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-aab93052e5a3c7025`, branch
+`row/BACKEND-VULKAN-BARRIERS`, base `origin/main` `c1716fd0` (rebased forward;
+the commits main gained carry NO `src/`, `examples/` or gate-test change, so the
+GB10 evidence stands). NOT PUSHED, no PR, by instruction. Scope: the batched
+dispatch path's barrier policy in `src/vt/vulkan/vulkan_context.{h,cpp}`, the
+SPIR-V binding-access reflection in `scripts/gen-vulkan-spirv.py` and its two
+generated artifacts, two new cases in `tests/vt/test_vulkan_backend.cpp`, and one
+`docs/ENVIRONMENT.md` row. NO kernel, shader, model, scheduler or op-table change.
+The row's headline result is that the llama.cpp Vulkan decode gate is ALREADY MET
+on current main (4.36 vs 4.35, 7 clean legs) independently of the lever;
+`VT_VULKAN_SMART_BARRIERS` is implemented, GB10-gated in 12 combinations, and
+ships DEFAULT-OFF pending an operator re-measurement. dgx build/gate roots
+`dgx.casa:~/vkbase` (main control) and `~/vkbar` (this branch), both
+`-DVLLM_CPP_VULKAN=ON`, every GPU stage under `flock $HOME/gpu.lock`.
+
 **Server binary release manifest W5 (`ENG-RELEASE-BINARIES`, 2026-08-08,
 `CLAIM-ENG-RELEASE-BINARIES-W5`).** Codex (GPT-5), isolated worktree
 `/home/mudler/_git/vllm.cpp-release-binaries`, branch

.agents/state.md

@@ -43725,3 +43725,92 @@ ggml-vulkan.cpp:3193). That is the largest identified residual and the next row.
 FIRST RESUME COMMAND:
     git log --oneline -1   # expect 66275b1f on row/BACKEND-VULKAN-HOSTDISPATCH
     ssh dgx.casa 'grep -E "^VLLM_CPP_VULKAN" ~/vk-hostdispatch/build-vk/CMakeCache.txt'  # must be ON
+
+## 2026-08-09 — BACKEND-VULKAN-BARRIERS: llama.cpp goal already MET on main (4.36 vs 4.35); the barrier lever is real (19.8% skipped, GPU -1.09 ms/tok) but e2e-unresolved at 12 pairs, ships default-OFF
+<!-- state: 2026-08-09T15:00 -->
+
+HEAD `row/BACKEND-VULKAN-BARRIERS`, rebased onto `origin/main`. NOT PUSHED, no
+PR, by instruction. Built and gated on GB10 from `c1716fd0`; the two commits main
+gained afterwards (`dc2139b3` docs/CI/release-tooling, `7534da65` a C-ABI symbol
+hoist in `src/capi/vllm_c.cpp`) leave `src/vt/`, `examples/` and the gate tests
+BYTE-IDENTICAL, verified by `git diff --stat`, so the measurements below stand on
+the rebased tree.
+
+THE GOAL WAS ALREADY MET BEFORE THIS ROW, AND NOBODY KNEW. No absolute tok/s
+number existed for main after `#191`. Eight legs of `c1716fd0` (drop_caches per
+leg, `flock`, 27B bf16 32-in/32-out c1) give a clean-leg median of 229.54 ms =
+**4.36 tok/s** against llama.cpp Vulkan's 4.35 on the same box and weights. Seven
+clean legs spread 1.59 ms (0.69%); the eighth (242.84) is discarded for a NAMED
+cause, a compile job this session started overlapping it. The 4.285 of record
+predates `#191`.
+
+THE BARRIER LEVER IS IMPLEMENTED, GATED AND MEASURED, AND SHIPS DEFAULT-OFF.
+`VT_VULKAN_SMART_BARRIERS=1` records a barrier only on a true RAW/WAW/WAR hazard
+against everything recorded since the previous barrier. The read and write sets
+are NOT plumbed from the op layer and NOT guessed from binding index: they are
+reflected out of the committed SPIR-V's `NonWritable` decorations, which glslang
+enforces (writing a `readonly buffer` is a compile error), and `GetPipeline` now
+hard-checks the module's declared binding count against what the host binds.
+MEASURED per decode token, two-length GPU-timestamp diff: barriers 900.0 -> 722.0
+(178 skipped, 19.8%), GPU span 231.38 -> 230.29 ms. e2e, 12 order-alternated AB/BA
+pairs across two regimes (the box rebooted mid-campaign and came back ~10 ms
+faster): 8 to smart, mean paired delta -0.94 ms, sign test p = 0.19, i.e. NOT
+established by the e2e instrument alone. The pre-reboot 8-pair block alone is
+6/8 and median 230.48 -> 229.00 ms.
+
+WHAT THE MEASUREMENT CORRECTS. The handoff named "~2.3 ms of GPU-timeline time
+OUTSIDE any timestamped dispatch" as this lever's size. Direct instrumentation
+(new `gpu_span_ms()`: per-command-buffer first-TOP-of-pipe to last-BOTTOM) reads
+**0.56 ms/token** outside the dispatch intervals, and that number does NOT move
+when 178 barriers per token are removed. `TOP_OF_PIPE` is timestamped when the
+command reaches the top of the pipe, which a preceding barrier does not hold
+back, so the drain is billed INSIDE the following dispatch's interval.
+Span-minus-sum understates barrier cost; the SPAN is the honest probe, and it is
+the only one that stays meaningful once dispatches may overlap.
+
+WHAT THE UNIT GATE CATCHES THAT VALUES DO NOT, MEASURED BY MUTATION ON GB10.
+Forcing the hazard predicate to always report "independent" fails
+`test_vulkan_backend` 32/35 with 330 VALUE failures (`0 == -150`, a dispatch
+reading memory before its producer wrote it) -- the unconditional barrier is
+load-bearing and this is the proof. Removing ONLY the write-after-read half fails
+34/35 with exactly TWO assertions, both of them the new structural barrier
+counters; every value assertion in the file still passes. A partial
+implementation that handles RAW and WAW and forgets WAR computes correct numbers
+on this hardware today. Both mutants were built IN PLACE: a `cp -a` of the build
+directory silently rebuilds the ORIGINAL sources (CMake caches absolute source
+paths) and produced a byte-identical binary and a false green before this was
+noticed; the tree was restored with source and binary md5s verified.
+
+CORRECTNESS RUN BEFORE ANY TIMING, ALL ON GB10, 12 COMBINATIONS (both barrier
+arms x `VT_VULKAN_INFLIGHT` 1 and 2): `test_vulkan_backend` 35/35 (2647/2650
+assertions), `test_backend_cross_device` 11/11 (132), `VLLM_CPP_DEVICE=vulkan
+test_opt_paged_engine` 6/6 prompts token-exact (96/96 tokens), 0 declines. A
+256-token decode gives BYTE-IDENTICAL token ids across all four
+{arm} x {INFLIGHT} combinations, md5 `02f7606ec96c9a21775210c2ab33a2c5`.
+
+WHY THE DEFAULT STAYS OFF. The e2e effect is ~0.6% and the e2e instrument alone
+does not resolve it past this box's session drift (the always-barrier arm medians
+230.48 ms in the AB block and 229.54 ms in the standalone main block, on
+behaviourally identical code); the goal is already met without it; and this is
+the same hazard class as the fence spin that was measured FASTER and computed
+garbage on GB10 two days ago. It is one environment variable away with the
+evidence attached.
+
+PROHIBITIONS: never two 27B processes on dgx at once; take `flock $HOME/gpu.lock`;
+`local-ai-worker` stopped before GPU work and restored to `--restart=always`
+after (restored at the end of this session).
+
+BLOCKER: the box OOM-REBOOTED partway through an attempt to extend the AB/BA
+block to 16 pairs (`NVRM ... NV_ERR_NO_MEMORY`, boot -1 ended 12:04:29). 17
+back-to-back legs each preceded by `drop_caches` refill 50 GB from cold on a
+119 GB unified box. The 8-pair block stands; a longer block needs either fewer
+cold reloads or a smaller model.
+
+NEXT LEVER: with the barrier gap measured at 0.56 ms/token and unmoved, the
+remaining decode time is 84% `vt_matmul_vec` at ~248 GB/s and 8.3%
+`vt_matmul_coopmat`. The named residuals are the lm_head shape's unexplained
+7% shortfall against its own measured streaming ceiling, and prefill.
+
+FIRST RESUME COMMAND:
+    git log --oneline -1   # row/BACKEND-VULKAN-BARRIERS
+    ssh dgx.casa 'grep -E "^VLLM_CPP_VULKAN" ~/vkbar/build-vk/CMakeCache.txt'  # must be ON

docs/BENCHMARKS.md

@@ -363,7 +363,7 @@ built on it rather than keeping the flattering one.
 | Memory footprint vs declared workload (`ROAD-V1-MEM`, #83) | **Never measured, and not measurable today**: there is no auto-sizing to compare against, because the KV pool is a hand-typed `--num-blocks`, so "what the run actually needed" has no number | Once M1's `MemoryBudget` lands: predicted-vs-actual bytes per allocation class, then peak footprint ours-auto vs vLLM at its 0.9 default on the same model and config |
 | Startup latency (cold to first `/health`) | **36.51 s vs vLLM 0.25.0's 221.51 s = 6.07x** (medians of 3, 27B-NVFP4, GB10). PROVISIONAL: 3 of 6 legs contended, repeat killed by a host reboot. [Detail](../.agents/benchmark-record.md) | Uncontended 3-rep re-run on a quiet box |
 | Speculation depth (`ROAD-V1-D3-SPEC-K`, #81) | **Never measured, MTP is k=1** (our port covers vLLM's k=1 branch only), so no acceptance-vs-depth curve exists | k=2..4 three-way greedy gate, then the c1/c>1 A/B + the per-workload (prose vs code) acceptance-vs-depth curve any dynamic or adaptive depth policy needs |
-| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | 25 NATIVE (+8 GDN). **27B prefill 21.5x**, decode **4.285**/4.35. Host MEASURED ~3 ms/tok not 5.3; pipelined submit **-1.41 ms/tok 5/5**; fence spin REJECTED. [source](../benchmarks/demo/vulkan_27b_llamacpp.json) | `VK-C` coopmat A/B on Thor (`VT_VULKAN_COOPMAT=0` A/Bs it): **11.1x-32.9x** vs our UNTILED scalar kernel, not vs a competent GEMM. `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, same GGUF, three columns |
+| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | 25 NATIVE (+8 GDN). **27B prefill 21.5x**; decode **4.36 vs 4.35, MET** (7 clean legs). Smart barriers skip 19.8%/tok, GPU -1.09 ms; e2e 8/12, unresolved. OFF. [source](../benchmarks/demo/vulkan_27b_llamacpp.json) | `VK-C` coopmat A/B on Thor (`VT_VULKAN_COOPMAT=0` A/Bs it): **11.1x-32.9x** vs our UNTILED scalar kernel, not vs a competent GEMM. `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, same GGUF, three columns |
 | ROCm (`BACKEND-GATE-ROCM-VLLM` / `-SGLANG`) | **NOT APPLICABLE: no number measured, claimed or owed.** W0 ctest-green on 4 gfx archs (#41); gfx1201 hipBLAS + Gemma-4 MoE (#140, contributor) ran M0/M1 on 2× R9700, our side CPU-link-verified only. No AMD HW here | The approach-(b) fix (PENDING community) unblocks the first APU model run (M2); the gate becomes a same-box vLLM-ROCm oracle once a model runs ([#41](https://github.com/mudler/vllm.cpp/issues/41)); floor: vLLM |
 | SGLang floor arms | Never ran | Both arms of the SGLang comparison |
 | Embeddings on the ONE surface (ROW 6, `LlamaModel` + `vllm_embed` + `/v1/embeddings`) | **NO number measured, claimed or owed.** Correctness-gated only, CPU: the 2026-08-08 fold (engine path == direct registry path, f64 LAST+normalize reference on the committed fixture) is plumbing, no speed claim | A REAL embedding checkpoint (e5-mistral class) + a same-box `vllm.LLM(task="embed")` oracle; only then does an embed-throughput bar exist |

docs/ENVIRONMENT.md

@@ -114,6 +114,7 @@ portable/reference path. In normal operation leave them unset.
 | `VT_VULKAN_INFLIGHT` | 2 | How many command-buffer submissions may be in flight at once. At 1 a flush submits and BLOCKS on its fence, so all host recording is serialized against an idle GPU; at 2 the flush returns and the next token records while the previous one runs, worth a MEASURED -1.41 ms/token on 27B (5/5 paired legs). Each slot owns a disjoint descriptor-ring slice, so raising this raises descriptor memory |
 | `VT_VULKAN_HOST_PROFILE` | off | `=1` prints a per-phase HOST time breakdown at exit (submit, record, descriptor update, bookkeeping, pipeline lookup, and time blocked on the GPU). Diagnostic companion to `VT_VULKAN_DISPATCH_STATS`, which covers the DEVICE side; use it to separate host cost from GPU wait |
 | `VT_VULKAN_BATCH` | on | `=0` forces the per-dispatch submit-and-wait path instead of recording many dispatches into ONE command buffer (with a pipeline barrier between them) and submitting once. MEASURED on GB10: decode **2.62x**, faster in **8 of 8** interleaved pairs (24.8 -> 64.5 tok/s), 40-46 dispatches per submit, opt-125m still 6/6 token-exact. Batching is sound only because all three host-read paths drain the batch first: `Backend::Copy` and `Memset` (host memcpy over the mapped allocation), `Synchronize`, and `Backend::FlushPending`, which op_provider.cpp calls before running a PORTABLE REFERENCE TIER kernel directly over device memory. Without that third one a host kernel would read stale bytes silently. It exists for the same-binary A/B and as the bisect lever. Vulkan-only |
+| `VT_VULKAN_SMART_BARRIERS` | off | `=1` records a `vkCmdPipelineBarrier` before a batched dispatch ONLY where a real dependency exists, instead of before every one. The batched path otherwise emits ~900 full COMPUTE->COMPUTE memory barriers per 27B decode token; llama.cpp instead calls `ggml_vk_sync_buffers` at 42 explicit call sites and tracks per-scratch-buffer `need_sync` flags (`ggml-vulkan.cpp:3193`, `:8687-8748` @ pin `237ad9b96`). This is that idea made general: each dispatch's READ set and WRITE set come from the committed SPIR-V's `NonWritable` decorations (`vulkan_spirv.h` § `writable_mask`), which glslang enforces, and a barrier is skipped only when none of read-after-write, write-after-write or write-after-read holds against everything recorded since the previous barrier. It DEFAULTS TO EMITTING: buffers are compared whole rather than by byte range, an operand that is both read and written counts as a write, and an unrecognised binding counts as writable, so every uncertainty costs a barrier rather than correctness. Both arms live in one binary because a cross-BUILD comparison of two barrier policies has already produced a false 1.2x in this campaign -- and because a policy that drops a real dependency computes WRONG NUMBERS, which is invisible on llvmpipe and was invisible for a whole benchmark block the last time it happened here. Vulkan-only |
 | `VT_VULKAN_GEMV` | on | `=0` forces the decode GEMV onto the portable scalar GEMM instead of `vt_matmul_vec`. The vec tactic gives each output element a WORKGROUP whose lanes stride K, so adjacent lanes read adjacent addresses; the scalar kernel puts one invocation on each output element and loops K there, which for the `MatmulBT` weight layout leaves adjacent lanes `K*2` bytes apart, each pulling its own cache line for 2 bytes of it. Selected only for `MatmulBT` at M=1 with K at least one workgroup wide -- the other orientation is already coalesced and the vec shape would make it worse. MEASURED on GB10: GEMM `ms/call` improves in 7 of 8 interleaved pairs, median 1.8x. Its K reduction is a tree, so unlike the scalar kernel it does NOT share the CPU's accumulation order; the opt-125m STRICT gate (6/6 token-exact) is what clears it. Vulkan-only |
 | `VT_VULKAN_RMSNORM` | auto | Which `vt::RmsNorm` SPIR-V module runs: `wide` forces the 1024-invocation subgroup-reducing one, `base` forces the portable 128-invocation one, unset lets the device capability decide (1024 invocations on the X axis plus compute subgroup BASIC and ARITHMETIC). The wide module exists because `RmsNorm` dispatches ONE WORKGROUP PER ROW and a batch-1 decode step has exactly one row: on Qwen3.6-27B that put 128 invocations on a 5120-wide row, four warps of one SM, with the rest of the GPU idle. MEASURED on GB10 by the two-length GPU-timestamp diff: **0.0611 -> 0.0123 ms/call, 7.88 -> 1.59 ms/token**, and paired decode **241.9 -> 235.6 ms** median TPOT. The tell that it was OCCUPANCY and not the reduction is that the SAME shader costs 0.066 ms/call during PREFILL, where 32 rows give it 32 workgroups and 32x the data. It exists for the same-binary A/B and so the unit gate can exercise the fallback on hardware that would always pick the wide arm. Vulkan-only |
 | `VT_VULKAN_MATMUL_NCOLS` | 4 | Output columns each lane of the portable scalar GEMM computes, in the `[K,N]` (non-transposed) orientation only. At 1 the kernel is the flat one-invocation-per-output-element body; above 1 a workgroup takes `128*NCOLS` CONSECUTIVE output columns of one row, so at each step of K it reads a contiguous run of that many elements instead of 128. This is the ONE decode GEMM that cannot reach the `vt_matmul_vec` tactic, because in `[K,N]` the lanes are already coalesced and the GEMV shape would make them strided; on the 27B it is the lm_head, `m=1 k=5120 n=248320`, 2.54 GB moved per token. MEASURED on GB10, 27B decode, `ms/call` medians over interleaved replicates: NCOLS 1 = 12.48, 2 = 12.46, **4 = 11.54**, 8 = 12.81, with 4 winning **6 of 6** interleaved pairs against 1. Blocking is a TRADE, not a monotone win: at 8 the dispatch falls to 243 workgroups (~31k threads) and the device runs out of work to hide memory latency with faster than the longer contiguous run buys back. It rides a specialization constant, so every arm is the same committed module and they A/B in one binary. Every arm is BIT-IDENTICAL -- each accumulator owns one output element and sums the whole K sequentially, which is the CPU kernel's order -- so this kernel keeps the byte-exact tier that the coopmat and GEMV tactics gave up; a memcmp gates that. Vulkan-only |

docs/STATUS.md

@@ -422,7 +422,7 @@ Parakeet ASR (2026-08-07): *CPU-correct, ON THE ONE SURFACE (ROW 1)*. Ids exact
 LoRA (W1 CPU runtime brick landed; not yet usable end-to-end), multi-GPU,
 Vulkan (opt-125m exact; 25 native +8 GDN, both
 recurrences + fused attn preamble; 27B prefill 21.5x, decode
-4.28/4.35; submit pipelined -1.41 ms/tok 5/5; #125
+4.36/4.35 MET; barriers -19.8%, GPU -1.09 ms; #125
 [campaign](../.agents/specs/vulkan-full-support.md)), ROCm (W0 community-green
 on 4 gfx archs (#41); the ratified (b) APU unified-memory fix is in
 (**blind-written, unverified**); M2 needs verification; gfx1201 hipBLAS +