perf(gdn): dispatch exact causal-conv chunks

[richiejp]

Row

KERNEL-SSM-MAMBA — exact causal-convolution chunk-dispatch sublever; one row per PR.

Before starting

• Issue/PR search and existing claim: no matching open issue or PR; this clean transplant supersedes the earlier local seven-commit benchmark branch.
• Roadmap or matrix row, plus scripts/ready-for-helper.py result when applicable: .agents/kernel-matrix.md:156 (KERNEL-SSM-MAMBA, INVENTORIED); feeds ROAD-V1-C2-LOCAL-BF16. The scoped spike is .agents/specs/sm120-qwen35-conv-chunking-2026-08-07.md.
• Exact current-code and test/evidence anchors inspected: src/vt/cuda/cuda_gdn.cu:702-819, include/vllm/v1/attention/backends/gdn_attn.h:33-43, tests/vt/test_ops_gdn.cpp, and pinned vLLM vllm/model_executor/layers/mamba/ops/causal_conv1d.py:15-28,71-79,123-124 at 555967922.

What changed

Build exact (sequence, 8-token chunk) causal-convolution descriptors once per engine step, upload them once, reuse them across GDN layers, and dispatch one register-kernel program per descriptor. Exact dispatch is default ON; VT_CONV_EXACT_CHUNKS=0 restores the former whole-sequence mapping and VT_CONV_REG=0 retains the tiled/scalar rollback. The PR also corrects vllm-bench to drive the production AsyncLLM frontend used to select and measure this kernel lever.

Evidence

• scripts/agent-preflight.sh passes: all record gates, mutation suites, live-row audit, and committed-range document checks green against canonical upstream/main f91a5917a.
• Tests covering this change: contained CPU/CUDA build; focused CPU CTest 6/6; full CUDA test_ops_gdn 66/66 cases, 4300/4300 assertions; cached Qwen3.5-4B 3/3 cases, 1672/1672 assertions. Rollback/default production token files compare byte-for-byte.
• Same-change doc obligations: docs/STATUS.md, docs/BENCHMARKS.md, and docs/FEATURES.md updated; complete repro and profiler evidence is in docs/bench-evidence/qwen35-4b-sm120-main-20260807.md.

Speed claims

• The operator ran the numbers under ${GPU_LOCK} and they are recorded in docs/BENCHMARKS.md with the repro recipe.

On the local RTX 5070 Ti / Qwen3.5-4B c32 workload, a clean-current-main same-binary Nsight Systems comparison reduces causal-conv GPU time from 720.216507 ms to 234.379395 ms (3.072866x) and improves the profiled enclosing run from 6587.66 to 6727.35 tok/s (+2.1205%), with byte-identical tokens. Three alternating unprofiled pairs measured +2.152% total/output throughput and improvements in TTFT, TPOT/ITL, and E2E latency. The sealed same-hardware vLLM throughput ratio becomes 1.021246x.

Honest gaps

• The hardware-unavailable 27B and 35B release gates remain pending; the 4B result is not extrapolated to them.
• Against the sealed vLLM denominator, TTFT, TPOT/ITL, and peak VRAM remain open. The causal-conv kernel retains a 1.611730x residual gap.
• Fresh full-oracle reruns were voided by FlashInfer/Torch/Triton JIT infrastructure failures, so the accepted cross-engine comparison retains the sealed same-box denominator; same-binary A/B and current-main reproduction are fresh.
• Broad generic Mamba support remains inventoried; this changes the existing Qwen GDN-specific path.
• The PR is draft pending independent review; no independent reviewer was available in this single-agent run.

[localai-bot]

Operator triage: parking blocker named per protocol — this draft is CONFLICTING against main (the GDN/qwen3_5 files moved under it: the Kimi runner fold #122 landed config-side GDN-geometry synthesis and main took several Vulkan/device lanes). Since it touches SACRED-adjacent files (qwen3_5.cpp, gdn_attn, cuda_gdn.cu), the standing bar applies: rebase onto current main, re-run the qwen35/qwen27 paged-forward + GDN suites on the rebased tree, and mark ready — then it enters the normal mutation-review + land flow. Happy to do the record-file reconciliation at landing (keyed files = main wholesale + your rows) if you handle the code-side rebase.

[localai-bot]

Approved. Verified on the merge result against current main, not just on the branch.

What I ran (merge of this branch into 1f3b7148, conflicts resolved):

• Every scripts/check-*.py: green, including device-leakage.
• Clean CPU build: 1103/1103 targets.
• ctest: 356/356. One failure under -j8 (test_engine_core_proc) passes serially — the known parallel-starvation flake, not a regression from this change.

On the code: the exact (sequence, 8-token chunk) program list is the right mirror of upstream compute_causal_conv1d_metadata, and I like that the descriptor is optional on CausalConv1dArgs with the pair validated in vt::CausalConv1dFwd — CPU keeps its scalar reference and no other backend has to care. Default-ON with VT_CONV_EXACT_CHUNKS=0 as the same-binary rollback matches our parity-enabler policy exactly.

On the CI that was red: device-leakage was our blocker, not yours. runner.cpp named DeviceType::kCUDA in the device-agnostic layer — my own regression, which you flagged in the honest-gaps section of this PR. Fixed on main in #159 via a Backend::SupportsAsyncSampledTokenReadback capability, in the shape of SupportsAuxStream. Thanks for reporting it rather than working around it.

Conflicts were records-only and resolved per our rules: .agents/state.md and .agents/benchmark-record.md unioned as append-logs; scripts/merged-gemm-consistency-allowlist.txt had a gemma4_moe key added independently on both sides, kept as one entry.

Merge is prepared and green locally. It is queued behind a repo-wide agent-record break (.agents/policy-cutover points at a pre-squash SHA that never landed on main), which I have raised with mudler — nothing for you to do.