perf(gdn): optimize post-conv and causal-conv kernels

[richiejp]

Row

KERNEL-SSM-MAMBA — Qwen GDN post-conv and causal-conv kernel sublevers; one row per PR.

Before starting

• Issue/PR search and existing claim: this is a stacked follow-on to perf(gdn): dispatch exact causal-conv chunks #127, which supplies the exact causal-conv chunk descriptors measured here. No separate matching open PR was found for these two opt-in kernel specializations.
• Roadmap or matrix row, plus scripts/ready-for-helper.py result when applicable: .agents/kernel-matrix.md:157 (KERNEL-SSM-MAMBA, INVENTORIED), feeding ROAD-V1-C2-LOCAL-BF16. Spikes: .agents/specs/sm120-qwen35-postconv-token-tile-2026-08-08.md and .agents/specs/sm120-qwen35-conv-channel-tile-2026-08-08.md.
• Exact current-code and test/evidence anchors inspected: src/vt/cuda/cuda_gdn.cu, src/vt/cuda/gdn_prefill_conv.h, tests/vt/test_gdn_prefill_conv.cpp, tests/vt/test_ops_gdn.cpp, and pinned vLLM/FLA causal-conv plus fused post-conv kernels recorded with file:line anchors in the two spikes.

What changed

Add two independently selectable, default-off CUDA specializations. VT_GDN_POSTCONV_TOKEN_TILE=1 ports the upstream-shaped 16-token/four-warp post-conv schedule while preserving the existing 128-lane reduction tree byte-for-byte. VT_CONV_CHANNEL_TILE=1 specializes the causal-conv path for width four at the existing one-channel-per-thread grid; arm =2 is retained as the measured 256-channel/two-channels-per-thread falsification arm. Production dispatch and the portable mutation gate share one callback dispatcher, so the tests prove that each selected arm is actually invoked.

Evidence

• scripts/agent-preflight.sh passes after rebasing onto canonical upstream/main c05cee1d2; all record gates, mutation suites, live-row audit, and committed-range document checks are green.
• Tests covering this change: contained CUDA rebuild including an explicit vllm-bench relink; portable dispatch 9/9 cases, 88/88 assertions; CUDA GDN 67/67 cases, 4631/4631 assertions; Qwen3.5 paged-forward 4/4 cases, 8/8 assertions. The accepted production arms produce identical token hashes.
• Same-change doc obligations: docs/STATUS.md and docs/BENCHMARKS.md updated; the model/feature surface did not expand. Full profiler evidence and the void stale-link series are retained in the two specs and .agents/benchmark-record.md.

Speed claims

• The operator ran the numbers under ${GPU_LOCK} and they are recorded in docs/BENCHMARKS.md with the repro recipe.

On the local RTX 5070 Ti / Qwen3.5-4B c32 workload, the post-conv tile reduces graph-node time 227.887 to 122.587 ms (1.859x) and improves every observed enclosing axis, including total/output throughput by 0.532%. The causal-conv K4 arm reduces graph-node time 234.605 to 219.506 ms (6.44%) and improves whole-run total/output throughput by about 0.12%. The 256-channel arm is 4.05% slower than the K4 arm and is recorded as falsified.

Honest gaps

• This PR is stacked on perf(gdn): dispatch exact causal-conv chunks #127. Until perf(gdn): dispatch exact causal-conv chunks #127 merges, GitHub shows its exact-chunks commits in this PR too; after that merge, this PR reduces to the opt-in follow-on commits.
• Both specializations remain opt-in pending repeated A/B and hardware-unavailable 27B/35B gates. The 4B result is not extrapolated to the release models.
• Same-tool residuals remain: post-conv is 1.135x and causal conv K4 is 1.509x slower than pinned vLLM.
• The first production causal-conv profile series was void because vllm-bench had not relinked the rebuilt CUDA library. The accepted series explicitly relinked the benchmark and trace-proved the specialized kernels.
• Current main c05cee1d2 independently fails check-device-leakage.py: src/vllm/v1/worker/gpu/runner.cpp contains one new hard-coded DeviceType::kCUDA (DSR kcuda 1 > baseline 0). perf(gdn): dispatch exact causal-conv chunks #127 and this PR inherit that merged-tree failure; neither kernel diff introduced it.

[localai-bot]

Read through this one on top of #127. The shape is right and I want it, with one sequencing note.

What I like:

• Both arms are genuinely opt-in. ConvChannelTileArmFromEnv accepts only the exact strings 1 and 2; unset, 0, and every other spelling keep arm 0, and non-K=4 shapes fall back regardless. GdnPostConvTokenTileFlagIsOn likewise keeps the byte-identical megablock default. That is the conservative default discipline we ask for, and the comments say plainly that a microbenchmark win implies no default flip.
• Putting the arm selection in one ConvChannelTileLaunchContractFor seam shared by the CUDA launcher and the tests is the right call — the tests exercise the production branch logic instead of a parallel copy, with no launch counter or debug state in the hot path.
• VT_CONV_CHANNEL_TILE is documented in docs/ENVIRONMENT.md with the fallback semantics spelled out, and both spikes are written up under .agents/specs/.

Sequencing: this is stacked on #127, so it lands after it. Its pr-size and documentation-checkpoint failures are largely inherited from the stack; once #127 is in, please rebase so this PR shows only its own delta and we can judge those two on their merits.

Not a blocker, a question: since both arms stay off until same-binary correctness and performance gates close — what would close them for you? If it is a GB10/sm_121 A/B I can run it here; if it needs sm_120 specifically, say so and I will note the arms as measured-on-sm_120-only in the kernel matrix so nobody later mistakes "opt-in" for "unproven".

Same note as on #127: agent-record is currently red repo-wide because .agents/policy-cutover points at a pre-squash SHA that never landed on main. Not yours.

[localai-bot]

Reviewed on GB10 (sm_121a, full production build: CUTLASS + FA2 + vendored Triton-AOT). The structure of this PR is exactly right — committed spikes with upstream anchors, two independently selectable default-off arms, and a retained falsification arm. Most of the red CI is staleness. But there is one real blocker, and it is in the new work.

Blocker: a 0-ulp equivalence regression that main does not have

TEST CASE: CUDA causal_conv1d_fwd register kernel (VT_CONV_REG) matches tiled 0-ulp
tests/vt/test_ops_gdn.cpp:726: ERROR: CHECK( bad == 0 ) is NOT correct!
  values: CHECK( 2661 == 0 )
  logged: c := 1024,  k := 4,  cb.in := 0,  cb.out := 0,  row_padding := 0

Side by side, same box, same build flags, same command:

tree	test_ops_gdn
current main (0cf3dbbb)	66 cases, 4300/4300, SUCCESS
this branch rebased on main	67 cases, 4630/4631, 1 FAILED

c=1024, k=4 is precisely the shape VT_CONV_CHANNEL_TILE specialises, and 2661 mismatching elements is not a rounding tail — it is a wrong result, at default settings, on a test that asserts bit equality. Note this fires with the arms unset, so it is not confined to the opt-in path.

Worth checking whether the specialisation changed what the "tiled" reference computes for k=4, rather than only what the register kernel computes — if the reference moved, the test is comparing two new things and the 0-ulp claim in the spike no longer holds.

What I verified, so you don't have to re-do it

Rebasing onto main clears three of the four red checks by itself:

• device-leakage — not your bug. Your branch predates 1f3b7148, which replaced the raw queue.device.type == kCUDA test in QueueSupportsAsyncInputCombine with the Backend::SupportsAsyncSampledTokenReadback() capability. Rebased, the DSR bucket is clean.
• pr-size — errors with base must be an ancestor of head, pure staleness. Passes after rebase.
• agent-record — green on current main.

And the default-off contract holds, which is the important safety property:

default (arms unset)        27B 235/235   35B 315/315   (SACRED)
VT_GDN_POSTCONV_TOKEN_TILE=1  27B 235/235
VT_CONV_CHANNEL_TILE=1        27B 235/235
test_gdn_prefill_conv         88/88

So both arms are token-safe on the 27B gate; the problem is confined to the conv equivalence test above.

Still owed after that

• doc-checkpoint is per-commit: commits 66087699 and the final record commit change .agents/benchmark-record.md without touching .agents/NOW.md. Both need the NOW row in the same commit.
• STATUS ratchet slack — test_the_live_page_keeps_the_character_ratchet_tight wants slack <= 25; re-pin chars to the measured value in the same change.

Heads-up for the rebase: main rewrote AGENTS.md and deleted the structured state record (.agents/state.csv, state-events/, state-index/), so drop any edits to those — narrative lives in the commit message now. I hit a conflict there myself and briefly re-added a 7k-line archived blob before catching it.

I haven't pushed the rebase or touched the code — findings go back to you rather than getting repaired in review. Fix the conv equivalence and the two doc gates and I'll re-run this whole set and merge.