Merge #127 (richiejp): perf(gdn) dispatch exact causal-conv chunks - #165
Merged
Conversation
Build and upload exact causal-convolution work descriptors once per step, reuse them across GDN layers, and make the exact register-kernel mapping the default. Keep the production AsyncLLM benchmark path in the same checkpoint so the measured scheduler and kernel improvement is reproducible on current upstream/main. Row: row/KERNEL-SSM-MAMBA-EXACT-CHUNKS FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Canonicalize semantic consumer bodies without Python-version-only AST fields, and gate the representation under Python 3.12 and 3.13. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Make the row-PR provenance visible to GitHub's synthetic merge check without changing the measured tree. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the merged Gemma-4 MoE path as known merged-GEMM drift and close the stale embeddings claim after PR #137 landed. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
perf(gdn): dispatch exact causal-conv chunks Contributed by Richard Palethorpe <io@richiejp.com>; merged with --no-ff so the four authored commits land under their own authorship. Mirrors upstream compute_causal_conv1d_metadata: grid.y becomes a flattened list of (sequence, 8-token chunk) programs instead of a rectangular grid that either padded or ran sequences serially. The descriptor is optional on CausalConv1dArgs and validated as a pair in vt::CausalConv1dFwd; CUDA consumes it, CPU keeps its scalar reference. Default ON per the parity-enabler policy, with VT_CONV_EXACT_CHUNKS=0 as the same-binary rollback. Contributor measured 720.047 -> 234.607 ms (3.07x) on the causal-conv family, sm_120 Qwen3.5-4B c32, byte-identical output tokens. The branch's only red check was device-leakage, which was OUR blocker: runner.cpp named DeviceType::kCUDA in the device-agnostic layer. Fixed on main in #159. Verified on the merge result: every scripts/check-*.py green, clean CPU build 1103/1103, ctest 356/356 (test_engine_core_proc fails only under -j8 and passes serially, the known parallel-starvation flake). CI on the #165 landing branch also had build-test-cpu, build-test-vulkan and device-leakage green. Conflicts, three files, all records: .agents/state.md and .agents/benchmark-record.md unioned per the append-log rule; scripts/merged-gemm-consistency-allowlist.txt had a gemma4_moe key added independently on both sides, kept as one entry with main's text. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Opus 5 (1M context)
mudler
force-pushed
the
row/KERNEL-SSM-MAMBA-EXACT-CHUNKS-merge
branch
from
August 8, 2026 22:22
d636a75 to
dbaeb86
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Landing vehicle for #127 by @richiejp. GitHub could not merge #127 directly because it went CONFLICTING against current main; this branch is that merge with the conflicts resolved, so richiejp's four commits land under their own authorship (real
--no-ffmerge, not a re-author).The change
Mirrors upstream
compute_causal_conv1d_metadata:grid.ybecomes a flattened list of(sequence, 8-token chunk)programs instead of a rectangular(channel-tile, sequence, chunk)grid that either padded or ran sequences serially. The descriptor is optional onCausalConv1dArgs(batch_ptr+token_chunk_offset_ptr, validated as a pair invt::CausalConv1dFwd); CUDA consumes it, CPU keeps its scalar reference.Default ON per the parity-enabler policy, with
VT_CONV_EXACT_CHUNKS=0as the same-binary rollback. Contributor measured 720.047 → 234.607 ms (3.07x) on the causal-conv family (sm_120, Qwen3.5-4B, c32), byte-identical output tokens.Verified on the merge result, not the branch
scripts/check-*.py: green, includingdevice-leakagectest: 356/356. One-j8failure (test_engine_core_proc) passes serially — the known parallel-starvation flake, not a regression.Note on #127's red CI
device-leakagewas ours, not theirs:runner.cppnamedDeviceType::kCUDAin the device-agnostic layer (my regression, which richiejp reported in their honest-gaps section). Fixed on main in #159 via aBackend::SupportsAsyncSampledTokenReadbackcapability.Conflicts (records only)
.agents/state.md,.agents/benchmark-record.md— union, per the append-log rulescripts/merged-gemm-consistency-allowlist.txt— both sides independently added agemma4_moekey; kept one entry with main's landed textCloses #127.