spike(kimi-linear): chunk_kda prefill AOT port — kernel set + pinned-config recipe (§17) - #110
Closed
localai-bot wants to merge 1 commit into
Closed
spike(kimi-linear): chunk_kda prefill AOT port — kernel set + pinned-config recipe (§17)#110localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…config recipe (§17) Phase-1 spike of the STRICT residual (c): vLLM runs the PROMPT through FLA's CHUNKED chunk_kda while we still run the recurrent form (a different reduction order that coin-flips the p7 near-tie). Grounds + de-risks the AOT port to mechanical execution. No GPU work, no STRICT claimed. - Exact forward-only kernel set (kda.py @ 555967922): 5 NEW Triton kernels (kda_gate_cumsum; 2x chunk_kda_scaled_dot_kkt inter/intra; recompute_w_u KDA per-channel; chunk_gla_o) + 1 NEW pin of chunk_delta_h.py (USE_GK=1,USE_EXP2=1) + 1 pure reuse (solve_tril -> gdn_tril_h32). Decode stays #104 recurrent (mirrors vLLM's prefill=chunk / decode=recurrent split). - 5 harness bodies authored (verbatim FLA ports, AOT-adapted) staged in .agents/specs/kda-chunk-aot/ (CI-safe: drift globs triton_kernels/*.py). - Pinned-config record + full _vllm_triton_aot_declare recipe + vt::KdaChunkPrefill op design + RED-first gate plan (spec §17). - USER 2026-08-07: bar is MEET vLLM SPEED (util-0.82 single-seq eager arm); the HW-forced-indirect framing is SUPERSEDED — §17.5 adds the matched-config vLLM speed arm + FlashInfer pre-warm / OOM-reboot protocol. - Repaired pre-existing env-doc gap: documented VT_KIMI_DEVICE_MLA (#107). Phase-2 (box): regen sm_121a cubins, add the §17.3 declarations, wire the op, run RED unit + FLA golden + 48.9B STRICT gate + vLLM speed ladder. Row ACTIVE. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
Collaborator
Author
|
Landed on main as the squash (mudler authorship). Phase-2 (regen + vt::KdaChunkPrefill + the STRICT gate + the matched-config vLLM speed ladder) is the next box campaign. 🤖 Generated with Claude Code |
mudler
added a commit
that referenced
this pull request
Aug 7, 2026
…rness bodies; vLLM-speed bar adopted (#110) row/KIMI-CHUNK-KDA-AOT squash, spike + records (no GPU claims). The exact forward-only launch map of FLA's chunk_kda_with_fused_gate at the pin: 5 NEW Triton kernels (gate_cumsum, scaled_dot_kkt intra/inter, recompute_w_u with KDA's per-K-channel exp2(gk), chunk_gla_fwd_o) + 1 NEW PIN of chunk_delta_h (the vendored GDN gdn_deltah cubin is NOT reusable — pinned USE_G=1,USE_GK=0,USE_EXP2=0 vs KDA's needs) + 1 genuine reuse (gdn_tril_h32, byte-identical sig). Decode stays the #104 recurrent kernel, mirroring vLLM's prefill=chunk/decode=recurrent split. §17: pin tables (BK/BV/BD, warps/stages, grids), exact AOT declarations, dtype map, scalar-literal bakes; the 5 harness bodies authored (verbatim FLA ports, py_compile-clean), staged CI-safe. Regen deferred to Phase-2 deliberately — the harness ABI couples to the vt::KdaChunkPrefill op's confirmed buffer dtypes (§17.4 design). USER DIRECTIVE folded in: the Kimi bar is MEET VLLM SPEED — §17.5 now mandates a matched-config vLLM arm at the §12 recipe (single-seq eager util 0.82, autotune pre-warmed at tiny util, worker parked, one attempt) on the same prompts, making distance-to-bar a measured number. Also repairs the undocumented VT_KIMI_DEVICE_MLA env var. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase-1 spike (this PR is the row claim;
row/KIMI-CHUNK-KDA-AOT). Grounds and de-risks the Kimi-Linear STRICT residual (c) — vLLM runs the PROMPT through FLA's CHUNKEDchunk_kda; we still run the recurrent form, a different reduction order that coin-flips the p7 near-tie (§14/§15/§16, #107). No GPU work, no STRICT claimed.Delivered
chunk_kda_with_fused_gate→_fwd, FLAkda.py@ 555967922): 5 NEW Triton kernels —kda_gate_cumsum_fwd_kernel, the twochunk_kda_scaled_dot_kktsub-kernels (inter/intra),recompute_w_u_fwd_kernel(KDA per-K-channel, ≠ GDNwy_fast.py),chunk_gla_fwd_kernel_o— + 1 NEW pin ofchunk_delta_h.py(USE_GK=1,USE_EXP2=1,USE_G=0,Hg=32; the GDNgdn_deltahpin is not reusable) + 1 pure REUSE (solve_tril→gdn_tril_h32, byte-identical sig). Decode STAYS the perf(kimi-linear): per-channel-decay KDA device kernel vt::KdaGatedDeltaRule — 106→122/128 AND 3.1x speed (§15) #104 recurrentvt::KdaGatedDeltaRule(mirrors vLLM's prefill=chunk / decode=recurrent split)..agents/specs/kda-chunk-aot/— CI-safe (drift check globstriton_kernels/*.pynon-recursively). All passpy_compile._vllm_triton_aot_declarerecipe + thevt::KdaChunkPrefillop design + the RED-first gate plan in spec §17.VT_KIMI_DEVICE_MLA(from Kimi-Linear STRICT close: device NoPE-MLA + chunk_kda/incremental (row/KIMI-STRICT-CLOSE) #107).Phase-2 (box, the next brick — coupled: harness signatures depend on the op's confirmed buffer dtypes, so regen follows the op)
Move harness →
triton_kernels/, add the §17.3 declarations, regen sm_121a cubins (scripts/regen-triton-aot.sh), wirevt::KdaChunkPrefill, run RED-first unit (vs #104 recurrent + #173 host refs + FLA-python golden) + the full 48.9B GB10 STRICT gate (DEVICE_KDA=1+ chunk-prefill) + the vLLM-matched-config tok/s + TTFT ladder. Row STAYS ACTIVE.Gates green:
agent-preflight --stagedexit 0 (state-order, NOW budget, doc-checkpoint, public-doc-tables, env-doc, supported-models, protocol/fusion/routing consistency).🤖 Generated with Claude Code
https://claude.ai/code/session_014fZAcg1WcU8V629k6HWKys