mudler · localai-bot · Aug 8, 2026 · Aug 7, 2026 · Aug 7, 2026 · Aug 7, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -28,6 +28,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | Supported-models list | **LANDED**: FEATURES arch table CI-bound (33 archs) | — |
 | `/v1/videos` OpenAI shape | **MERGED** (#71): Sora `model`/`size`/`seconds` + `GET /{id}/content` | `row/SERVE-VIDEOS-REFS` PR open: reference conditioning |
 | `BACKEND-ROCM` W0 | Skeleton in; **HIP never compiled** (no AMD HW) | #41 contributors build it; a compile error IS the deliverable |
+| Release | SPIKE; 30/30 | #129 |
 | Surface coverage (`ARCH-ONE-SURFACE`) | **ROW 8 LANDED; #139 repair CPU-GREEN**: ABI v14 stable; registry-resolved named platform; DSR 39→32; execution guard 52/52 | Fresh re-review #139; CUDA A/B residual |
 
 In-flight (default-OFF, not pushed): `laguna-fp4proj-prod`, laguna

diff --git a/.agents/coordination.md b/.agents/coordination.md
@@ -118,6 +118,24 @@ without the selected contention proof for their entire run are discarded.
 
 ## Active claims
 
+**Server binary release-matrix spike (`ENG-RELEASE-BINARIES`, 2026-08-07,
+`CLAIM-ENG-RELEASE-BINARIES-SPIKE`).** Codex (GPT-5), isolated worktree
+`/home/mudler/_git/vllm.cpp-release-binaries`, branch
+`row/ENG-RELEASE-BINARIES`, base `origin/main` `9cb799cb`. Records/spec-only
+claim for issue #117 and the user-approved hybrid stable/preview downloadable
+server-bundle matrix, including the fully-static feasibility boundary. Owns
+only NEW `.agents/specs/release-binary-matrix.md`, the new
+`ENG-RELEASE-BINARIES` engine-matrix row, its single roadmap portfolio link,
+this claim, the required current-state checkpoint in `docs/STATUS.md` and
+`docs/BENCHMARKS.md`, the engine inventory count pin in
+`scripts/check-agent-record.py`, `.agents/NOW.md`, and the append-only
+`.agents/state.md`.
+No release workflow, CMake/package implementation, source, test, model, kernel,
+generated fixture, download, build, GPU, or service change is in scope. Draft
+PR [#129](https://github.com/mudler/vllm.cpp/pull/129) is the claim. The spike
+contract incorporates the user-reviewed fat-CUDA/adaptive-CPU primary-artifact
+revision and awaits fresh review; the row stays `SPIKE`.
+
 **CPU grouped keep-quant GEMM activation-dtype P0 (`QUANT-GGUF-CIQ-GEMM`,
 2026-08-06, `CLAIM-QUANT-GGUF-CIQ-GROUPED-DTYPE`).** Claude Code
 (claude-opus-5), isolated worktree `/home/mudler/_git/vllmcpp-ciq-grouped`,
@@ -1395,6 +1413,7 @@ table, tests, CMake. Details in the state-log entry of the same date.
 
 | Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
 |---|---|---|---|---|---|---|---|
+| `CLAIM-ENG-RELEASE-BINARIES-SPIKE` | `ENG-RELEASE-BINARIES` | Codex (GPT-5) | `/home/mudler/_git/vllm.cpp-release-binaries`; records/spec only, no build/GPU/download/service work | `row/ENG-RELEASE-BINARIES`, base `9cb799cb`, draft PR #129 | Accepted release-bundle spike plus its matrix/roadmap/coordination/state/NOW and public-doc lifecycle checkpoint; no CMake, workflow, source, test, or artifact implementation | `ACTIVE` | 2026-08-07 — user-reviewed revision complete: primary fat CUDA + adaptive CPU per host ABI, optional per-SM diagnostics; row stays `SPIKE`; awaiting fresh review |
 | `CLAIM-KIMI-LINEAR-W0` | `MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm` (stays SPIKE — dedicated W0 spike) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a771cc029e6843be0`; CPU-only, records-only — NO build, NO GPU, NO download (two GPU jobs queued ahead; the W0 GPU golden capture is a SEPARATE later step) | branch `worktree-agent-a771cc029e6843be0`, base `origin/main` HEAD `10dd23ee` | The FULL dedicated W0 spike for `KimiLinearForCausalLM` (Kimi-Linear-48B-A3B) per the spike-first protocol, so W1 implementation can start immediately. Owns ONLY: NEW `.agents/specs/kimi-linear.md`, the `MODEL-TEXT-kimi-linear-*` matrix row (Spike link → the dedicated spec + note; STAYS `SPIKE` because it is actively claimed — protocol forbids a claimed row from being `READY`; checklist mark stays `📋`), this claim row + the narrative block below, the roadmap breadth note, `docs/STATUS.md`/`docs/BENCHMARKS.md`/`docs/FEATURES.md` one-liners, the `.agents/NOW.md` live-claim row + stamp, and one `.agents/state.md` entry. **NON-COLLISION:** records-only — touches NO model/kernel/registry/loader source, NO CMake, NO test. Co-owns the Kimi-Linear row with `CLAIM-MLA-DEEPSEEK` (MLA half) + `CLAIM-KDA-KERNEL` (KDA host refs), exactly as the K3 row is co-owned by `CLAIM-KIMI-K3-SCOPE`/`CLAIM-KIMI-K3-W2-W5`. | `DONE` | 2026-08-05 — **W0 DEDICATED SPIKE LANDED (records-only, NOT pushed).** Authoritative `config.json` fetched (27 layers = 20 KDA + 7 NoPE-MLA; 256e/top-8/1-shared sigmoid `noaux_tc` `routed_scaling=2.446` `first_k_dense_replace=1`; `num_nextn_predict_layers=0` ⇒ no MTP); reuse-vs-new map with our `file:line`; HW-fit (FITS one GB10, 91.5 GiB / 0.77× pool — real e2e SACRED gate reachable); W0-W7 breakdown + GPU golden-capture recipe. NET-NEW = KDA device kernel (host refs landed, `CLAIM-KDA-KERNEL`), NoPE-MLA branch, hybrid schedule/het-KV, loader name-map. Record checkers green (`check-model-checklist`, `check-agent-record`, `check-doc-checkpoint`, `check-now-current`). |
 | `CLAIM-KIMI-LINEAR-W1` | `MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm` (stays SPIKE — W1 registry/config/loader scaffolding; forward REFUSES-by-name) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-a11ae231a71e8694b`; CPU-only `build-cpu` (`-DVLLM_CPP_CUDA=OFF` Release); NO GPU, NO download — additive TU + one REGISTER line, forward is a `VT_CHECK(false)` stub | branch `worktree-agent-a11ae231a71e8694b`, base `origin/main` HEAD `053116df` | Kimi-Linear-48B-A3B W1 — registry + config + loader + KV-spec scaffolding so the W3-W6 forward can start. Owns ONLY: NEW `include/vllm/model_executor/models/kimi_linear.h`, NEW `src/vllm/model_executor/models/{kimi_linear_registry,kimi_linear_weights,kimi_linear}.cpp`, NEW `tests/vllm/models/test_kimi_linear_scaffold.cpp`, its two CMake registration lines (`CMakeLists.txt` source list + `tests/CMakeLists.txt`), the `KimiLinearForCausalLM` sorted-set + error-message + model-property inserts in `tests/vllm/models/test_model_registry.cpp`, the `MODEL-TEXT-kimi-linear-*` row cells + checklist Status (STAYS `SPIKE`/`📋` — the forward refuses, rollup UNCHANGED), this claim, `.agents/NOW.md`, `docs/STATUS.md`/`docs/BENCHMARKS.md`/`docs/FEATURES.md` one-liners, and one `.agents/state.md` entry. **NON-COLLISION:** additive TU + one REGISTER line ⇒ ZERO edit to any shared array; the forward is a `VT_CHECK(false)` refuse stub so NO production/device path changes; co-owns the Kimi-Linear row with `CLAIM-MLA-DEEPSEEK` + `CLAIM-KDA-KERNEL` + `CLAIM-KIMI-LINEAR-W0`; MUST NOT touch README, Metal/SACRED/apex/darwin, or any other model/kernel source. | `DONE` | 2026-08-05 — **W1 CPU SCAFFOLDING LANDED (foreground, NOT pushed).** Additive registry (`kimi_linear`/`KimiLinearForCausalLM`, `is_hybrid`, `supports_multimodal=false`) + `ParseKimiLinearParams` (20 KDA + 7 NoPE-MLA schedule, MLA 512/128/64/128 q_lora-null, 256e/top-8/1-shared sigmoid `noaux_tc` 2.446, `first_k_dense_replace=1`; asserts `mla_use_nope`/`q_lora==null`) + `EnumerateKimiLinearTensors` name-map VERIFIED vs the real HF safetensors index (MoE = `block_sparse_moe.*`, the correction of the DERIVED K3 `mlp.*`) + loader THROWS BY NAME on missing/mis-shaped tensors + het KV spec (MLA latent-576 + KDA mamba `12288×3`/`32×128×128`) + REFUSE-by-name forward. `test_kimi_linear_scaffold` **9/9·83** + `test_model_registry` **24/24** + `test_kimi_k3_scaffold` 6/6 unchanged; clean CPU Release build; `check-fusion-consistency`/`check-runner-routing-consistency` green (refuse stub skipped); record checkers rc=0. RESIDUAL (W3-W6): KDA device kernel + NoPE-MLA route + sigmoid-noaux MoE + het-KV born-on-runner forward + the e2e SACRED gate (spec §8 recipe). |
 | `CLAIM-KIMI-LINEAR-W2` | `MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm` (stays SPIKE — W2-W6 CPU REFERENCE forward; the DEVICE runner forward stays refuse-by-name) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/agent-aec13af094cd4692f`; CPU-only `build` (`-DVLLM_CPP_CUDA=OFF` RelWithDebInfo); NO GPU, NO download — additive forward TU + a host-materialization pass in the existing loader + additive test | branch `laguna-s21-w7-speed-profile` (worktree), base `origin/main` HEAD `0988db48` | Kimi-Linear-48B-A3B W2-W6 — the CPU REFERENCE forward, per-op gated, replacing the refuse-by-name host `KimiLinearModel::Forward`. Owns ONLY: NEW `src/vllm/model_executor/models/kimi_linear_forward.cpp`, NEW `tests/vllm/models/test_kimi_linear_forward.cpp`, its two CMake lines (`CMakeLists.txt` source list + `tests/CMakeLists.txt`), ADDITIVE host-weight structs + per-op decls in `include/vllm/model_executor/models/kimi_linear.h`, the host-materialization pass in `src/vllm/model_executor/models/kimi_linear_weights.cpp`, the device-refuse doc/rename in `src/vllm/model_executor/models/kimi_linear.cpp`, the `MODEL-TEXT-kimi-linear-*` row cells + checklist Status (STAYS `SPIKE`/`📋` — the DEVICE forward refuses; rollup UNCHANGED), this claim, `.agents/NOW.md`, `docs/STATUS.md`/`docs/BENCHMARKS.md` one-liners, and one `.agents/state.md` entry. **NON-COLLISION:** additive TU + additive header structs; the host `Forward` becomes real but `ForwardDevice` (the DEFAULT gather_logits runner path) STAYS `VT_CHECK(false)` so the born-on-runner CI guard skips it (NO allowlist) and NO production/device path changes; co-owns the Kimi-Linear row with `CLAIM-MLA-DEEPSEEK` + `CLAIM-KDA-KERNEL` + `CLAIM-KIMI-LINEAR-W0/W1`; MUST NOT touch README, Metal/SACRED/apex/darwin, or any other model/kernel source. | `DONE` | 2026-08-05 — **W2-W6 CPU REFERENCE FORWARD LANDED (foreground, NOT pushed).** Real host `KimiLinearModel::Forward` composes the whole 27-layer hybrid from the landed primitives: KDA layer (`vllm::kimi_kda` refs + the gated-delta recurrence `fused_recurrent.py:122-149`), NoPE-MLA (materialized-MHA reference, scaling `qk_head**-0.5`, no RoPE), sigmoid `noaux_tc` MoE (bias-select/unbiased-weight, renormalize, `routed_scaling=2.446`) + shared expert, dense layer-0 SwiGLU; loader now MATERIALIZES the host float weights (bf16/f32→f32). `test_kimi_linear_forward` **6/6·246** (KDA==kimi_kda composition; NoPE-MLA==materialized-MHA ref; router+block==hand-computed; loader materializes; whole 2-layer forward finite/coherent; greedy-decodes 5 tokens, context advances) + `test_kimi_linear_scaffold` 9/9·83 + `test_kimi_kda` 14/14 UNCHANGED; clean CPU build; `check-fusion-consistency`/`check-runner-routing-consistency` green (DEVICE forward refuse-skipped, no allowlist). RESIDUAL: the DEVICE born-on-runner forward + the W0/W7 e2e SACRED token golden on GB10 (spec §8 recipe) + speed. |

diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md
@@ -42,11 +42,11 @@ forensics: roadmap_v1.md and the parity ledger.
 | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 3 | 0 | 1 | 5 |
 | Structured output and tools | 7 | 0 | 3 | 0 | 0 | 2 | 0 | 0 | 2 |
 | Speculative decoding | 21 | 0 | 0 | 1 | 0 | 4 | 0 | 4 | 11 |
-| Serving, API, CLI, library | 24 | 10 | 2 | 1 | 0 | 4 | 2 | 1 | 4 |
+| Serving, API, CLI, library | 25 | 10 | 2 | 2 | 0 | 4 | 2 | 1 | 4 |
 | LoRA and adapters | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
 | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
 | Loading, tokenizer, config | 9 | 3 | 3 | 0 | 0 | 0 | 1 | 1 | 1 |
-| **Total** | **142** | **36** | **16** | **4** | **7** | **20** | **8** | **9** | **41** |
+| **Total** | **143** | **36** | **16** | **5** | **7** | **20** | **8** | **9** | **41** |
 
 ## Engine core and scheduling
 
@@ -207,6 +207,7 @@ claims it.
 | `SERVE-CLI-BENCH` | Serve and latency/throughput/serve benchmark modes | T0 | `vllm/entrypoints/cli/serve.py:44`; `vllm/entrypoints/cli/benchmark/main.py:29` | separate binaries + explicit scheduler-capacity flags `examples/server/main.cpp:63,96,116,170`; `examples/bench/main.cpp:40,109`; `examples/bench/bench_core.h:96,468` | server help contract `examples/CMakeLists.txt:34`; benchmark `tests/examples/test_bench.cpp:15,48` | `planned: specs/cli-serve-bench.md` | `PARTIAL` | - |
 | `SERVE-GATE-ONLINE` | Same-corpus online correctness, TTFT/TPOT/ITL, throughput and peak-memory gate vs vLLM v0.25.0 | T0 | `vllm/benchmarks/serve.py:1,581-615`; [v0.25 audit](sync/2026-07-12-702f481.md); `tests/benchmarks/test_serve_cli.py:1` | Schema-v5 harness plus [trace controller](../include/vt/cuda/cuda_profiler_control.h#L13), [production component driver](../scripts/dgx-gdn-packed-component.sh), and fail-closed [component finalizer](../tools/bench/gdn_packed_component.py) | **BINDING `9ecd9d0`: 114/124** (async default ON; mem 4/4, c1 20/20, c2 20/20, c16 19/20, c4 & c32 18/20, c8 15/20; `benchmark_binding` refers here, superseding `3f256ab` 55/124 and `246a23c` 49/124, both retained immutable). Two-grid totality with `f0fb727` (111/124) is 115/124 effective parity vs vLLM 0.25.0 (27B). Async CLOSED the c16/c32 ITL tails (ours now BEATS vLLM: c16 p99 1.055, c32 p90 1.034/p99 1.078) and leaves a stable c8 `p99_itl` ~0.86 residual, ROOT-CAUSED (2026-07-18, `CLAIM-C8-P99-TAIL-1`, [spec](specs/c8-p99-itl-tail-2026-07-18.md)) as IRREDUCIBLE-AS-MIRRORED: our deterministic synchronous forward keeps co-admitted c8 requests in byte-identical lockstep where vLLM's async-future jitter de-phases them; the c16/c32 INVERSION proves this is the trailing edge of the per-step determinism that wins c16/c32 + throughput, not a capability gap (scheduler + async placeholder byte-identical, `tests/vllm/v1/test_scheduler_wave.cpp:265`, [tail spec](specs/tail-stall-analysis-2026-07-16.md)). Full grid + per-binding forensics: roadmap_v1.md + parity ledger; no packed speed credit | [online serving gate](specs/cuda-online-serving-gate.md); [merged GDN projections](specs/gdn-merged-input-projections.md); [packed decode](specs/gdn-packed-decode.md) | `ANCHOR-BACKFILL` | CLAIM-SERVE-GATE-1 |
 | `SERVE-E2E-NIGHTLY` | Server conformance and real-model nightly suites for all release gates | T0 | `tests/entrypoints/openai/`; `tests/v1/e2e/`; `.buildkite/test-pipeline.yaml` | current unit/conformance tests only; no scheduled DGX suite | `tests/vllm/entrypoints/openai/test_conformance.cpp:1`; `tests/parity/test_qwen36_paged_engine.cpp:78`; `tests/parity/test_qwen27_paged_engine.cpp:110` | `planned: specs/server-e2e-nightly.md` | `INVENTORIED` | - |
+| `ENG-RELEASE-BINARIES` | Downloadable host-ABI-specific `vllm-server` bundles: adaptive CPU and fat CUDA primary artifacts, optional per-SM diagnostics, and literal-static feasibility boundary | T0 | vLLM release lanes `.buildkite/release-pipeline.yaml:1-18,34-170` @ `555967922`; release-image dependency boundary `docker/Dockerfile.cpu:262-290` | server target only `examples/CMakeLists.txt:54-64`; CPU per-TU/runtime-dispatch baseline `CMakeLists.txt:870-890`, `src/vt/cpu/cpu_matmul_elem.cpp:553-612`, `src/vt/cpu/cpu_quant_dot_arm.cpp:39-77`; cross-family CUDA fat/per-source-gencode and multi-SM AOT gaps remain; no install/archive/publish implementation | help smoke only `examples/CMakeLists.txt:59-63`; issue `#117`; user-reviewed fat-CUDA/adaptive-CPU matrix and gates in [release-binary-matrix.md](specs/release-binary-matrix.md) | [release-binary-matrix.md](specs/release-binary-matrix.md) | `SPIKE` | `CLAIM-ENG-RELEASE-BINARIES-SPIKE` |
 | `SERVE-CLI-CHAT` | Interactive `chat` and `complete` commands against a running OpenAI-compatible server, plus preservation of the existing local-model completion invocation | T1 | registration `vllm/entrypoints/cli/main.py:17-37,73-98`; client/model resolution + stream shaping `vllm/entrypoints/cli/openai.py:30-100`; chat `:155-234`; complete `:237-312` at `5559679229` | current in-process completion only `examples/cli/main.cpp:1-207`; remote command implementation absent | C-ABI stream baseline `tests/capi/test_capi.cpp:567-711`; chat-template baseline `tests/capi/test_chat_prompt.cpp:37-89`; command/fake-server tests absent | [cli-chat-complete.md](specs/cli-chat-complete.md) | `ANCHOR-BACKFILL` | `CLAIM-SERVE-CLI-CHAT-SPIKE` |
 | `SERVE-POOLING-ENDPOINTS` | Embeddings, pooling, score, rerank, classify HTTP surface (`/v1/embeddings`, `/pooling`, `/score`, `/rerank`, `/classify`). **SPIKED 2026-07-28 (`CLAIM-POOLING`):** the whole pooling task class is scoped in [pooling-task-class.md](specs/pooling-task-class.md) — endpoints depend on the pooling RUNNER (`ENG-POOLING-RUNNER`, W3 LANDED 2026-07-29 — `PoolingRunner` returns pooled data via the model `Pooler`) and a concrete pooling model. Endpoint protocol + handler port is the W4 brick | T2 | `vllm/entrypoints/pooling/embed/api_router.py:28`; `vllm/entrypoints/pooling/scoring/api_router.py:37,71`; `vllm/entrypoints/pooling/classify/api_router.py:26` | - | - | [pooling-task-class.md](specs/pooling-task-class.md) | `SPIKE` | `CLAIM-POOLING` |
 | `ENG-POOLER-SEQ` | The non-generative POOLER OP — turn hidden states into a pooled embedding/logit row instead of a sampled token. **W1 LANDED + CPU-GATED 2026-07-28 (`CLAIM-POOLING`, NOT pushed):** the sequence pooling methods `CLSPool`/`LastPool`/`MeanPool` (+ `GetSeqPoolingMethod` factory) over a packed `[num_tokens, hidden]` CPU buffer keyed by a minimal `PoolingCursor` (CLS/MEAN reject partial prefill, LAST allows it, MeanPool upcasts to float32) and the activation heads `PoolerIdentity`/`PoolerNormalize` (L2 `F.normalize`)/`PoolerMultiLabelClassify` (sigmoid)/`PoolerClassify` (sigmoid if `num_labels<2` else `softmax`). Unit-gated vs DOUBLE-PRECISION references, RED-first. **W2 LANDED + CPU-GATED 2026-07-29 (`CLAIM-POOLING`, NOT pushed):** the pooler HEADS composite (`EmbeddingPoolerHead` = projector→matryoshka→normalize; `ClassifierPoolerHead` = classifier→`(logit-mean)/sigma`→activation), the `SequencePooler` (method∩head task intersection) + `PoolerForEmbed`/`PoolerForClassify` factories, the `DispatchPooler` groupby-task routing (`ForEmbedding`/`ForSeqCls` + a mixed embed+classify batch + ctor task-support validation), and the `PoolerConfig`/`PoolingParams`/`PoolingParamsUpdate` structs; `test_pooler_heads` 27/27 (240 asserts) vs double-precision refs, RED-first (disable matryoshka slice + logit_mean calibration → 8 cases / 50 asserts fail). RESIDUALS (named, spec §Work breakdown): the endpoints (W4), tokwise `AllPool`/`StepPool` (W5), a concrete pooling MODEL + real-oracle cosine gate (W3-model — see `ENG-POOLING-RUNNER`) | T2 | `vllm/model_executor/layers/pooler/seqwise/methods.py:35-121`; `vllm/model_executor/layers/pooler/activations.py:106-158`; `vllm/model_executor/layers/pooler/seqwise/heads.py:19-196`; `vllm/model_executor/layers/pooler/seqwise/poolers.py:41-138`; `vllm/model_executor/layers/pooler/special.py:23-140`; `vllm/model_executor/layers/pooler/common.py:12-30`; `vllm/pooling_params.py:35-70`; `vllm/config/pooler.py:16-90`; `vllm/v1/pool/metadata.py:13-71`; `tests/model_executor/layers/test_pooler_methods.py`, `tests/model_executor/layers/test_pooler_activations.py`, `tests/model_executor/layers/test_pooler_heads.py` | `include/vllm/model_executor/layers/pooler/{methods,activations,pooling_metadata,common,pooling_params,pooler_config,heads,poolers,dispatch_pooler}.h` + `src/vllm/model_executor/layers/pooler/{methods,activations,heads,poolers,dispatch_pooler}.cpp` — anchor `src/vllm/model_executor/layers/pooler/dispatch_pooler.cpp:13` | `tests/vllm/model_executor/layers/pooler/test_pooler.cpp` (CLS/LAST/MEAN + factory + activations, 50 asserts) + `test_pooler_heads.cpp` (Embedding/Classifier heads + SequencePooler + DispatchPooler, 240 asserts) — anchor `tests/vllm/model_executor/layers/pooler/test_pooler.cpp:81` | [pooling-task-class.md](specs/pooling-task-class.md) | `ACTIVE` | `CLAIM-POOLING` |