mudler · localai-bot · Aug 6, 2026 · Aug 6, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -26,8 +26,10 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): #31 async mirror on classic dense Qwen3; gate RED→GREEN, SACRED 184/184 | Residual: sibling scope one-liner |
 | MXFP4 parity goal | c8 same-tool diff (`QUANT-CT-MXFP4-C8-DIFF`) SETTLES #50: marlin grouped-5-GEMM (gate_up unfused, 180 vs 144) DOMINANT +0.9/+1.4ms; eager gap +0.88ms graph-closeable nets +1.3%; flash same-grid +11% | NEXT: grouped→dense-direct + gate_up fuse |
 
-In-flight branches (gated default-OFF, not pushed): `laguna-fp4proj-prod`
-(fp4 opt-in), laguna bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.
+In-flight branches (default-OFF, not pushed): `laguna-fp4proj-prod` (fp4),
+laguna bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`. Records:
+`row/SPEC-DECODE-INVENTORY`: 13 vLLM spec-decode methods enumerated from
+source, 9 gaps `INVENTORIED` ([inventory](specs/spec-decode-inventory.md)).
 
 ## Current gate
 
@@ -52,12 +54,11 @@ MEASUREMENT arbitrates; distrust aggregate bytes/time and CROSS-TOOL comparisons
    f32-out caller) once the Laguna fix proves the mechanism.
 4. **Restore `local-ai-worker`** on dgx when the GPU campaign ends
    (`docker update --restart=always` + `docker start`).
-5. **Protocol substrate — partly done.** Claim triage DONE; `docs/STATUS.md`
-   under a shrink-only ratchet; roadmap compacted; `AGENTS.md` tiered. REMAINING:
-   anchor backfill (98 rows `SPIKE`/`ACTIVE`, need code/test anchors; 6 model rows
-   need a DECISION, architecture unregistered); record-era rollover BLOCKED on
-   `check-agent-record.py` binding `DONE` rows to `parity-ledger.md` LINE anchors
-   (re-anchor by ROW ID first; `state.md`/`benchmark-record.md` can roll now).
+5. **Protocol substrate — partly done.** `docs/STATUS.md` shrink-ratchet;
+   roadmap compacted; `AGENTS.md` tiered. REMAINING: anchor backfill (98
+   `SPIKE`/`ACTIVE` rows need anchors; 6 model rows need a DECISION); record-era
+   rollover BLOCKED on `check-agent-record.py` `DONE`->`parity-ledger.md` LINE
+   anchors (re-anchor by ROW ID first).
 
 **Operator/helper protocol**
 ([spec](specs/operator-helper-protocol.md)): roles DECLARED then MATERIALIZED

diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md
@@ -41,12 +41,12 @@ forensics: roadmap_v1.md and the parity ledger.
 | Parallelism | 6 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 5 |
 | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 3 | 0 | 1 | 5 |
 | Structured output and tools | 7 | 0 | 3 | 0 | 0 | 2 | 0 | 0 | 2 |
-| Speculative decoding | 12 | 0 | 0 | 1 | 0 | 4 | 0 | 4 | 2 |
+| Speculative decoding | 21 | 0 | 0 | 1 | 0 | 4 | 0 | 4 | 11 |
 | Serving, API, CLI, library | 22 | 10 | 2 | 1 | 0 | 2 | 2 | 1 | 4 |
 | LoRA and adapters | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
 | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
 | Loading, tokenizer, config | 9 | 3 | 3 | 0 | 0 | 0 | 1 | 1 | 1 |
-| **Total** | **131** | **36** | **16** | **4** | **4** | **21** | **8** | **9** | **32** |
+| **Total** | **140** | **36** | **16** | **4** | **4** | **21** | **8** | **9** | **41** |
 
 ## Engine core and scheduling
 
@@ -179,6 +179,15 @@ claims it.
 |---|---|---|---|---|---|---|---|---|
 | `SPEC-DRAFT-MODEL` | Generic model-agnostic SEPARATE draft-model proposer (`method="draft_model"`): a full smaller standalone LM runs K autoregressive greedy steps to propose K draft tokens; the target verifies in one forward and the longest-accepted-prefix is emitted. Distinct from MTP/EAGLE/DFlash — `pass_hidden_states_to_model=False`, shares NEITHER embeddings NOR lm_head with the target. **W0 spike + W1 CPU brick LANDED 2026-07-29 (`CLAIM-SPEC-DRAFT-MEDUSA`, NOT pushed):** the greedy k-step autoregressive propose (`DraftModelProposeGreedy`/`Batch`) over a `DraftLogitsFn` next-token oracle, reusing the LANDED `SPEC-REJECTION` verify/accept UNCHANGED (only the proposer is net-new, mirror of the SPEC-NGRAM shape). Unit-gated RED-first: propose->verify->accept equivalence (accepted tokens == the target's own greedy run, every draft/target (dis)agreement pattern) + full-acceptance on a matching draft (num_sampled==k+1) + the RED witness that full acceptance DEPENDS on the autoregressive feed-back (5/6 fail with feed-back dropped). `ParseSpeculativeConfigJson` accepts `"draft_model"` (requires `model` + `num_speculative_tokens`). Additive + default-inert (no runner construction; engine byte-identical with no `SpeculativeConfig`). Clean CPU `-Werror`. **RESIDUAL (W3, DGX-offline):** the real draft-model forward behind the oracle (paged KV + CUDA-graph) + e2e greedy our-ON==vLLM-ON token-exact gate + throughput speed gate. | T2 | `vllm/v1/spec_decode/draft_model.py:19` (pass_hidden_states :29, no shared embed/lm_head :108-115); propose `vllm/v1/spec_decode/llm_base_proposer.py:502-767` (`_greedy_sample` :428-438, `set_inputs_first_pass` :838-851, K-1 feed-back :682-761); config `vllm/config/speculative.py:684,692-701,1195`; runner `vllm/v1/worker/gpu_model_runner.py:604-609`; e2e `tests/v1/e2e/spec_decode/test_spec_decode.py:500-561` @ `555967922` | `include/vllm/v1/spec_decode/draft_model_proposer.h`; `src/vllm/v1/spec_decode/draft_model_proposer.cpp`; `src/vllm/config/speculative.cpp` (`draft_model` accept); reuses `src/vllm/v1/spec_decode/rejection_sampler.{h,cpp}` — anchor `src/vllm/v1/spec_decode/draft_model_proposer.cpp:27` | `tests/vllm/v1/spec_decode/test_draft_model_proposer.cpp` (6/6, 41 assertions, CPU; RED-first 5/6 fail with feed-back dropped); ledger [parity-ledger.md](parity-ledger.md) 2026-07-29 — anchor `tests/vllm/v1/spec_decode/test_draft_model_proposer.cpp:143` | [specs/draft-model-medusa-spec.md](specs/draft-model-medusa-spec.md) | `ACTIVE` | `CLAIM-SPEC-DRAFT-MEDUSA` |
 | `SPEC-MEDUSA` | Medusa multi-head speculator (`method="medusa"`): the target carries N extra Medusa LM heads, each predicting ONE future position from the SAME target hidden state in a single (non-autoregressive) pass; `draft_tokens = stack([argmax(head_logits)])` -> `[batch, num_heads]`, `num_speculative_tokens == num_heads`. Verify/accept is the SAME `SPEC-REJECTION` loop (linear, not tree, at this pin). **W0 spike ONLY (`CLAIM-SPEC-DRAFT-MEDUSA`, 2026-07-29):** proposer scoped in [specs/draft-model-medusa-spec.md](specs/draft-model-medusa-spec.md); deferred to W2 because its multi-head target-tap propose needs the target model's Medusa heads (a model change) a pure host brick cannot meaningfully stand up. No code yet. | T2 | `vllm/v1/spec_decode/medusa.py:18` (propose :40-58: `model(hidden)`->per-head `compute_logits`->stacked argmax); config `vllm/config/speculative.py:822,888-889`; runner `vllm/v1/worker/gpu_model_runner.py:642-645` @ `555967922` | - (spike only, not implemented) | - (W2) | [specs/draft-model-medusa-spec.md](specs/draft-model-medusa-spec.md) | `SPIKE` | `CLAIM-SPEC-DRAFT-MEDUSA` |
+| `SPEC-NGRAM-GPU` | On-device n-gram proposer (`method="ngram_gpu"`): the KMP suffix match + batch propose run on GPU (pinned idx/val buffers + a device token table) rather than the CPU `SPEC-NGRAM` path. Draft-FREE, gate-model usable — the cheapest breadth gap (value #1). | T2 | `vllm/v1/spec_decode/ngram_proposer_gpu.py:217`; dispatch `vllm/v1/worker/gpu_model_runner.py:610-626` | - (enumerated, not implemented) | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-SUFFIX` | Suffix Decoding proposer (`method="suffix"`, arXiv 2411.04975): a per-prompt suffix tree over prompt+output proposes drafts (tree-depth / spec-factor / min-token-prob knobs). Draft-FREE but depends on the EXTERNAL `arctic_inference` package (lazy import). Value #2. | T2 | `vllm/v1/spec_decode/suffix_decoding.py:9,26`; dispatch `vllm/v1/worker/gpu_model_runner.py:634-635` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-EAGLE` | EAGLE(1) proposer (`method="eagle"`), distinct from `SPEC-EAGLE3`: a separate small draft consuming the target's LAST hidden state (single aux tap) that runs K AR draft steps; `EagleProposer` is shared with eagle3/mtp/dspark via `use_eagle()`. Separate draft weights; the eagle-draft `max_position_embeddings` clamp is already ported (`SpeculativeConfig::MaybeOverrideDraftMaxPositionEmbeddings`) ahead of the loader. Value #3. | T2 | `vllm/v1/spec_decode/eagle.py:10`; `use_eagle()` `vllm/config/speculative.py:1324-1328`; models `vllm/model_executor/models/{llama_eagle,mistral_eagle,cohere_eagle}.py` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-MTP-FAMILY` | MTP method breadth beyond Qwen3.5 (`SPEC-MTP` DONE): the canonical `mtp` method dispatched per family by `draft_model_config.hf_config.model_type`. DeepSeek-V4 nextn is already ACTIVE-W1 (weight-blocked GGUF, model row `MODEL-SPEC-deepseek-v4-deep-seek-v4-mtp`); the ~18 other family heads (glm4_moe / ernie / nemotron_h / longcat_flash / bailing_hybrid / exaone / mimo / hy_v3 / minimax_m3 / inkling / openpangu / qwen3_next / gemma4 / step3p5) are model-matrix-INVENTORIED. gemma4_mtp / step3p5_mtp take dedicated proposers. | T2 | `vllm/config/speculative.py:37-59` (MTPModelTypes); dispatch `vllm/v1/worker/gpu_model_runner.py:627-637`; models `vllm/model_executor/models/*_mtp.py` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-ACCEPT-VARIANTS` | Acceptance / draft-sampling variants beyond `SPEC-REJECTION`'s greedy `standard`: `rejection_sample_method` ∈ {standard, synthetic, block} and `draft_sample_method` ∈ {greedy, probabilistic} (typical / rejection acceptance for non-greedy sampling). Cross-cuts every proposer. | T2 | `vllm/config/speculative.py:77-78,216,283`; MRV2 `vllm/v1/worker/gpu/spec_decode/rejection_sampler.py:82-91` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-DYNAMIC` | Dynamic speculation length (`num_speculative_tokens_per_batch_size`): a batch-size -> k schedule that adapts draft length to load (`uses_dynamic_speculative_decoding()`). A config axis layered on top of any drafter. | T2 | `vllm/config/speculative.py:1336-1337`; `vllm/v1/spec_decode/dynamic/utils.py:7,77` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-CUSTOM-CLASS` | Pluggable custom proposer (`method="custom_class"`): the proposer class path is read from `speculative_config.model` and must expose a `propose` method — serves the extensibility-first priority (drop-in third-party speculators). | T2 | `vllm/v1/spec_decode/custom_class_proposer.py:12`; dispatch `vllm/v1/worker/gpu_model_runner.py:596-599` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-EXTRACT-HIDDEN` | Hidden-state extraction proposer (`method="extract_hidden_states"`): an offline / data-collection path that runs the target with aux hidden-state outputs (for training EAGLE/MTP drafts) rather than accelerating decode; model row `MODEL-SPEC-extract-hidden-states-extract-hidden-states-model`. | T2 | `vllm/v1/spec_decode/extract_hidden_states.py:29`; dispatch `vllm/v1/worker/gpu_model_runner.py:646-650` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
+| `SPEC-MLP-SPECULATOR` | MLP speculator (`method="mlp_speculator"`): a small MLP head predicting K tokens. UPSTREAM-DEPRECATED at the pin — the enum still resolves it but the V1 dispatch has NO branch, so it raises `ValueError` (`gpu_model_runner.py:651-654`); V0-only. Rowed for enumeration completeness, lowest priority. | T2 | `vllm/model_executor/models/mlp_speculator.py`; enum `vllm/config/speculative.py:69,890` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
 
 ## Serving surface, CLI, and library