feat(abi): embeddings/pooling through the ONE surface — LlamaModel arch, PoolingRunner in the engine step, vllm_embed (ABI v15), live /v1/embeddings

.agents/NOW.md

@@ -30,7 +30,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | `BACKEND-ROCM` W0 | Skeleton in; **HIP never compiled** (no AMD HW) | #41 contributors build it; a compile error IS the deliverable |
 | TP spike #287 (PR #143) | **LANDED** ([spec](specs/tensor-parallelism-spike.md)); DSpark rider grounded | dispatch TP-W1 (CPU-able) |
 | Release | SPIKE; 30/30 | #129 |
-| Surface coverage (`ARCH-ONE-SURFACE`) | **ROW 8 LANDED; #139 repair CPU-GREEN**: ABI v14 stable; registry-resolved named platform; DSR 39→32; execution guard 52/52 | Fresh re-review #139; CUDA A/B residual |
+| Surface coverage (`ARCH-ONE-SURFACE`) | ROW 8 + #139 IN; **ROW 6 IN REVIEW (#137): embeddings LIVE — `LlamaModel` arch, PoolingRunner in the step, `vllm_embed` v15, `/v1/embeddings`, fold gate 4/4-231, 9 kills** | Merge #137; real-ckpt oracle cosine residual |
 
 In-flight (default-OFF, not pushed): `laguna-fp4proj-prod`, laguna
 bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.
@@ -46,10 +46,8 @@ both gate models, reproduced 2–3x on an idle box. See [gates.md](gates.md) and
 
 ## Next actions
 
-1. **Spike the Parakeet encoder row.** Upstream vLLM has `parakeet.py` +
-   `conformer_encoder.py` as the audio encoder of `nano_nemotron_vl.py`, which we
-   already carry `MODEL-MM-nano-nemotron-vl-*` rows for, so it is owed mirror work.
-   The transducer decode half (RNN-T/TDT/CTC) is NOT in vLLM: separate scope call.
+1. **Spike the Parakeet encoder row** (vLLM carries it inside
+   `nano_nemotron_vl.py`; the transducer half is NOT in vLLM: separate call).
 2. **Qwen3.5-4B serving follow-up:** bind the default-ON async-serving path
    against the same oracle before attributing the remaining TPOT gap.
 2. **Merge the invocation-parity prevention** (CI guard + AGENTS.md checklist);

.agents/coordination.md

@@ -1619,6 +1619,8 @@ items a-runner/b stay with the async/GDN `runner.cpp` owners.
 | `CLAIM-POOLING` | `ENG-POOLER-SEQ` (INVENTORIED-implicit→ACTIVE, W1→**W2**), `ENG-POOLING-RUNNER` (**NEW row, ACTIVE, W3**), `SERVE-POOLING-ENDPOINTS` (INVENTORIED→SPIKE) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/claim-pooling-w2w3` (CPU build `build-cpu` `-DVLLM_CPP_CUDA=OFF -DVLLM_CPP_SERVER=ON` Release + CPU run; NO dgx/GPU — the pooling reductions + activations + runner are host arithmetic) | branch `claim-pooling-w2w3`, base `main` `edf68c91` (confirmed via `git rev-parse HEAD`) | Pooling task class HIGH-priority feature-gap #2. W0 spike + W1 CPU pooler OP (prior pass); **W2 pooler HEADS composite + `SequencePooler`/`DispatchPooler` + `PoolerConfig`/`PoolingParams` and W3 pooling RUNNER path (this pass).** Owns ONLY: NEW `include/vllm/model_executor/layers/pooler/{pooling_metadata,methods,activations,common,pooling_params,pooler_config,heads,poolers,dispatch_pooler}.h` + `src/vllm/model_executor/layers/pooler/{methods,activations,heads,poolers,dispatch_pooler}.cpp` + NEW `include/vllm/v1/worker/gpu/pool/pooling_runner.h` + `src/vllm/v1/worker/gpu/pool/pooling_runner.cpp`; NEW `tests/vllm/model_executor/layers/pooler/{test_pooler,test_pooler_heads}.cpp` + `tests/vllm/v1/worker/gpu/pool/test_pooling_runner.cpp`; `CMakeLists.txt` (6 source lines) + `tests/CMakeLists.txt` (3 tests); NEW `.agents/specs/pooling-task-class.md`; the `ENG-POOLER-SEQ` + NEW `ENG-POOLING-RUNNER` engine-matrix rows + `SERVE-POOLING-ENDPOINTS` note + engine Serving/Total rollup (Serving 21→22/ACTIVE 6→7, Total 130→131/ACTIVE 47→48) + `scripts/check-agent-record.py` ENGINE 130→131; the record surfaces (this row, `roadmap_v1.md` gap #2, `docs/STATUS.md`, `docs/BENCHMARKS.md`, `feature-matrix.md` MODEL-POOLING note, `parity-ledger.md`, `state.md`). **NON-COLLISION:** additive NEW files only — the sole edits to existing compiled headers are ADDITIVE (methods.h defaulted virtuals, pooling_metadata.h new fields); ZERO edits to any existing production forward/runner path; NO pooling MODEL row created (concrete embedding model + real-oracle cosine gate is the named W3-model residual), so README/Metal/model-matrix rows untouched. | `ACTIVE` | 2026-07-29 — **W2 + W3 LANDED + CPU-GATED (foreground, NOT pushed).** `test_pooler_heads` 27/27 (240 asserts, Embedding/Classifier heads + SequencePooler + DispatchPooler incl. mixed embed+classify batch + ctor validation) and `test_pooling_runner` 5/5 (14 asserts, runner path + STRUCTURAL cosine-parity gate vs double-precision LAST+normalize ref) — plus W1 `test_pooler` 17/17 unchanged. RED-first proven: disable matryoshka slice + logit_mean → 8 cases/50 asserts fail (heads); CLS-instead-of-LAST drops cosine <0.5 + disable normalize → 2 unit-L2 asserts fail (runner). Clean CPU `-Wall -Wextra -Werror` 0-warn full-library build. **HONEST RESIDUAL:** the cosine gate is STRUCTURAL (synthetic weights) — the real-model `vllm.LLM(task="embed").encode` oracle cosine gate needs a registered concrete embedding model forward (W3-model, no number fabricated). Residuals (spec §Work breakdown): concrete pooling MODEL + real-oracle cosine gate (W3-model), endpoints /v1/embeddings+score+rerank+classify (W4), tokwise AllPool/StepPool (W5). Prior 2026-07-28 — W0 spike + W1 pooler OP LANDED + CPU-GATED: `test_pooler` 17/17 (50 asserts) vs double-precision refs, RED-first proven. |
 | `CLAIM-DSV4-GGUF-LOADER` | `QUANT-GGUF-IQ2_XXS` (INVENTORIED→ACTIVE), `QUANT-GGUF-Q2_K` (INVENTORIED→ACTIVE); cross-refs `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` (stays `SPIKE`, owned by `CLAIM-DEEPSEEK-V4-IMPL`) | Claude Code (opus-4-8) | isolated worktree `.claude/worktrees/gguf-iquant-dsv4` (CPU-only `build-cpu` `-DVLLM_CPP_CUDA=OFF`; NO GPU, NO 90 GB download — the dequant unit gate uses known packed bytes; the GGUF header was HTTP-range-read, no download) | branch `feat/gguf-iquant-dsv4`, base `main` `4d1be010` (confirmed via `git rev-parse HEAD`) | GGUF IQ2_XXS + Q2_K dequant, the DeepSeek-V4-Flash single-Spark GGUF quant-path brick (W1). Owns ONLY the GGUF/quant PATH (NOT the forward — the forward TUs stay owned by `CLAIM-DEEPSEEK-V4-*`): the two `dequantize_row_*` decoders + grid/sign tables in `src/vt/cpu/cpu_quant_dequant.cpp`; the `kQ2_K`/`kIQ2_XXS` vt block dtype registration in `src/vt/dtype.{h,cpp}` + `src/vt/ops.cpp`; the id-16 reader trait in `gguf_reader.cpp` + ids-10/16 dispatch in `gguf_dequant.cpp`; `tests/vllm/test_gguf_dequant.cpp` + `tests/vt/test_ops_quant_traits.cpp`; the two `QUANT-GGUF-*` rows; NEW `.agents/specs/gguf-iquant-dsv4.md`; the V4 GGUF-loadable note on the model-matrix V4 row (row stays SPIKE); the record surfaces. **NON-COLLISION:** additive within the existing GGUF dequant switch + vt block table — does NOT touch any DeepSeek-V4 forward TU (`deepseek_v4.{cpp,h}`/`_dsa`/`_weights`), README, or Metal; the k-quant/NVFP4 decoders are byte-unchanged. | `ACTIVE` | 2026-07-29 — **W1 LANDED + CPU-GATED (foreground, NOT pushed).** IQ2_XXS (id 16, codebook `iq2xxs_grid`+signs+4-bit scale) + Q2_K (id 10, nibble sub-scale/min) ported 1:1 from llama.cpp `ggml-quants.c` `237ad9b96`; both DEQUANT-ONLY (no vec_dot ⇒ route to expand-bf16). `test_gguf_dequant` **15/15·480** (hand-derived literals: IQ2_XXS grid[1] byte0=0x2b→5.375, ksigns[1] flips j=0,7→±3.0, db 0.125/0.375; Q2_K 5.75/-0.25/2.5/0.25) + `test_ops_quant_traits` **9/9·5643** (dequant-only contract). All 7 changed TUs clean under full `-Werror`; the `voxtral.cpp` GCC-13 `-Werror=array-bounds` FP PROVEN pre-existing (fails at base with this diff's `dtype.h` reverted), neutralized only to link the test binaries. **W2 (V4-GGUF loader) DERIVED not landed:** HTTP-range-read the real `UD-IQ2_XXS` header — `general.architecture=deepseek4`, `general.file_type=19` (=IQ2_XXS), `split.tensors.count=1328`, full `deepseek4.*` config-KV schema; the tensor NAME manifest is beyond the CDN range cap + uncached ⇒ the V4 registry GGUF reject STAYS. Residuals: V4 forward (W3-W8, multi-Spark) + the V4-GGUF name map (W2, manifest-blocked) + a vec_dot perf leaf. |
 
+| `CLAIM-EMBEDDINGS-ONE-SURFACE` | `ENG-POOLING-RUNNER` (live engine-step invocation), `SERVE-POOLING-ENDPOINTS` (SPIKE→ACTIVE, `/v1/embeddings`), `MODEL-EMBED-llama-llama-for-causal-lm` (INVENTORIED→ACTIVE; PARTIAL on merge — only the `LlamaModel` membership registered); cross-refs `ENG-POOLER-SEQ` (stays `CLAIM-POOLING`, ops untouched) | Claude Code (fable-5) helper, task #285 | isolated worktree `/home/mudler/_git/vllm.cpp-embeddings-one-surface` (CPU-only; lean per-target builds under disk pressure) | branch `row/EMBEDDINGS-ONE-SURFACE`, base `main` `b44ad337`, DRAFT PR #137 (the reservation) | ARCH-ONE-SURFACE fold ROW 6: embeddings/pooling through the ONE surface. Owns: NEW `src/vllm/model_executor/models/llama_embedding_registry.cpp` + `LoadLlamaModelEmbeddingWeights` (llama_weights.cpp) + `Qwen3DenseModel::ForwardHidden` (qwen3.{h,cpp} additive tail); the ADDITIVE task-gated pooling plumb (`LoadedModel::pooler()`, runner `pooling_runner_`+`pool_tokens`, `Request/EngineCoreRequest::pooling_params`, `ModelRunnerOutput::pooler_output`, scheduler pooling stop, `EngineCoreOutput/RequestOutput::pooling_output`, `LLMEngine::add_pooling_request/embed`, `ResolveAsyncEnabled(is_pooling_model)`); `vllm_embed`/`vllm_embedding_result_free` ABI v15 (vllm.h + vllm_c.cpp incl. the refuse-both-directions guards + the v13 `vllm_complete_tokens` missing-guard fix); `handle_embeddings` + `set_embedder` + task-conditional route (api_server.{h,cpp}) + server main pooling dispatch; NEW fixture `tests/vllm/models/fixtures/llama_embed_e2e` + `scripts/mm/llama_embed_fixture_gen.py` + `tests/vllm/models/test_llama_embedding_fold.cpp`; test/guard updates (test_capi v15 section + floor pin >= 15, test_dlopen symbols, c_header_compile.c, test_api_server embeddings section, test_model_registry/gguf arch pins, check-supported-models ARCH_TOKEN_RE); allowlist row removal + FEATURES/STATUS/BENCHMARKS rows + matrices + specs. **NON-COLLISION:** every engine hook is task-gated on `is_pooling_model`/`pooling_params` (nullopt/false = byte-identical text path); no SACRED path rewritten; no example added. | `ACTIVE` | 2026-08-08 — CPU-LANDED on the branch: fold gate `test_llama_embedding_fold` 4/4-231 (engine path == direct registry path + f64 LAST+normalize ref + chunked is_valid arm), `test_capi` 48/48-462, `test_dlopen` 30/30, server suite 50/50, registry 24/24-820, engine suites green (scheduler 423, llm_engine 204, engine_core 44, output_processor 77, qwen3_forward 1557, async_llm 342, llama_forward 509); 9 mutation kills (floor pin, refuse both directions, route gating both ways, engine-step invocation, scheduler stop, registry info pin, async-off wire). RESIDUAL: real embedding checkpoint + `LLM(task="embed")` oracle cosine. |
+
 
 <!-- claim-view:begin -->
 <!-- claim-view:generated 2026-08-04 -->