mudler · localai-bot · Aug 7, 2026 · Aug 6, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -26,6 +26,7 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): async mirror on classic dense Qwen3; SACRED 184/184 | Residual: sibling scope one-liner |
 | CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Op-dispatch profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn**. **G5 not next** | Parakeet encoder; attn dtype hoist |
 | Supported-models list (`row/DOCS-SUPPORTED-MODELS-MATRIX`) | **DRAFT PR**: FEATURES per-arch table CI-bound to registry (30 archs) | Reviewer merge |
+| `/v1/videos` OpenAI shape (`row/SERVE-VIDEOS-OAI`) | **PR open**: Sora `model`/`size`/`seconds` + `GET /{id}/content`, CPU-gated | Follow-up row: reference conditioning |
 
 In-flight (default-OFF, not pushed): `laguna-fp4proj-prod`, laguna
 bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.
@@ -47,29 +48,26 @@ throughput ⇒ audit the context; per-shape MEASUREMENT arbitrates).
    `conformer_encoder.py` as the audio encoder of `nano_nemotron_vl.py`, which we
    already carry `MODEL-MM-nano-nemotron-vl-*` rows for, so it is owed mirror work.
    The transducer decode half (RNN-T/TDT/CTC) is NOT in vLLM: separate scope call.
-2. **Qwen3.5-4B serving follow-up:** the synchronous 0.9971x harness remains
-   speed-pending; bind the default-ON async-serving path against the same oracle
-   before attributing the remaining TPOT gap.
+2. **Qwen3.5-4B serving follow-up:** bind the default-ON async-serving path
+   against the same oracle before attributing the remaining TPOT gap.
 2. **Merge the invocation-parity prevention** (CI guard + AGENTS.md checklist);
    CUDA build-verify the byte-exact `kGemvHeuristicAlgos` refactor on dgx.
 3. **Same-tool re-verify deepseek_v4's bf16 resident tower** (the one other
    f32-out caller) once the Laguna fix proves the mechanism.
 4. **Restore `local-ai-worker`** on dgx when the GPU campaign ends
    (`docker update --restart=always` + `docker start`).
-5. **Protocol substrate — partly done.** Claim triage + live-state audit DONE
-   (10 unevidenced rows → `READY`, 11 claims retired, 9 amended); `STATUS.md`
-   ratcheted; `AGENTS.md` tiered. REMAINING: anchor backfill
-   (6 model rows need a DECISION); record-era rollover BLOCKED on `DONE` rows
-   bound to `parity-ledger.md` LINE anchors (re-anchor by ROW ID).
-   ★ The gate SELF-BLINDS on those same 10 (audit §➁a); its fix owes an 8-row
-   adjudication. workflow.md states the `ACTIVE` precondition.
+5. **Protocol substrate — partly done.** Triage/audit, `STATUS.md` ratchet and
+   the `AGENTS.md` tiering are DONE. REMAINING: anchor backfill (6 model rows
+   need a DECISION); record-era rollover BLOCKED on `DONE` rows bound to
+   `parity-ledger.md` LINE anchors (re-anchor by ROW ID). ★ The gate SELF-BLINDS
+   on the 10 audited rows (audit §➁a); its fix owes an 8-row adjudication.
 
 **Operator/helper protocol**
 ([spec](specs/operator-helper-protocol.md)): roles DECLARED then MATERIALIZED
 into a lock or worktree+PR; operator merges PRs first and does features only via
 sub-agents; helpers use worktrees on `row/<ROW-ID>` and open a DRAFT PR at the
 START, which IS the claim. **W0-W5 LANDED**; role discipline ENFORCING,
-`--require-role` is the DEFAULT. Queue: 10 rows — 6 are audit-vacated, with LANDED gate anchors; READ before picking. Backfill: 79 rows, 30 anchored; blocker is claim FAMILIES.
+`--require-role` is the DEFAULT. Queue: 10 rows (6 audit-vacated, LANDED gate anchors; READ before picking). Backfill: 79 rows, 30 anchored; blocker is claim FAMILIES.
 **Upstream inventory** ([spec](specs/upstream-derived-inventory-2026-08-05.md),
 drift-gated, arch parity BOTH ways): SM060/061/070 below vLLM's floor =
 OUT-OF-SCOPE; COMP-*/DISTRIBUTED-* are REAL unported work; **all 362 archs now have rows**; llama.cpp's 11 extra devices are IN SCOPE, spike-gated

diff --git a/.agents/coordination.md b/.agents/coordination.md
@@ -1628,6 +1628,7 @@ binding. Correctness (memcmp byte-identity) IS gatable on x86 and is the bar
 this claim will meet. The tiled row is speed-gatable on dgx.
 
 | `CLAIM-KERNEL-CPU-ELEM-WIDE-1` | `KERNEL-GEMM-CPU-ELEM-X86WIDE` + `KERNEL-GEMM-CPU-TILED` | Claude Code (opus-5) | worktree `/home/mudler/_git/vllm.cpp-x86wide`, base `cbc56f12` | branch `row/KERNEL-GEMM-CPU-ELEM-X86WIDE` | see ownership above | `SPIKE` | 2026-08-06 spike committed; W4 (widen the test tier sweep) is the first implementation step, deliberately before any kernel |
+| `CLAIM-SERVE-VIDEOS-OAI` | `SERVE-VIDEOS-OAI` (NEW engine-matrix row; the MiniMax-H3 model rows and the `row/H3-FP4-SPEED` fp4 lane are UNTOUCHED) | Claude Opus 5 (1M context) sub-agent | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a152dd219c723cb12` (CPU-only; no GPU, no download; foreground) | branch `row/SERVE-VIDEOS-OAI`, base `main` `add5e9f9` | `/v1/videos` in OpenAI's Sora WIRE SHAPE (`model`, `size`, `seconds`) + `GET /v1/videos/{id}/content`. Owns `{include,src}/vllm/entrypoints/openai/video_api.*`, the video handlers/routes in `api_server.{h,cpp}`, both video test files, `specs/minimax-h3.md` §9 and the record surfaces. ADDITIVE: no generation kernel, no model forward, no VAE and no non-video route touched; the routes still register only with a `VideoRunner`. SPLIT: reference conditioning is a stacked follow-up row (spec §10), which is why this one carries no `examples/server/main.cpp` change. | `ACTIVE` | 2026-08-06 CPU-LANDED + gated (`test_video_api` 11/11-125, `test_openai_api_server` 40/40-509, `server` -Werror clean). Real-weights leg rides the H3 GB10/disk window. |
 
 ## Handoff queue
 

diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md
@@ -42,11 +42,11 @@ forensics: roadmap_v1.md and the parity ledger.
 | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 3 | 0 | 1 | 5 |
 | Structured output and tools | 7 | 0 | 3 | 0 | 0 | 2 | 0 | 0 | 2 |
 | Speculative decoding | 21 | 0 | 0 | 1 | 0 | 4 | 0 | 4 | 11 |
-| Serving, API, CLI, library | 22 | 10 | 2 | 1 | 0 | 2 | 2 | 1 | 4 |
+| Serving, API, CLI, library | 23 | 10 | 2 | 1 | 0 | 3 | 2 | 1 | 4 |
 | LoRA and adapters | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
 | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
 | Loading, tokenizer, config | 9 | 3 | 3 | 0 | 0 | 0 | 1 | 1 | 1 |
-| **Total** | **140** | **36** | **16** | **4** | **7** | **18** | **8** | **9** | **41** |
+| **Total** | **141** | **36** | **16** | **4** | **7** | **19** | **8** | **9** | **41** |
 
 ## Engine core and scheduling
 
@@ -213,6 +213,7 @@ claims it.
 | `ENG-POOLING-RUNNER` | The pooling RUNNER path — where the generation runner SAMPLES a token, the pooling runner applies the model's `Pooler` to the last hidden state and returns the POOLED DATA (embedding vector / classification logit row). **W3 LANDED + CPU-GATED 2026-07-29 (`CLAIM-POOLING`, NOT pushed):** `PoolingRunner` over a packed `[num_tokens, hidden]` last-hidden-state buffer + a `PoolingMetadata` — `Pool()` delegates to the model pooler (`DispatchPooler.ForEmbedding`), `GetSupportedTasks()`, `ComputeValid()` (`seq_lens==prompt_len`). GATE: a STRUCTURAL cosine-parity gate — the runner's embedding vs an independent double-precision LAST+normalize reference is cosine≈1 (5 cases / 14 asserts), RED-first (CLS-instead-of-LAST drops cosine <0.5; disable normalize → 2 unit-L2 asserts fail). GENERALIZATION DEVIATION: upstream `pooling_runner.py` hardcodes LAST+normalize; we route through the model `Pooler` (the general bert.py path), strictly more capable. HONEST RESIDUAL (named): the REAL-model oracle cosine gate (`vllm.LLM(task="embed").encode`) needs a registered concrete embedding model's forward — no such model is registered yet (W3-model), so no cosine-vs-oracle number is fabricated. The InputBatch→PoolingMetadata construction (logits_indices gather) rides the endpoint brick (W4) | T2 | `vllm/v1/worker/gpu/pool/pooling_runner.py:18-46`; `vllm/tasks.py:10`; `tests/models/language/pooling/test_embedding.py` (real-oracle gate, DEFERRED) | `include/vllm/v1/worker/gpu/pool/pooling_runner.h` + `src/vllm/v1/worker/gpu/pool/pooling_runner.cpp` — anchor `src/vllm/v1/worker/gpu/pool/pooling_runner.cpp:11` | `tests/vllm/v1/worker/gpu/pool/test_pooling_runner.cpp` (runner path + structural cosine gate, 14 asserts, RED-first) — anchor `tests/vllm/v1/worker/gpu/pool/test_pooling_runner.cpp:136` | [pooling-task-class.md](specs/pooling-task-class.md) | `ACTIVE` | `CLAIM-POOLING` |
 | `SERVE-RESPONSES-MESSAGES` | Responses, Anthropic messages, audio | T2 | `vllm/entrypoints/openai/responses/api_router.py:48`; `vllm/entrypoints/anthropic/api_router.py:49`; `vllm/entrypoints/speech_to_text/transcription/api_router.py:1` | - | - | `planned: specs/responses-messages-endpoints.md` | `INVENTORIED` | - |
 | `SERVE-ADMIN` | Abort-requests, sleep, pause/resume, profiling, RL weight updates. **`/abort_requests` LANDED + CPU-GATED 2026-07-28 (`CLAIM-C8-SERVE-ENDPOINTS`, NOT pushed):** `POST /abort_requests` (from the dev/rlhf admin router) parses `{request_ids:[...]}` and aborts exactly those (external) ids via an injected abort callback wired to the engine abort path (`AsyncLLM::abort`); an empty/missing list means "abort all in-flight" (the callback decides). Response `{"status":"aborted","aborted":<count>}`; malformed JSON → 400 `{"detail":"Invalid JSON format"}`; abort failure → 500 `{"error":...}` — all three shapes mirror the upstream router verbatim. ADDITIVE + opt-in (route registered only when the abort callback is attached → 404 otherwise). **PRODUCTION `main.cpp` WIRING LANDED + CPU-GATED 2026-07-28 (`CLAIM-C8-SERVE-PROD-WIRING`, NOT pushed):** the shipped `vllm-server` binary now wires `/abort_requests` to the LIVE `AsyncLLM::abort` through the shared `ConfigureUtilityEndpoints` seam, DEV-mode gated behind the new `--enable-server-dev-mode` flag — mirroring vLLM registering the dev/rlhf router only under `if envs.VLLM_SERVER_DEV_MODE` (api_server.py:238; envs.py:157 default 0). Explicit-id abort tears the request down and reports the exact drop in unfinished requests (before−after); empty `request_ids` (abort-ALL) reports 0 — NAMED RESIDUAL (AsyncLLM exposes no active-request-id accessor). RESIDUAL: the abort-ALL enumeration (missing `AsyncLLM::active_request_ids()`); `/sleep`/`/wake_up`/`/is_sleeping`, `/pause`/`/resume`, `/start_profile`/`/stop_profile`, weight-update/EP endpoints still INVENTORIED | T2/T3 | `vllm/entrypoints/serve/dev/rlhf/api_router.py:94-138` (abort_requests); dev-mode gate `vllm/entrypoints/openai/api_server.py:238-240`, `vllm/entrypoints/serve/__init__.py:35`, `vllm/envs.py:157`; `vllm/entrypoints/serve/dev/sleep/api_router.py:21`; `vllm/entrypoints/serve/dev/rlhf/api_router.py:29,74,136`; `vllm/entrypoints/serve/profile/api_router.py:21` | handler `src/vllm/entrypoints/openai/api_server.cpp:488` (`handle_abort_requests`); opt-in setter `include/vllm/entrypoints/openai/api_server.h:156` (`set_abort_requests`); production seam `src/vllm/entrypoints/openai/api_server.cpp` (`ConfigureUtilityEndpoints`, before/after delta-count) + `examples/server/main.cpp` (`--enable-server-dev-mode`); engine abort path `include/vllm/v1/engine/async_llm.h:115` (`abort`) | `tests/vllm/entrypoints/openai/test_api_server.cpp:1104` (shape + callback wiring: explicit ids passthrough, empty→abort-all branch, malformed→400),`:1143` (aborts an in-flight AsyncLLM request → `has_unfinished_requests()` false),`:1250` (opt-in route gate: 404 no-callback → 200 attached, RED-first),`:1319` (**production seam: dev-mode gate 404→200, live abort exact delta-count==1, empty→0**) — in the 32/32 / 420-assertion suite | [admin-endpoints.md](specs/admin-endpoints.md) | `ANCHOR-BACKFILL` | `CLAIM-C8-SERVE-PROD-WIRING` |
+| `SERVE-VIDEOS-OAI` | `/v1/videos` in OpenAI's Sora WIRE SHAPE, over the vLLM-Omni-derived job endpoints. **CPU-LANDED + GATED 2026-08-06 (`CLAIM-SERVE-VIDEOS-OAI`):** the OpenAI request spellings (`model`, `size` "WxH", `seconds` as a number OR the string enum OpenAI actually types) parse as ALIASES onto the existing native members, NATIVE-wins precedence applied PER-AXIS, both spellings validated either way so a malformed alias is a 400 even when overridden; an unserved `model` is a job `warning` echoed for the job's whole life, never a rejection (a Sora client cannot know the local model's name); and `GET /v1/videos/{id}/content` serves the finished MP4 (404 unknown / 409 unfinished / 500 failed / 500 vanished), without which a caller could start and poll a job but never fetch the result over HTTP. All four routes still register ONLY with a `VideoRunner` attached, now gated over a REAL socket. RESIDUALS (named): OpenAI's status vocabulary/id shape is not mirrored; reference conditioning (`input_reference`, the `metadata` video/audio references) is a stacked follow-up row; the real-weights leg rides the H3 GB10/disk window. | T2 | OpenAI Sora video API (`POST /v1/videos`, `GET /v1/videos/{video_id}/content`); vLLM-Omni `vllm/entrypoints/openai/video/api_router.py` (the async/sync job pair we already mirror) | `include/vllm/entrypoints/openai/video_api.h:31`; `src/vllm/entrypoints/openai/video_api.cpp:98`; `src/vllm/entrypoints/openai/api_server.cpp:279` | `tests/vllm/entrypoints/openai/test_video_api.cpp:64`; `tests/vllm/entrypoints/openai/test_api_server.cpp:1751` | [minimax-h3.md §9](specs/minimax-h3.md) | `ACTIVE` | `CLAIM-SERVE-VIDEOS-OAI` |
 | `SERVE-OTLP` | OpenTelemetry traces | T2 | `vllm/config/observability.py:18,36,128` | - | - | `planned: specs/otlp-tracing.md` | `INVENTORIED` | - |
 | `SERVE-BATCH-API` | Offline OpenAI Batch API runner — read a JSONL of `BatchRequestInput` (`custom_id`/`method`/`url`/`body`), dispatch each line to the matching serving handler, collect `BatchRequestOutput` rows (custom_id echoed, per-line error isolation), write the response JSONL. **W0 SPIKE + W1 CPU BRICK LANDED + CPU-GATED 2026-07-29 (`CLAIM-BATCH-API`, NOT pushed):** `RunBatch` (RunLine/RunLines/Run) + `RunBatchFile` (local paths) as a pure ORCHESTRATOR over the existing `OpenAIServingChat::create_chat_completion` (the SAME handler `handle_chat_completions` drives — NO reimplemented generation), 1:1 with vLLM's endpoint_registry url→handler map. `/v1/chat/completions` wired; `BatchResponseData`/`BatchRequestOutput` schema + `vllm-<uuid>`/`vllm-batch-<uuid>` ids; the `run_request` AllResponse/ErrorResponse/stream branches; the unsupported-endpoint (handler None) + unsupported-url error rows. RECORDED DEVIATION: a malformed input line is ISOLATED into an error row (batch continues) where upstream aborts the job (deviation lives in the library; the abort-on-bad-line CLI exit code is a W2 residual). RESIDUALS (named, spec §Work breakdown): the `vllm run-batch` CLI + `BatchFrontendArgs` (W2); embeddings/score/rerank dispatch (W3, rides `SERVE-POOLING-ENDPOINTS`); audio transcription/translation + media fetch (W4); http(s)/data-URL file I/O, metrics server, overlapped `AsyncLLM` submission (W5) | T2 | `vllm/entrypoints/openai/run_batch.py:148-228` (schema),`:508-570` (run_request/make_error),`:722-777,815-847` (dispatch/run loop); `tests/entrypoints/openai/test_run_batch.py:375,402,432` | `include/vllm/entrypoints/openai/run_batch.h`; `src/vllm/entrypoints/openai/run_batch.cpp:70,84,87,142` | `tests/vllm/entrypoints/openai/test_run_batch.cpp:434` (7 cases / 80 assertions, CPU, RED-first: dropping the custom_id echo fails 9 assertions) | [batch-api.md](specs/batch-api.md) | `ANCHOR-BACKFILL` | `CLAIM-BATCH-API` |