mudler · localai-bot · Aug 7, 2026 · Aug 6, 2026
diff --git a/.agents/NOW.md b/.agents/NOW.md
@@ -25,7 +25,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): async mirror on classic dense Qwen3; SACRED 184/184 | Residual: sibling scope one-liner |
 | CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn**. **G5 not next** | Parakeet encoder; attn dtype hoist |
 | Supported-models list (`row/DOCS-SUPPORTED-MODELS-MATRIX`) | **DRAFT PR**: FEATURES per-arch table CI-bound to registry (30 archs) | Reviewer merge |
-| `/v1/videos` OpenAI shape (`row/SERVE-VIDEOS-OAI`) | **PR open**: Sora `model`/`size`/`seconds` + `GET /{id}/content`, CPU-gated | Follow-up row: reference conditioning |
+| `/v1/videos` OpenAI shape | **MERGED** (#71): Sora `model`/`size`/`seconds` + `GET /{id}/content` | `row/SERVE-VIDEOS-REFS` PR open: reference conditioning |
 | `BACKEND-ROCM` W0 | Skeleton in; **HIP never compiled** (no AMD HW) | #41 contributors build it; a compile error IS the deliverable |
 
 In-flight (default-OFF, not pushed): `laguna-fp4proj-prod`, laguna

diff --git a/.agents/coordination.md b/.agents/coordination.md
@@ -1629,7 +1629,8 @@ this claim will meet. The tiled row is speed-gatable on dgx.
 
 | `CLAIM-ROCM-W0-1` | `BACKEND-ROCM` (INVENTORIED→ACTIVE) | Claude Code (opus-5) | worktree `/home/mudler/_git/vllm.cpp-rocm-w0`, base `main` `4cfeee13`; CPU-only Release build, NO GPU of any kind | branches `row/backend-rocm-seam` (PR #87) + `row/backend-rocm-w0` (PR #88, stacked) | Owns ONLY the ROCm skeleton and the platform-SELECTION gate it uncovered: NEW `include/vt/rocm/{rocm_arch,rocm_runtime}.h`, NEW `src/vt/rocm/{rocm_backend,rocm_rmsnorm,rocm_ops}.hip`, NEW `src/vllm/platforms/rocm.cpp`, NEW `tests/vt/{test_rocm_arch,test_rocm_backend}.cpp`, NEW `.agents/specs/rocm-backend-w0.md`; ADDITIVE edits: `include/vt/device.h` (`kROCM`), `include/vllm/platforms/interface.h` + `src/vllm/platforms/platform.cpp` (`CurrentPlatformPriority()` + kROCM in the walk), `tests/vllm/platforms/test_platform.cpp`, `tests/vt/test_backend_cross_device.cpp` (kROCM in the discovery list), `CMakeLists.txt`/`tests/CMakeLists.txt`, this row + the `BACKEND-ROCM` matrix row + README/BUILD/ROCM/STATUS/FEATURES/BENCHMARKS + NOW/state. **NON-COLLISION:** every new file is ROCm-only; the only shared-code edits are one enum entry and one selection-array entry, neither of which changes behaviour for any registered backend (nothing registers kROCM in a non-HIP build). | `ACTIVE` | 2026-08-06 — **W0 LANDED, CPU-GATED, HIP UNBUILT.** The three `.hip` TUs have never been compiled by anyone (no AMD hardware here) — weaker than `build-supported`, and labelled UNBUILT everywhere. Verified: clean `-Werror` build, `ctest` full suite, `test_rocm_arch` 40/40 (upstream's own worked examples), `test_platform` incl. the new walk gate, device-leakage unchanged at 32; the two non-HIP legs are object-compiled in every build as a bit-rot guard. PENDING (hardware): every HIP compile, `test_rocm_backend`, the cross-device RmsNorm-vs-CPU-oracle comparison, and both `BACKEND-GATE-ROCM-*` rows. Next event is a contributor's first `-DVLLM_CPP_HIP=ON` on #41. |
 | `CLAIM-KERNEL-CPU-ELEM-WIDE-1` | `KERNEL-GEMM-CPU-ELEM-X86WIDE` + `KERNEL-GEMM-CPU-TILED` | Claude Code (opus-5) | worktree `/home/mudler/_git/vllm.cpp-x86wide`, base `cbc56f12` | branch `row/KERNEL-GEMM-CPU-ELEM-X86WIDE` | see ownership above | `SPIKE` | 2026-08-06 spike committed; W4 (widen the test tier sweep) is the first implementation step, deliberately before any kernel |
-| `CLAIM-SERVE-VIDEOS-OAI` | `SERVE-VIDEOS-OAI` (NEW engine-matrix row; the MiniMax-H3 model rows and the `row/H3-FP4-SPEED` fp4 lane are UNTOUCHED) | Claude Opus 5 (1M context) sub-agent | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a152dd219c723cb12` (CPU-only; no GPU, no download; foreground) | branch `row/SERVE-VIDEOS-OAI`, base `main` `add5e9f9` | `/v1/videos` in OpenAI's Sora WIRE SHAPE (`model`, `size`, `seconds`) + `GET /v1/videos/{id}/content`. Owns `{include,src}/vllm/entrypoints/openai/video_api.*`, the video handlers/routes in `api_server.{h,cpp}`, both video test files, `specs/minimax-h3.md` §9 and the record surfaces. ADDITIVE: no generation kernel, no model forward, no VAE and no non-video route touched; the routes still register only with a `VideoRunner`. SPLIT: reference conditioning is a stacked follow-up row (spec §10), which is why this one carries no `examples/server/main.cpp` change. | `ACTIVE` | 2026-08-06 CPU-LANDED + gated (`test_video_api` 11/11-125, `test_openai_api_server` 40/40-509, `server` -Werror clean). Real-weights leg rides the H3 GB10/disk window. |
+| `CLAIM-SERVE-VIDEOS-OAI` | `SERVE-VIDEOS-OAI` (NEW engine-matrix row; the MiniMax-H3 model rows and the `row/H3-FP4-SPEED` fp4 lane are UNTOUCHED) | Claude Opus 5 (1M context) sub-agent | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a152dd219c723cb12` (CPU-only; no GPU, no download; foreground) | branch `row/SERVE-VIDEOS-OAI`, base `main` `add5e9f9` | `/v1/videos` in OpenAI's Sora WIRE SHAPE (`model`, `size`, `seconds`) + `GET /v1/videos/{id}/content`. Owns `{include,src}/vllm/entrypoints/openai/video_api.*`, the video handlers/routes in `api_server.{h,cpp}`, both video test files, `specs/minimax-h3.md` §9 and the record surfaces. ADDITIVE: no generation kernel, no model forward, no VAE and no non-video route touched; the routes still register only with a `VideoRunner`. SPLIT: reference conditioning is the stacked row `SERVE-VIDEOS-REFS` (spec §10), which is why this one carries no `examples/server/main.cpp` change. | `ACTIVE` | 2026-08-06 CPU-LANDED + gated (`test_video_api` 11/11-125, `test_openai_api_server` 40/40-509, `server` -Werror clean). Real-weights leg rides the H3 GB10/disk window. |
+| `CLAIM-SERVE-VIDEOS-REFS` | `SERVE-VIDEOS-REFS` (NEW engine-matrix row, stacked on `SERVE-VIDEOS-OAI`) | Claude Opus 5 (1M context) sub-agent | isolated worktree `/home/mudler/_git/vllm.cpp/.claude/worktrees/agent-a152dd219c723cb12` (CPU-only; no GPU, no download; foreground) | branch `row/SERVE-VIDEOS-REFS`, base `row/SERVE-VIDEOS-OAI` | Reference conditioning over `/v1/videos`: `input_reference` -> fl2va first-frame conditioning, the `metadata` video/audio references -> ref2va blocks, the exclusivity rule at the request boundary, and the `examples/server` runner wiring with lazily-loaded VAE encoder halves. Owns the reference members of `{include,src}/vllm/entrypoints/openai/video_api.*`, the video runner in `examples/server/main.cpp`, both video test files, `specs/minimax-h3.md` §10 and the record surfaces. ADDITIVE: no generation kernel and no model forward touched. | `ACTIVE` | 2026-08-06 CPU-LANDED + gated (`test_video_api` 14/14-167, `test_openai_api_server` 41/41-525, `server` -Werror clean). Real-weights leg rides the H3 GB10/disk window. |
 
 ## Handoff queue
 

diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md
@@ -42,11 +42,11 @@ forensics: roadmap_v1.md and the parity ledger.
 | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 3 | 0 | 1 | 5 |
 | Structured output and tools | 7 | 0 | 3 | 0 | 0 | 2 | 0 | 0 | 2 |
 | Speculative decoding | 21 | 0 | 0 | 1 | 0 | 4 | 0 | 4 | 11 |
-| Serving, API, CLI, library | 23 | 10 | 2 | 1 | 0 | 3 | 2 | 1 | 4 |
+| Serving, API, CLI, library | 24 | 10 | 2 | 1 | 0 | 4 | 2 | 1 | 4 |
 | LoRA and adapters | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
 | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
 | Loading, tokenizer, config | 9 | 3 | 3 | 0 | 0 | 0 | 1 | 1 | 1 |
-| **Total** | **141** | **36** | **16** | **4** | **7** | **19** | **8** | **9** | **41** |
+| **Total** | **142** | **36** | **16** | **4** | **7** | **20** | **8** | **9** | **41** |
 
 ## Engine core and scheduling
 
@@ -214,6 +214,7 @@ claims it.
 | `SERVE-RESPONSES-MESSAGES` | Responses, Anthropic messages, audio | T2 | `vllm/entrypoints/openai/responses/api_router.py:48`; `vllm/entrypoints/anthropic/api_router.py:49`; `vllm/entrypoints/speech_to_text/transcription/api_router.py:1` | - | - | `planned: specs/responses-messages-endpoints.md` | `INVENTORIED` | - |
 | `SERVE-ADMIN` | Abort-requests, sleep, pause/resume, profiling, RL weight updates. **`/abort_requests` LANDED + CPU-GATED 2026-07-28 (`CLAIM-C8-SERVE-ENDPOINTS`, NOT pushed):** `POST /abort_requests` (from the dev/rlhf admin router) parses `{request_ids:[...]}` and aborts exactly those (external) ids via an injected abort callback wired to the engine abort path (`AsyncLLM::abort`); an empty/missing list means "abort all in-flight" (the callback decides). Response `{"status":"aborted","aborted":<count>}`; malformed JSON → 400 `{"detail":"Invalid JSON format"}`; abort failure → 500 `{"error":...}` — all three shapes mirror the upstream router verbatim. ADDITIVE + opt-in (route registered only when the abort callback is attached → 404 otherwise). **PRODUCTION `main.cpp` WIRING LANDED + CPU-GATED 2026-07-28 (`CLAIM-C8-SERVE-PROD-WIRING`, NOT pushed):** the shipped `vllm-server` binary now wires `/abort_requests` to the LIVE `AsyncLLM::abort` through the shared `ConfigureUtilityEndpoints` seam, DEV-mode gated behind the new `--enable-server-dev-mode` flag — mirroring vLLM registering the dev/rlhf router only under `if envs.VLLM_SERVER_DEV_MODE` (api_server.py:238; envs.py:157 default 0). Explicit-id abort tears the request down and reports the exact drop in unfinished requests (before−after); empty `request_ids` (abort-ALL) reports 0 — NAMED RESIDUAL (AsyncLLM exposes no active-request-id accessor). RESIDUAL: the abort-ALL enumeration (missing `AsyncLLM::active_request_ids()`); `/sleep`/`/wake_up`/`/is_sleeping`, `/pause`/`/resume`, `/start_profile`/`/stop_profile`, weight-update/EP endpoints still INVENTORIED | T2/T3 | `vllm/entrypoints/serve/dev/rlhf/api_router.py:94-138` (abort_requests); dev-mode gate `vllm/entrypoints/openai/api_server.py:238-240`, `vllm/entrypoints/serve/__init__.py:35`, `vllm/envs.py:157`; `vllm/entrypoints/serve/dev/sleep/api_router.py:21`; `vllm/entrypoints/serve/dev/rlhf/api_router.py:29,74,136`; `vllm/entrypoints/serve/profile/api_router.py:21` | handler `src/vllm/entrypoints/openai/api_server.cpp:488` (`handle_abort_requests`); opt-in setter `include/vllm/entrypoints/openai/api_server.h:156` (`set_abort_requests`); production seam `src/vllm/entrypoints/openai/api_server.cpp` (`ConfigureUtilityEndpoints`, before/after delta-count) + `examples/server/main.cpp` (`--enable-server-dev-mode`); engine abort path `include/vllm/v1/engine/async_llm.h:115` (`abort`) | `tests/vllm/entrypoints/openai/test_api_server.cpp:1104` (shape + callback wiring: explicit ids passthrough, empty→abort-all branch, malformed→400),`:1143` (aborts an in-flight AsyncLLM request → `has_unfinished_requests()` false),`:1250` (opt-in route gate: 404 no-callback → 200 attached, RED-first),`:1319` (**production seam: dev-mode gate 404→200, live abort exact delta-count==1, empty→0**) — in the 32/32 / 420-assertion suite | [admin-endpoints.md](specs/admin-endpoints.md) | `ANCHOR-BACKFILL` | `CLAIM-C8-SERVE-PROD-WIRING` |
 | `SERVE-VIDEOS-OAI` | `/v1/videos` in OpenAI's Sora WIRE SHAPE, over the vLLM-Omni-derived job endpoints. **CPU-LANDED + GATED 2026-08-06 (`CLAIM-SERVE-VIDEOS-OAI`):** the OpenAI request spellings (`model`, `size` "WxH", `seconds` as a number OR the string enum OpenAI actually types) parse as ALIASES onto the existing native members, NATIVE-wins precedence applied PER-AXIS, both spellings validated either way so a malformed alias is a 400 even when overridden; an unserved `model` is a job `warning` echoed for the job's whole life, never a rejection (a Sora client cannot know the local model's name); and `GET /v1/videos/{id}/content` serves the finished MP4 (404 unknown / 409 unfinished / 500 failed / 500 vanished), without which a caller could start and poll a job but never fetch the result over HTTP. All four routes still register ONLY with a `VideoRunner` attached, now gated over a REAL socket. RESIDUALS (named): OpenAI's status vocabulary/id shape is not mirrored; reference conditioning (`input_reference`, the `metadata` video/audio references) is a stacked follow-up row; the real-weights leg rides the H3 GB10/disk window. | T2 | OpenAI Sora video API (`POST /v1/videos`, `GET /v1/videos/{video_id}/content`); vLLM-Omni `vllm/entrypoints/openai/video/api_router.py` (the async/sync job pair we already mirror) | `include/vllm/entrypoints/openai/video_api.h:31`; `src/vllm/entrypoints/openai/video_api.cpp:98`; `src/vllm/entrypoints/openai/api_server.cpp:279` | `tests/vllm/entrypoints/openai/test_video_api.cpp:64`; `tests/vllm/entrypoints/openai/test_api_server.cpp:1751` | [minimax-h3.md §9](specs/minimax-h3.md) | `ACTIVE` | `CLAIM-SERVE-VIDEOS-OAI` |
+| `SERVE-VIDEOS-REFS` | REFERENCE CONDITIONING over `/v1/videos`: the image an OpenAI request starts from, plus the two modalities OpenAI's schema has no slot for. **CPU-LANDED + GATED 2026-08-06 (`CLAIM-SERVE-VIDEOS-REFS`, stacked on `SERVE-VIDEOS-OAI`):** OpenAI's `input_reference` (a filesystem path or an RFC 2397 `data:` URL, decoded by the SAME `DecodeDataUri` the chat multimodal parts use) maps to fl2va FIRST-FRAME conditioning via `MiniMaxH3EncodeKeyframeCondRows`, because OpenAI documents it as the frame the video starts from and ref2va would silently change what the API promises; the silent-video and audio references ride the standard free-form `metadata` map (`input_reference_video`, a DIRECTORY of `frame_%06d.ppm` since no demuxer is vendored; `input_reference_audio`, a 16-bit PCM WAV) and become ref2va blocks, an audio reference ATTACHING to the video block when both are given. fl2va-vs-ref2va exclusivity is enforced in the PARSER, mirroring `minimax_h3_pipeline.cpp:251`, so an illegal pair is a 400 naming it rather than a dropped reference. Both VAE ENCODER halves load lazily and once. RESIDUALS (named): reference images are binary PPM at the output resolution (no PNG/JPEG codec, no resampler vendored); a video reference is a frame directory; OpenAI's real upload is multipart, ours is the JSON spelling. | T2 | OpenAI Sora video API (`input_reference`, `metadata`); exclusivity rule `src/vllm/model_executor/models/minimax_h3_pipeline.cpp:251` | `include/vllm/entrypoints/openai/video_api.h:51`; `src/vllm/entrypoints/openai/video_api.cpp:66`; `examples/server/main.cpp:96` | `tests/vllm/entrypoints/openai/test_video_api.cpp:172`; `tests/vllm/entrypoints/openai/test_api_server.cpp:1827` | [minimax-h3.md §10](specs/minimax-h3.md) | `ACTIVE` | `CLAIM-SERVE-VIDEOS-REFS` |
 | `SERVE-OTLP` | OpenTelemetry traces | T2 | `vllm/config/observability.py:18,36,128` | - | - | `planned: specs/otlp-tracing.md` | `INVENTORIED` | - |
 | `SERVE-BATCH-API` | Offline OpenAI Batch API runner — read a JSONL of `BatchRequestInput` (`custom_id`/`method`/`url`/`body`), dispatch each line to the matching serving handler, collect `BatchRequestOutput` rows (custom_id echoed, per-line error isolation), write the response JSONL. **W0 SPIKE + W1 CPU BRICK LANDED + CPU-GATED 2026-07-29 (`CLAIM-BATCH-API`, NOT pushed):** `RunBatch` (RunLine/RunLines/Run) + `RunBatchFile` (local paths) as a pure ORCHESTRATOR over the existing `OpenAIServingChat::create_chat_completion` (the SAME handler `handle_chat_completions` drives — NO reimplemented generation), 1:1 with vLLM's endpoint_registry url→handler map. `/v1/chat/completions` wired; `BatchResponseData`/`BatchRequestOutput` schema + `vllm-<uuid>`/`vllm-batch-<uuid>` ids; the `run_request` AllResponse/ErrorResponse/stream branches; the unsupported-endpoint (handler None) + unsupported-url error rows. RECORDED DEVIATION: a malformed input line is ISOLATED into an error row (batch continues) where upstream aborts the job (deviation lives in the library; the abort-on-bad-line CLI exit code is a W2 residual). RESIDUALS (named, spec §Work breakdown): the `vllm run-batch` CLI + `BatchFrontendArgs` (W2); embeddings/score/rerank dispatch (W3, rides `SERVE-POOLING-ENDPOINTS`); audio transcription/translation + media fetch (W4); http(s)/data-URL file I/O, metrics server, overlapped `AsyncLLM` submission (W5) | T2 | `vllm/entrypoints/openai/run_batch.py:148-228` (schema),`:508-570` (run_request/make_error),`:722-777,815-847` (dispatch/run loop); `tests/entrypoints/openai/test_run_batch.py:375,402,432` | `include/vllm/entrypoints/openai/run_batch.h`; `src/vllm/entrypoints/openai/run_batch.cpp:70,84,87,142` | `tests/vllm/entrypoints/openai/test_run_batch.cpp:434` (7 cases / 80 assertions, CPU, RED-first: dropping the custom_id echo fails 9 assertions) | [batch-api.md](specs/batch-api.md) | `ANCHOR-BACKFILL` | `CLAIM-BATCH-API` |