Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .agents/NOW.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# NOW — the one-Read resume surface

<!-- now-updated: 2026-08-07 -->
<!-- now-updated: 2026-08-08 -->

Read this FIRST, every session. A SNAPSHOT, rewritten in place: what is live,
the gate being chased, what to do next. Never a log — evidence lives in the
Expand Down Expand Up @@ -28,7 +28,7 @@ Working head: `row/backend-rocm-w0` (#41). Prior: benchmark checkpoint
| Supported-models list | **LANDED**: FEATURES arch table CI-bound (33 archs) | — |
| `/v1/videos` OpenAI shape | **MERGED** (#71): Sora `model`/`size`/`seconds` + `GET /{id}/content` | `row/SERVE-VIDEOS-REFS` PR open: reference conditioning |
| `BACKEND-ROCM` W0 | Skeleton in; **HIP never compiled** (no AMD HW) | #41 contributors build it; a compile error IS the deliverable |
| Surface coverage (`ARCH-ONE-SURFACE`) | **ROW 8 IN REVIEW (`row/DEVICE-KNOB`)**: ABI v14 `device` knob, absent-cuda fails loud, `--device` on server+cli; **#135 H3 device-seam repair IN** (ABI 0/1 via `DeviceType`, DSR 32) | Merge after review; CUDA A/B |
| Surface coverage (`ARCH-ONE-SURFACE`) | **ROW 8 LANDED; #139 repair CPU-GREEN**: ABI v14 stable; registry-resolved named platform; DSR 39→32; execution guard 52/52 | Fresh re-review #139; CUDA A/B residual |

In-flight (default-OFF, not pushed): `laguna-fp4proj-prod`, laguna
bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.
Expand Down
13 changes: 13 additions & 0 deletions .agents/coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -1670,6 +1670,19 @@ this claim will meet. The tiled row is speed-gatable on dgx.

## Closing and archival

- 2026-08-08: `CLAIM-ARCH-ONE-SURFACE-DEVICE-LEAKAGE-V2` CLOSED on draft PR
#139. Base `b44ad337` inherited PR #136's seven shared-layer CUDA literals
(DSR 39 vs baseline 32). The repair keeps ABI-v14 0/1/2 and the public
`auto`/`cpu`/`cuda` names, represents slot 2 internally as a named-platform
selection, resolves its canonical name through `FindPlatformByName`, and
propagates the registry's `DeviceType` without a backend-specific shared
branch. RED-first compiler failure pinned the new seam; the non-CUDA `kXPU`
resolver assertion kills a hidden constant return. GREEN: DSR 32
(`kcuda=0`) with baseline/allowlist unchanged; checker mutations 25/25;
CPU Release `-Werror` `test_platform` 11/11·85,
`test_loaded_engine_dense` 9/9·65, `test_capi` 45/45·428. H3 source was
untouched; no CUDA execution, download, benchmark or service change.

A block closes only when every row in its declared scope is `DONE`. In the same
change:

Expand Down
1 change: 1 addition & 0 deletions .agents/parity-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -926,3 +926,4 @@ Columns:
| 2026-08-07 (`row/H3-ENC-BF16-COND-DIFF`; `ROAD-V1-H3`; model `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit`; MEASUREMENT, lifecycle unchanged) | **MiniMax-H3 - THE NUMBER: what quantizing the TEXT ENCODER to Q4_K_M does to the conditioning.** Same prompt (wuxia, 233 tokens), same tokenizer, same 50-layer truncation, same `MiniMaxH3EncoderTextForwardDevice`, same f32 activations - only the weight bytes differ (Q4_K_M ggml blocks vs the original bf16 14-shard release); both arms self-report IDENTICAL geometry (50/5120/64/8/128/25600), which is what establishes they are the same model. Conditioning `[233, 5120]` f32 via `--encoder-only --save-embeds`. | Not a vLLM-parity change: H3 is BEYOND-PIN (vllm-omni), and vLLM-Omni serves NO quantized H3 at all (BF16-only), so there is no upstream arm to compare against - the bf16 release IS the reference here, and it is the one the loader added in `6d454b00` makes runnable. The quantization-sensitivity premise is ComfyUI PR 15298 (H3's partial split-half RoPE produces channel-wise magnitude outliers that corrupt even INT8), and the measurement CONFIRMS its mechanism concretely: token 0 is an attention sink at norm 15,522 vs a 366 mean (42x), carrying 68% of the total squared error with its DIRECTION intact (cos 0.99962). | **MEASURED on Thor sm_110, build `d1085374` (built and measured as `d1085374`, amended for the row-branch trailer; IDENTICAL tree `dd9283cf`, so the measurement binary IS this commit), GPU idle.** Q4_K_M vs bf16: max|diff| 154.0, RMS 0.5045, rel RMS **0.03403** (0.06849 excluding the sink token), per-token cosine min 0.90916 / mean **0.99745** / median 0.99810, rotation median 3.535 deg / max 24.61 deg, 232 of 233 tokens below cosine 0.999. NOT a scale change: norm ratio mean 0.99010 but the best global rescale only moves 0.03403 -> 0.03280, so it is DIRECTIONAL. CALIBRATION arm (bf16 encoder, ONE-WORD prompt edit `at night`->`at dawn`, also 233 tokens): rel RMS 0.01897 / 0.06666 excl. sink, cosine mean 0.99769 median 0.99963, 172 of 233 tokens above 0.999. So quantization moves the conditioning by the SAME total energy as rewriting a word of the prompt (6.85% vs 6.67%) but with the opposite SHAPE - diffuse over every token instead of concentrated on the words that changed. Cost: Q4 arm 40 s / 18.0 GiB peak; bf16 arm 40 s / 45.41 GiB uploaded / host conversion peak 0.0195 MiB / 51.95 GiB total peak, streamer counters `layers=50 tensors=400 direct=350 converted=200 fused=100` proving the shard path ran. `benchmark_binding=false` (no throughput claim). EXPLICITLY NOT ESTABLISHED: that the RENDER changes - nothing here measures the DiT's sensitivity to a 3.5-degree median rotation; the owed follow-up is a same-DiT/same-seed render A/B over the two saved embeds. |
| 2026-08-08 (`row/ARCH-ONE-SURFACE`; H3 ABI-v12 device dispatch follow-up; lifecycle unchanged) | **MiniMax-H3 video engine device selection is backend-parameterized.** The stable public selector remains 0=CPU / 1=CUDA, but shared code maps it once to `vt::DeviceType`, creates one queue through `GetBackend(device_type)`, and uses that queue's device instead of naming CUDA twice. | vLLM-Omni pipeline ownership remains unchanged; this is a vllm.cpp C-ABI/backend-seam correction with no upstream behavioral delta. | RED: DSR 34 (`kcuda=2`) vs baseline 32. GREEN: DSR 32 with baseline/allowlist unchanged; checker mutations 25/25. New fold unit pins 0/1 and invalid selectors; CPU compile/test NOT RUN locally because the shared filesystem reached 100%, pending GitHub CI. No GPU, download, or performance claim. |
| 2026-08-08 (`row/ARCH-ONE-SURFACE`; H3 device-dispatch mutation-gate follow-up; lifecycle unchanged) | **Pins the single-queue and device-provenance contract through the real `MiniMaxH3VideoEngine::Load` path.** Adds a read-only engine `device()` query and a counting CUDA-backend fixture whose queue reports a distinctive CPU:7 device. | No vLLM-Omni behavior changes; this is test observability for the vllm.cpp backend seam above. | Reviewer mutation reproduced first: replacing `im.device = stream_queue.device` with a second `CreateQueue().device` remained GREEN at 5 cases / 135 assertions. RED-first test then reported `2 == 1`; restoring reuse is GREEN at 6/137. Independent mutation deleting queue-device provenance fails the CPU:7 assertion. DSR remains 32 and all 25 leakage-checker mutations pass. No GPU, download, generation, or performance claim. |
| 2026-08-08 (`row/ARCH-ONE-SURFACE-DEVICE-LEAKAGE-V2`; `ARCH-ONE-SURFACE` ROW 8 follow-up; PR #139; CPU-only; lifecycle unchanged) | **Removes PR #136's seven shared CUDA literals without changing ABI-v14 device selection.** Wire 0/1/2 and public `auto`/`cpu`/`cuda` stay exact; internal slot 2 becomes a named-platform tag, `FindPlatformByName` resolves the canonical name, and shared loading propagates the registered `DeviceType`. Explicit CPU, absent-CUDA-before-I/O and CAPI slot 2 are preserved; H3 is untouched. | vLLM `config/device.py:13,61-66` @ `555967922`: supported device name assigned verbatim and never silently substituted. The registry lookup is the in-tree Platform/Backend portability seam; no upstream behavior delta. | RED: inherited DSR 39 (`kcuda=7`) and focused compile failures for the new enum/API/signature. Mutation pin: resolver input `kXPU` must return `kXPU`. GREEN: DSR 32 (`kcuda=0`) with baseline/allowlist unchanged; checker suite 25/25; CPU Release `-Werror` `test_platform` 11/11·85, `test_loaded_engine_dense` 9/9·65, `test_capi` 45/45·428. No CUDA runtime or performance claim; GPU A/B remains residual. |
58 changes: 57 additions & 1 deletion .agents/specs/one-surface-abi.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ONE SURFACE — every capability ships through the C ABI

Row: `ARCH-ONE-SURFACE`. Status: **AUDIT DONE; remediation IN PROGRESS — ROW 1 (Parakeet ASR / audio transcription) LANDED 2026-08-07: ABI v11 `vllm_transcribe`, live `/v1/audio/transcriptions`, registry refuse-by-task, example folded, ratchet 12 -> 11. ROW 2 (MiniMax-H3 video+audio generation) LANDED 2026-08-08 (`row/H3-VIDEO-ABI`, task #283): ABI v12 `vllm_video_engine_load`/`vllm_video_generate`/`vllm_video_result_free` + `vllm_video_mux_argv` over the `MiniMaxH3VideoEngine` library seam, `/v1/videos` routed through the SAME seam, both H3 examples rewritten as `vllm.h` clients byte-identical to the pre-fold binary, ratchet 11 -> 9. GB10 real-video re-verification is a NAMED RESIDUAL (box on the Kimi campaign). ROW 8 (explicit device selection) LANDED 2026-08-08 (`row/DEVICE-KNOB`, task #284): ABI v14 `vllm_model_params.device` (0=auto/1=cpu/2=cuda, the vLLM `DeviceConfig.device` names) -> `EngineParams::device` -> `SelectQueue`; explicit cpu forces the CPU queue without probing, an explicitly named ABSENT device fails LOUD before any model I/O (the vllm/config/device.py:61-66 never-substitute mirror), `--device` on server + cli as pure field consumers; the #123 review's three MINOR findings closed in the same change (c_header_compile.c now references the v11+v12 surface + the v14 field; the v12 changelog block moved to chronological position; the fold fixture's flag list gained `--keep-quant`). CUDA-build A/B (auto->CUDA vs explicit-cpu->CPU on a GPU box) is a NAMED RESIDUAL — the CPU tier pins that half through the pure `ResolveExplicitDeviceType` matrix instead.**
Row: `ARCH-ONE-SURFACE`. Status: **AUDIT DONE; remediation IN PROGRESS — ROW 1 (Parakeet ASR / audio transcription) LANDED 2026-08-07: ABI v11 `vllm_transcribe`, live `/v1/audio/transcriptions`, registry refuse-by-task, example folded, ratchet 12 -> 11. ROW 2 (MiniMax-H3 video+audio generation) LANDED 2026-08-08 (`row/H3-VIDEO-ABI`, task #283): ABI v12 `vllm_video_engine_load`/`vllm_video_generate`/`vllm_video_result_free` + `vllm_video_mux_argv` over the `MiniMaxH3VideoEngine` library seam, `/v1/videos` routed through the SAME seam, both H3 examples rewritten as `vllm.h` clients byte-identical to the pre-fold binary, ratchet 11 -> 9. GB10 real-video re-verification is a NAMED RESIDUAL (box on the Kimi campaign). ROW 8 (explicit device selection) LANDED 2026-08-08 (`row/DEVICE-KNOB`, task #284): ABI v14 `vllm_model_params.device` (0=auto/1=cpu/2=cuda, the vLLM `DeviceConfig.device` names) -> `EngineParams::device` -> `SelectQueue`; explicit cpu forces the CPU queue without probing, an explicitly named ABSENT device fails LOUD before any model I/O (the vllm/config/device.py:61-66 never-substitute mirror), `--device` on server + cli as pure field consumers; PR #139 follow-up removes PR #136's seven shared CUDA literals by resolving the stable public name through the platform registry and propagating its `DeviceType` (DSR 39 -> 32, `kcuda=0`, baseline/allowlist unchanged). CUDA-build A/B (auto->CUDA vs explicit-cpu->CPU on a GPU box) is a NAMED RESIDUAL — the CPU tier pins that half through the pure `ResolveExplicitDeviceType` matrix instead.**

## The defect

Expand Down Expand Up @@ -202,3 +202,59 @@ must report that exact device after exactly one `CreateQueue` call. The original
second-queue mutation was false-GREEN at 5/135, then RED at `2 == 1`; deleting
the queue-device reuse is independently RED. The final CPU fold target is GREEN
at 6 cases / 137 assertions in the isolated `/dev/shm` build.

### ROW 8 shared-device follow-up (`row/ARCH-ONE-SURFACE-DEVICE-LEAKAGE-V2`, PR #139)

PR #136 implemented ABI-v14 explicit selection correctly but encoded its CUDA
identity seven times in shared configuration/loading code, regressing DSR from
32 to 39. The repair preserves the wire values and public names exactly:
`Device::kNamedPlatform` remains integer 2 and `DeviceName()` remains `"cuda"`.
Shared loading now passes that canonical name to `FindPlatformByName`, then
propagates the registered platform's `DeviceType`; no CUDA `DeviceType` literal
remains in the shared selector. This is a real abstraction rather than textual
evasion: the pure resolver test supplies `kXPU` and requires `kXPU` back, while
an isolated integration target registers a distinctive XPU-shaped platform and
backend, requires exact canonical-name lookup (rejecting prefix and malformed
near-matches), and proves the production selector propagates that platform type
into the created queue. Explicit CPU still ignores the accelerator lookup,
absent CUDA still throws before model path I/O, the C ABI still maps slot 2,
and H3 dispatch is untouched.

RED was the focused compiler failure for the missing enum/API/signature plus
the inherited checker result (`kcuda=7`, DSR 39). GREEN is DSR 32 with
`kcuda=0`, unchanged baseline/allowlist, all 25 checker mutations, and a CPU
Release `-Werror` build: `test_platform` 11/11·85,
`test_device_selection` 2/2·11, `test_loaded_engine_dense` 9/9·65 and
`test_capi` 45/45·428. Each of the three review mutations (wrong returned
platform identity, forced-CPU selector input and prefix name matching) is RED.
The standalone selector target is non-vacuous under
`scripts/check-test-registration.py`: a disposable CPU-only CMake configure and
File-API codemodel query prove that the target exists with its exact source,
then the checker selects the Release configuration, materializes that
configuration's disposable artifact, and requires `ctest -C Release` JSON to
resolve the enabled test to the exact one-argument executable command. This
works for single- and multi-config generators. The workflow guard structurally
normalizes quoted/spaced YAML mapping keys and accepts only the dedicated direct
argv block owned by a job/step with no `if`, `continue-on-error`, or custom
`shell`. Preflight is not inferred from loop text: the checker executes a
disposable copy with instrumented `python3`/`git` shims and requires the checker
and suite argv to occur exactly once, so empty bodies, `continue`, an outer false
branch, and an unset array are all red.

The 52/52 suite covers the earlier registration deaths plus disabled CTest,
non-gating Actions variants, and the preflight execution variants. Its mutation
inventory is protected by a third, independent layer in the production checker:
the canonical path and content SHA-256 are pinned there, as are the named
integrity method and wrapper-body contracts. Deleting or renaming M18 in both the
suite and manifest, redirecting the suite to a byte-identical manifest, or
deleting the integrity test is therefore red. A distinct direct test passes the
unchanged canonical suite source with a byte-identical alternate manifest path
and requires only the exact path-specific error; deleting the production path
guard now makes that test red. The production-owned AST contract separately
requires M42 to retain `assertEqual(errors, [the exact path diagnostic])`, so
deleting or replacing that outcome assertion cannot be masked by M42's
independent byte-identity assertion. This is precisely a claim of
resistance to tandem two-layer suite/manifest shrinkage; it does not claim to
resist an arbitrary simultaneous rewrite of the suite, manifest, and production
checker.
No CUDA runtime was used; the existing GPU A/B residual remains.
2 changes: 1 addition & 1 deletion .agents/specs/surface-coverage-2026-08-07.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ lanes are leaves of `ARCH-ONE-SURFACE` (do not open parallel rows).
| 5 | Kimi-Linear incremental | expose the incremental decode path through the runner/engine (the recompute forward already routes) | rewrite `kimi_linear_gen` | S–M | `KimiDecodeCache` on the runner |
| 6 | Embeddings/pooling | `vllm_embed`/pooling entry point + live `/v1/embeddings`; register a pooling arch (`is_pooling_model=true`); invoke `PoolingRunner` in the step | — | M | pooler live-wiring |
| 7 | Multimodal input | multimodal-content entry point on `vllm_chat`; run the vision/audio tower in the engine step (`mm_features`→`ModelForwardInput.mm`) | wire `chat_mm` seam into the ABI | L | `MM-SERVE-E2E` engine mm-forward residual |
| 8 | Device-selection knob | **DONE (ROW 8, 2026-08-08, `row/DEVICE-KNOB`)**: `vllm_model_params.device` (ABI v14: 0=auto/1=cpu/2=cuda, the vLLM `DeviceConfig.device` names, device.py:13) → `EngineParams::device` → `SelectQueue` explicit arms via `LoadedEngine::ResolveExplicitDeviceType`; explicit cpu never probes, explicit ABSENT cuda fails LOUD before any model I/O (device.py:61-66 never-substitute mirror); `--device` on server + cli | **DONE**: both thin clients consume the field; zero value byte-identical (auto probe) | S | mirror vLLM `--device`/`DeviceConfig` |
| 8 | Device-selection knob | **DONE (ROW 8, 2026-08-08, `row/DEVICE-KNOB`; leakage follow-up PR #139)**: `vllm_model_params.device` (ABI v14: 0=auto/1=cpu/2=cuda, the vLLM `DeviceConfig.device` names, device.py:13) → `EngineParams::device` → `SelectQueue`; the stable public name resolves through `FindPlatformByName` and its registered `DeviceType` is propagated without a shared CUDA literal; explicit cpu never probes, explicit ABSENT cuda fails LOUD before model I/O; DSR 32 / `kcuda=0` | **DONE**: both thin clients consume the field; zero value byte-identical (auto probe) | S | mirror vLLM `--device`/`DeviceConfig` |
| 9 | Voxtral + audio chat seam | register `VoxtralForConditionalGeneration` + fold `VoxtralGenerateGreedy` into the registry forward; audio-capable chat fn + an engine consumer for `AudioKwargs` mm_features | rewrite tests→clients | M | mirror upstream `voxtral.py:309`, `SupportsTranscription` |
| 10 | Gemma-4 audio e2e | bf16 device audio forward + audio→text merge (residual `gemma4_audio.h:41`); USM log-mel front end | fold into the registered mm forward | M | `MM-SERVE-E2E` |
| 11 | Tokenizer/bench ABI + real-load gate | `vllm_tokenize`/`vllm_detokenize`; token-id/count fields on the stream callback for bench; **gate `vllm_engine_load` on a REAL tiny checkpoint at least once** (today bad-path only, `test_capi.cpp:474`) | rewrite `tokenize`/`bench` as clients | S-M | `/tokenize` route exists (`api_server.cpp:432`) |
Expand Down
Loading
Loading