Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
| **Container images (inventoried)** | `ENG-RELEASE-CONTAINERS`: GHCR images from GitHub Actions; lanes `-cuda`/`-vulkan`/`-cpu`, amd64+arm64 manifests | **No number owed:** record-only. No Dockerfile, workflow, registry package or image exists; the image is the unimplemented `ENG-RELEASE-BINARIES` bundle | n/a |
| **Developer agent entry point (implemented)** | `DOCS-AGENT-PROTOCOL-ENTRYPOINT`: public contribution guide + synchronized, mutation-gated pre-claim intake rule | Rebased documentation/protocol only; benchmark void | n/a |
| **ARCH audit: ABI is text-only** | 4 capabilities (H3 video, Laguna, Kimi-Linear, DeepSeek-V4) reachable only from `examples/`, none registry-backed. No gate asks whether a CONSUMER can reach a capability. Documentation only |
| **DSR fix: server TU profiler guards (2026-08-09)** | **No number owed:** comments only. #189 moved the server body into the shared layer with its 5 `VT_BENCH_PROFILE_CONTROL` guards, taking DSR 32 -> 37; they are `DSR-ALLOW`'d per site, baseline unchanged at 32 |
| **DSR fix: async readback capability (2026-08-08)** | **No number owed**: behavior-neutral (CPU/CUDA async-ON, discrete non-CUDA async-OFF, unchanged); moves a `kCUDA` check onto `Backend`, unblocking red CI on #127/#154/#155 |
| **`ROAD-V1-MEM` M1+M2 (2026-08-08)** | KV auto-sizing CPU brick: `--kv-cache-memory` sizes the pool from a byte budget via the group-aware `KVBytesPerBlock` divisor (ABI v16, CPU-gated). M3 profile run dgx-gated |
| **Record/checker repair 2026-08-07–08** | Restored red record/env gates; made release AST semantic pins Python 3.12/3.13-stable; recorded merged Gemma-4 MoE as known merged-GEMM drift and closed the stale embeddings claim. No runtime/performance change |
Expand Down
14 changes: 6 additions & 8 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1116,16 +1116,14 @@ inner-state-free; `test_model_registry` 24/24. Test-golden sync (2026-07-29):
`test_model_loader_gguf`'s hardcoded supported-architectures string was likewise
stale (predated the breadth-sweep + frontier registrations); synced to include
DeepseekV4/Gemma4/KimiK3/Qwen3VL, `test_model_loader_gguf` 3/3 (was RED on main).
Env-doc hygiene (2026-07-29): two production env vars recent lanes added were
undocumented (left `check-env-doc` RED) — `VLLM_PLUGINS` (user-facing plugin
allowlist) documented in `docs/ENVIRONMENT.md`, `VLLM_GEMMA4_MM_DEBUG` (a
diagnostic print switch) classified on `scripts/env-doc-allowlist.txt`;
Env-doc hygiene (2026-07-29): `VLLM_PLUGINS` documented in
`docs/ENVIRONMENT.md`, `VLLM_GEMMA4_MM_DEBUG` classified kernel-internal;
`check-env-doc` rc=0. And the `check-device-leakage` DSR ratchet (RED from the
W7-device lane): the 8 `kCUDA` op-lookups in `deepseek_v4_device.cpp` (the
DeepSeek-V4 CUDA device-forward resolver TU) are allowlisted with a reason +
a recorded FOLLOW-UP to thread the runner `DeviceType` through them (deferred,
needs a GB10 re-gate — DGX offline); `check-device-leakage` rc=0. All six record
checkers now green on main.
V4 device-forward resolver TU) are allowlisted with a reason + a FOLLOW-UP to thread the runner `DeviceType` through them (deferred,
needs a GB10 re-gate — DGX offline). #189 then moved the server TU into the
shared layer with 5 `VT_BENCH_PROFILE_CONTROL` guards (32→37), DSR-ALLOW'd per
site; baseline held at 32. Record checkers green on main.
The matrix opens with an
architecture-support checklist (a per-architecture status roll-up covering every
engaged model) that a CI checker keeps in lockstep with the detailed rows.
Expand Down
2 changes: 2 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,8 @@ to one built without video support. See
| `--enable-metrics` / `--disable-metrics` | on | Serve the metrics endpoint |
| `--enable-thinking` / `--no-enable-thinking` | off | Set the `enable_thinking` chat-template variable for templates that gate a reasoning block on it (Gemma-4 and friends). Our spelling of vLLM's `--default-chat-template-kwargs enable_thinking` |
| `--verbose`, `-v` | off | Verbose server logging |
| `--cuda-profile-graph-replays N` | `0` (off) | Trace-only diagnostic: arm the CUDA-graph-replay profiler and stop after N replays, printing a pid to signal with `SIGUSR2`. Requires a build with `VT_BENCH_PROFILE_CONTROL` |
| `--cuda-profile-graph-batch N` | `16` when replays are armed | Batch size the profiler traces. Must not exceed `--max-num-seqs` |
| `-h`, `--help` | | Print usage and exit |

For a production deployment, use [LocalAI](https://localai.io), which can embed
Expand Down
5 changes: 5 additions & 0 deletions scripts/check-pr-size.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,11 @@
"scripts/check-role-discipline.py": "tests/scripts/test_check_pr_size.py",
"scripts/check-doc-checkpoint.py": "tests/scripts/test_doc_checkpoint.py",
"scripts/check-protocol-consistency.py": "tests/scripts/test_check_protocol_consistency.py",
# Its suite predates the test_check_<name> convention and CI runs it under
# the older name, so the derived path pointed at a file that does not
# exist and NO change to this checker could ever satisfy its own evidence
# rule. Mapped to the file CI actually runs.
"scripts/check-device-leakage.py": "tests/scripts/test_device_leakage.py",
}

# New entrypoints cannot appear in their own historical policy enforcement
Expand Down
12 changes: 11 additions & 1 deletion scripts/check-public-doc-tables.py
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,17 @@ def features_errors(text: str) -> list[str]:
# milestone updated the OpenAI server status cell. Its endpoint-by-endpoint
# prose already lives in docs/USAGE.md, so the cell now keeps only the
# capability groups and binding limitations. Net -366 from rebased main.
"chars": 244126,
#
# 244015 since 2026-08-09 (measured 244015): #189 moved the server TU into
# the shared layer, which made "All six record checkers now green on main"
# false -- check-device-leakage went 32 -> 37 on 5 VT_BENCH_PROFILE_CONTROL
# guards. Correcting that claim costs chars, so it is paid for by collapsing
# the env-doc hygiene sentence beside it to its binding result: it spelled
# out which of two env vars went to ENVIRONMENT.md and which to the
# allowlist, and both files state that authoritatively. Net -111, and
# re-pinned byte-tight to match the convention this ratchet was set with
# rather than quietly keeping the slack as headroom.
"chars": 244015,
"h2_sections": 11,
"long_paragraphs": 82,
"oversized_cells": 44,
Expand Down
5 changes: 5 additions & 0 deletions src/vllm/entrypoints/openai/server_main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
#include <atomic>
#include <sys/wait.h>
#include <unistd.h>
// DSR-ALLOW(ARCH-ONE-SURFACE): VT_BENCH_PROFILE_CONTROL is a build-option guard for the CUDA-graph-replay profiler, not a device fork; #189 moved it here verbatim from examples/server/main.cpp, which the DSR scanner never covered.
#ifdef VT_BENCH_PROFILE_CONTROL
#include <cerrno>
#include <chrono>
Expand Down Expand Up @@ -89,6 +90,7 @@
#include "vllm/v1/kv_offload/kv_connector.h"
#include "vllm/v1/worker/gpu/runner.h"
#include "vt/backend.h"
// DSR-ALLOW(ARCH-ONE-SURFACE): VT_BENCH_PROFILE_CONTROL is a build-option guard for the CUDA-graph-replay profiler, not a device fork; #189 moved it here verbatim from examples/server/main.cpp, which the DSR scanner never covered.
#ifdef VT_BENCH_PROFILE_CONTROL
#include "vt/cuda/cuda_profiler_control.h"
#endif
Expand Down Expand Up @@ -690,6 +692,7 @@ int VllmServerMain(int argc, char** argv) {
const vllm::tok::Tokenizer& tokenizer = loaded->tokenizer();

if (args.cuda_profile_graph_replays > 0) {
// DSR-ALLOW(ARCH-ONE-SURFACE): VT_BENCH_PROFILE_CONTROL is a build-option guard for the CUDA-graph-replay profiler, not a device fork; #189 moved it here verbatim from examples/server/main.cpp, which the DSR scanner never covered.
#ifdef VT_BENCH_PROFILE_CONTROL
vt::cuda::ConfigureCudaGraphReplayProfiler(
static_cast<uint32_t>(args.cuda_profile_graph_replays),
Expand Down Expand Up @@ -938,6 +941,7 @@ int VllmServerMain(int argc, char** argv) {
}
std::cerr << ")\n";

// DSR-ALLOW(ARCH-ONE-SURFACE): VT_BENCH_PROFILE_CONTROL is a build-option guard for the CUDA-graph-replay profiler, not a device fork; #189 moved it here verbatim from examples/server/main.cpp, which the DSR scanner never covered.
#ifdef VT_BENCH_PROFILE_CONTROL
std::atomic<bool> benchmark_shutdown_waiter_ready{false};
std::atomic<bool> benchmark_shutdown_received{false};
Expand Down Expand Up @@ -1020,6 +1024,7 @@ int VllmServerMain(int argc, char** argv) {

const bool listen_ok = server.listen(args.host, args.port);

// DSR-ALLOW(ARCH-ONE-SURFACE): VT_BENCH_PROFILE_CONTROL is a build-option guard for the CUDA-graph-replay profiler, not a device fork; #189 moved it here verbatim from examples/server/main.cpp, which the DSR scanner never covered.
#ifdef VT_BENCH_PROFILE_CONTROL
if (benchmark_shutdown_thread.joinable()) {
benchmark_shutdown_cancelled.store(true, std::memory_order_release);
Expand Down
25 changes: 25 additions & 0 deletions tests/scripts/test_check_pr_size.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,31 @@
SPEC.loader.exec_module(checker)


class CheckerEvidenceMapping(unittest.TestCase):
# A checker whose evidence path does not exist can never satisfy the
# governance_checker rule: every change to it fails with "requires semantic
# mutation evidence in <file that is not there>". That is how
# check-device-leakage.py became unmodifiable -- its suite predates the
# test_check_<name> convention, so the derived path missed by one word.
KNOWN_UNTESTED = {
# No suite at all, and not wired into ci.yml either. Named here so the
# gap is visible in a test rather than invisible in a naming rule.
"scripts/check-dsv4-gguf-namemap.py",
}

def test_every_checker_maps_to_an_evidence_file_that_exists(self) -> None:
root = Path(checker.__file__).resolve().parents[1]
missing = []
for path in sorted(root.glob("scripts/check-*.py")):
rel = f"scripts/{path.name}"
if rel in self.KNOWN_UNTESTED:
continue
evidence = checker.recognized_evidence(rel)
if not (root / evidence).is_file():
missing.append(f"{rel} -> {evidence}")
self.assertEqual(missing, [])


class PathClassification(unittest.TestCase):
def test_each_mutable_surface_has_an_explicit_class(self) -> None:
expected = {
Expand Down
12 changes: 12 additions & 0 deletions tests/scripts/test_check_public_doc_tables.py
Original file line number Diff line number Diff line change
Expand Up @@ -429,6 +429,18 @@ def test_the_live_page_is_inside_its_ratchet(self) -> None:
doc_tables.status_errors(doc_tables.STATUS.read_text(encoding="utf-8")), []
)

def test_the_char_ratchet_is_not_slack(self) -> None:
"""Headroom is not a growth budget.

The cheap way out of a red char ratchet is to RAISE the number instead
of shrinking the page, and the gate then measures nothing. Bounding the
slack means clearing it still costs a real collapse.
"""
live = len(doc_tables.STATUS.read_text(encoding="utf-8"))
slack = doc_tables.STATUS_RATCHET["chars"] - live
self.assertGreaterEqual(slack, 0, "the live page is already over its ratchet")
self.assertLessEqual(slack, 2000, "ratchet headroom is cover for bloat")

def test_the_live_page_keeps_the_character_ratchet_tight(self) -> None:
text = doc_tables.STATUS.read_text(encoding="utf-8")
slack = doc_tables.STATUS_RATCHET["chars"] - len(text)
Expand Down
24 changes: 24 additions & 0 deletions tests/scripts/test_device_leakage.py
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,30 @@ def test_baseline_matches_the_tree_exactly(self) -> None:
rc = dl.main([])
self.assertEqual(rc, 0, out.getvalue() + err.getvalue())

def test_server_main_dsr_allows_cover_only_the_profiler_guard(self) -> None:
# #189 moved the server body into the shared layer, bringing 5
# `#ifdef VT_BENCH_PROFILE_CONTROL` sites with it and pushing vt_ifdef
# 32 -> 37. They are exempted per-site with DSR-ALLOW rather than by a
# per-file budget, because a budget can be spent on something else: swap
# one guard for a real device fork and the count still reads 5. This pins
# that every exempted guard in that TU is the profiler one.
path = ROOT / "src/vllm/entrypoints/openai/server_main.cpp"
lines = path.read_text(encoding="utf-8").splitlines()
guarded = [
index
for index, line in enumerate(lines)
if line.lstrip().startswith("#ifdef VT_")
or line.lstrip().startswith("#if defined(VT_")
]
self.assertTrue(guarded)
for index in guarded:
self.assertIn("VT_BENCH_PROFILE_CONTROL", lines[index])
self.assertTrue(
dl.RE_DSR_ALLOW.search(lines[index])
or (index and dl.RE_DSR_ALLOW.search(lines[index - 1])),
f"line {index + 1} has no DSR-ALLOW on it or directly above it",
)

def test_every_allowlisted_path_exists(self) -> None:
# A stale allowlist entry is a silent exemption for a file that may later
# be recreated with different contents.
Expand Down
Loading