fix(ci): DSR-ALLOW the server TU's profiler guards, and unbreak the evidence map - #194
Conversation
…vidence map Two fixes, both for gates that were red or unusable on main. 1) check-device-leakage was RED: DSR bucket vt_ifdef 37 > baseline 32. Cause is mechanical, not a real leak. #189 moved the server body out of examples/server/main.cpp, which the scanner never looked at, into src/vllm/entrypoints/openai/server_main.cpp, which IS the device-agnostic shared layer, carrying its 5 `#ifdef VT_BENCH_PROFILE_CONTROL` sites with it. Not one line of that code changed; it crossed the scanner's boundary. Exempted per-site with the checker's own `// DSR-ALLOW(<row-id>):` hatch -- comments only under src/, no behavior change. Per-site rather than a per-file ALLOWLIST budget because a budget can be spent on something else: swap one profiler guard for a real device fork and a count of 5 still reads 5. The baseline is NOT raised; DSR returns to 32 == 32 and the 5 exemptions print in CI output every run, so the exception budget stays visible. The genuine repair is to put the profiler seam behind a Platform capability query like every other device fork. That is a change to #189's TU, not to this gate, and is not attempted here. 2) check-pr-size could never accept a change to check-device-leakage.py. recognized_evidence derives tests/scripts/test_check_<name>.py, but that suite predates the convention and ci.yml runs it as tests/scripts/test_device_leakage.py, so every change to that checker failed demanding a file that does not exist. Added to CHECKER_EVIDENCE_OVERRIDES, which exists for exactly this. An audit of all 30 checkers found one more: check-dsv4-gguf-namemap.py has no suite at all and is not in ci.yml -- named in a KNOWN_UNTESTED set so the gap is visible in a test rather than invisible in a naming rule. The docs this change owes are real, not filler. USAGE: --cuda-profile-graph-replays and --cuda-profile-graph-batch are parsed by the server and were undocumented. They are the flags that DRIVE the guarded code this change annotates, so they belong here. BENCHMARKS: a "No number owed" row in the same shape as the 2026-08-08 async-readback DSR row directly above it. STATUS said "All six record checkers now green on main", which #189 made false. Corrected to the current state. It costs chars against a byte-tight ratchet, so it is paid for by collapsing the env-doc hygiene sentence beside it: that sentence spelled out which of two env vars went to ENVIRONMENT.md and which to the allowlist, and both files state that authoritatively. Net -111, and the ratchet is re-pinned byte-tight to 244015 to match the convention #190 set it with, rather than quietly keeping the slack as headroom. The evidence test for that re-pin bounds the slack in both directions, so a future red ratchet cannot be cleared by inflating the number instead of shrinking the page. Mutations, bytecode disabled: - downgrade one DSR-ALLOW to a plain comment -> per-site test fails "line 50 has no DSR-ALLOW on it or directly above it" AND the ratchet fails 33 > 32 - remove the evidence override -> "Lists differ: ['scripts/check-device-leakage.py -> ...'] != []" - ratchet 248000 -> FAILED (3); 243000 -> FAILED (4); 244015 -> OK Gates: check-device-leakage OK (32 == 32), check-agent-record, check-public-doc-tables, check-env-doc, check-state-order, check-now-current, check-role-discipline, check-readme-structure, check-policy, check-protocol-consistency; test_device_leakage 26/26, test_check_public_doc_tables 51/51, test_check_pr_size 24/24. Pinned at dc2139b. Only comments change under src/. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude-Code:claude-opus-5 [Claude Code]
e827056 to
3a9f059
Compare
|
Took the DSR-ALLOW route as directed. The PR now fixes the red gate itself, not just the mapping that made it unfixable.
Per-site rather than a per-file The docs this triggered are real, not filler. Touching
The STATUS correction costs chars against #190's byte-tight ratchet, so it is paid for by collapsing the env-doc sentence beside it — that sentence spelled out which of two env vars went to Mutations (bytecode disabled):
Still not attempted, and still the right long-term fix: put the profiler seam behind a |
The bug
check-pr-sizerequires a checker change to ship mutation evidence in a companion test, and derives that path astests/scripts/test_check_<name>.py.check-device-leakage.py's suite predates that convention — it lives attests/scripts/test_device_leakage.py, andci.ymlruns it under that name.So the derived path pointed at a file that does not exist, and every change to that checker failed with "requires semantic mutation evidence in
tests/scripts/test_check_device_leakage.py". A checker nothing could modify.Not hypothetical:
check-device-leakageis red on main right now —vt_ifdef 37 > baseline 32, because #189 moved the server body into the shared layer carrying its 5#ifdef VT_BENCH_PROFILE_CONTROLsites — and theALLOWLISTrepair its own error message recommends is exactly what this blocked.Fixed via
CHECKER_EVIDENCE_OVERRIDES, which exists for this and already carries four entries.The audit
All 30 checkers, one more gap:
check-dsv4-gguf-namemap.pyhas no test suite at all and is not inci.yml. Inventing a suite for it here would be speculative, so it is named in aKNOWN_UNTESTEDset — visible in a test rather than invisible in a naming rule, and explicitly not papered over.The new test asserts every
scripts/check-*.pymaps to an evidence file that exists, so this class cannot recur silently.Mutation: removing the override gives
Lists differ: ['scripts/check-device-leakage.py -> tests/scripts/test_check_device_leakage.py'] != [].What this does NOT do, and needs your ruling
It does not fix the DSR failure itself. Both available routes need a decision that isn't a CI repair:
// DSR-ALLOWhatchsrc/, socheck-doc-checkpointdemands STATUS + BENCHMARKS + USAGE updates for a comment-only change — anddocs/STATUS.mdhas 3 chars of ratchet headroomALLOWLISTcheck-device-leakage.py, whichcheck-pr-sizerejects with "has no affected POL rule mapping" — no POL rule inpolicy.csvnames that checker, so binding it inCREATED_CHECKER_RULESasserts a governance mappingI verified the DSR-ALLOW route works (DSR back to 32 == 32, baseline untouched, exemptions printed every run) and had it passing with a per-site test pinning that every exempted
#ifdef VT_in that TU isVT_BENCH_PROFILE_CONTROL— a per-file count of 5 would still read 5 if someone swapped one guard for a real device fork. I pulled it back out rather than manufacture three doc edits to get a comment past a gate.Say which route you want and I'll land it. The genuine repair, separately, is to put the profiler seam behind a
Platformcapability query like every other device fork — a change to #189's TU, not to the gate.Gates
check-agent-record,check-public-doc-tables,check-env-doc,check-state-order,check-now-current,check-role-discipline,check-readme-structure,check-policy,check-protocol-consistency, andtest_check_pr_size24/24, on a worktree pinned atdc2139b3. No product code touched.check-device-leakagestays red — this does not claim to fix it.Please squash-merge.