diff --git a/.agents/state.md b/.agents/state.md index 32804a6e..7786b03a 100644 --- a/.agents/state.md +++ b/.agents/state.md @@ -40218,3 +40218,44 @@ SIGSEGVs at an UNRELATED CUDA case (line 3503) that PASSES in isolation (585) an the new case — pre-existing cross-test CUDA resource-accumulation flake, not this change. Box left clean (no procs, GPU idle, locks released, worker parked, my artifacts pruned, ckpts kept). Records: spec §8.8 + §8.2, STATUS/BENCHMARKS/FEATURES, benchmark-record, NOW. + +- **2026-08-07** — **PROTOCOL VIOLATION, self-reported: the Vulkan campaign (PR + #80, merged `5397e91d`) split feature commits from record commits, so + `documentation-checkpoint` FAILED for that push range.** + + The gate requires every commit touching a checkpoint path (`src/`, `tests/`, + `scripts/`, `.agents/specs/`, …) to update `docs/STATUS.md` AND + `docs/BENCHMARKS.md` **in that same commit**. Eight commits did not: + `9579f94e`, `ba5ea0cf`, `196ea46f`, `e32c5ed3`, `3bfa1f12`, `34a3efe6`, + `2c86f79e`, `f4738bb8` — each a `feat(vulkan)`/`refactor(vulkan)` touching + `src/` and `tests/`, with its doc update deferred into a following + `record(vulkan)` commit. + + **Substance vs process.** The public docs on `main` ARE current — STATUS, + BENCHMARKS and FEATURES all describe the shipped 16-native-kernel / + 71-on-the-tier state with the llvmpipe-only and no-speed-number caveats intact. + What was violated is the gate's per-COMMIT granularity, which exists so that a + bisect lands on a commit whose docs match its code, and so a half-landed series + cannot leave the public surface describing something that does not exist. + + **Why it was not caught locally.** `scripts/check-doc-checkpoint.py` with no + arguments validates the single committed HEAD, and it was run that way and + passed. The CI job is DIFF-scoped over `before..sha` and checks EVERY commit in + the range independently. The correct local invocation for a series is + `--base origin/main --head HEAD`, which would have caught all eight. + + **Not repaired by rewriting history.** `main` had already moved (#86 landed on + top) and other sessions branch from it, so a force-push to fix eight commit + messages-worth of file grouping would cost more than the defect. The gate is + diff-scoped, so the NEXT push to `main` only covers new commits — no ongoing + red. What is permanently true is that this one range is recorded as + gate-failed, and no later run re-covers it. Recorded rather than quietly left. + + **Rule going forward:** a feature commit carries its own STATUS/BENCHMARKS + update. If the numbers are not yet known, the honest line is "pending/void with + the reason", which is what the gate's own error message asks for — not a + deferred record commit. + + Note also that `agent-record` was ALREADY failing on `main` at `4cfeee13` + before this merge (`check-fusion-consistency`, `minimax_h3_video_vae_device`), + verified on `origin/main` itself. That one is not from this work. diff --git a/.agents/workflow.md b/.agents/workflow.md index 01819784..02695f9e 100644 --- a/.agents/workflow.md +++ b/.agents/workflow.md @@ -101,6 +101,18 @@ Run `scripts/agent-onboard.py --probe` to see what is still unresolved. `python3 tests/scripts/test_doc_checkpoint.py` pass: every feature or iteration checkpoint updates the obligated public surfaces below in the same change, even when the honest result is pending, failed, or void; + - **for a SERIES of commits, additionally + `python3 scripts/check-doc-checkpoint.py --base origin/main --head HEAD`.** + `--staged` inspects the STAGED paths only, so it is VACUOUS when nothing is + staged — running it after `git commit` (the natural order, and what + `agent-preflight.sh` does) checks nothing at all. CI is DIFF-scoped over the + whole pushed range and checks EVERY commit independently, so a branch that + defers its doc update into a following `record(...)` commit passes locally + and fails on `main`, where the range can never be re-covered. This is not + hypothetical: PR #80 (Vulkan) landed eight such commits and reddened + `documentation-checkpoint` for its range (state log, 2026-08-07). A feature + commit carries its OWN STATUS/BENCHMARKS update; if the numbers are not yet + known, write the pending/void line with its reason; - tests green (op-parity / behavioral / e2e as applicable); - every feature/milestone that can plausibly affect speed, latency, scheduling, memory traffic, loading, or peak memory completes its own diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 3e159a97..5d0ab2ad 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -317,7 +317,7 @@ built on it rather than keeping the flattering one. | Memory footprint vs declared workload (`ROAD-V1-MEM`, #83) | **Never measured, and not measurable today**: there is no auto-sizing to compare against, because the KV pool is a hand-typed `--num-blocks`, so "what the run actually needed" has no number | Once M1's `MemoryBudget` lands: predicted-vs-actual bytes per allocation class, then peak footprint ours-auto vs vLLM at its 0.9 default on the same model and config | | Startup latency (cold launch to first `/health`) | **PENDING: no number measured or claimed.** Listed in the gate protocol, never captured (readiness polled every 5 s, duration discarded). Harness landed 2026-08-06. [Detail](../.agents/specs/startup-latency-axis.md) | Run `dgx-online-serving.sh --startup-only --model 27` on dgx (needs a CUDA `server` build; the box was at 100% disk on 2026-08-06), then record ours/vLLM medians and the ratio here | | Speculation depth (`ROAD-V1-D3-SPEC-K`, #81) | **Never measured, MTP is k=1** (our port covers vLLM's k=1 branch only), so no acceptance-vs-depth curve exists | k=2..4 three-way greedy gate, then the c1/c>1 A/B + the per-workload (prose vs code) acceptance-vs-depth curve any dynamic or adaptive depth policy needs | -| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | **NOT APPLICABLE: nothing measured, claimed or owed.** 16 NATIVE kernels; 71 on the host reference tier. opt-125m runs e2e token-exact; no speed measured. [Detail](../.agents/specs/vulkan-full-support.md) | `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, `llama-bench`, same GGUF, three columns (ours-Vulkan, llama.cpp-Vulkan, ours-CUDA). Until then: `GetReferenceTierHits()` must reach 0 | +| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | **NOT APPLICABLE: nothing measured, claimed or owed.** 16 NATIVE kernels; 71 on the host tier. opt-125m runs e2e token-exact on llvmpipe; no speed measured. [Detail](../.agents/specs/vulkan-full-support.md) | `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, `llama-bench`, same GGUF, three columns (ours-Vulkan, llama.cpp-Vulkan, ours-CUDA). Until then: `GetReferenceTierHits()` must reach 0 | | SGLang floor arms | Never ran | Both arms of the SGLang comparison | | cuBLAS invocation-parity guard | CI guard landed (CPU); `kGemvHeuristicAlgos` refactor build-verify owed | `nvcc` rebuild + SACRED gate on dgx | | Ampere consumer (`sm_86`, RTX 3090 class) | **No number owed; no such board here.** 2026-08-06 build-verify: 7/7 FA2 TUs 0-warn, real `sm_86` SASS. [Detail](../.agents/benchmark-record.md) | External RTX 3090 report. Floor is llama.cpp on that card (GGUF, not our Blackwell-only NVFP4 grid) | diff --git a/docs/STATUS.md b/docs/STATUS.md index 0157445a..74960f65 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -394,8 +394,8 @@ The correctness form and the full D0-D14 measured chronology live in ## Not supported yet LoRA (W1 CPU runtime brick landed; not yet usable end-to-end), multi-GPU, -Vulkan (16 native kernels, 71 on the CPU tier; opt-125m -runs e2e token-exact; +Vulkan (16 native kernels, 71 on CPU tier; opt-125m +e2e token-exact, llvmpipe; [campaign](../.agents/specs/vulkan-full-support.md)), and the full tool-calling template surface. **Scale-out / distributed execution is scoped but unbuilt** (2026-07-28): the engine is single-GPU today — no NCCL / tensor-parallel / diff --git a/scripts/agent-preflight.sh b/scripts/agent-preflight.sh index 030054a2..528d05e2 100755 --- a/scripts/agent-preflight.sh +++ b/scripts/agent-preflight.sh @@ -171,6 +171,19 @@ for suite in "${SUITES[@]}"; do run "$suite" python3 "tests/scripts/$suite.py" done +# The COMMITTED range, checked the way CI checks it. Deliberately OUTSIDE the +# --staged block: `--staged` inspects staged paths and is therefore VACUOUS after +# `git commit`, which is when preflight normally runs -- so the obligation went +# unchecked for a whole series and PR #80 landed eight commits that reddened +# documentation-checkpoint on main, where a diff-scoped range is never re-covered. +# Gating this on --staged would reproduce that hole exactly. +if git rev-parse --verify -q origin/main >/dev/null 2>&1 && + [ "$(git rev-list --count origin/main..HEAD 2>/dev/null || echo 0)" -gt 0 ]; then + echo "Committed range vs origin/main:" + run "doc-checkpoint range" python3 scripts/check-doc-checkpoint.py \ + --base origin/main --head HEAD +fi + if [ "$STAGED" -eq 1 ]; then echo "Staged change:" run "doc-checkpoint --staged" python3 scripts/check-doc-checkpoint.py --staged