Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .agents/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -40218,3 +40218,44 @@ SIGSEGVs at an UNRELATED CUDA case (line 3503) that PASSES in isolation (585) an
the new case — pre-existing cross-test CUDA resource-accumulation flake, not this change.
Box left clean (no procs, GPU idle, locks released, worker parked, my artifacts pruned, ckpts
kept). Records: spec §8.8 + §8.2, STATUS/BENCHMARKS/FEATURES, benchmark-record, NOW.

- **2026-08-07** — **PROTOCOL VIOLATION, self-reported: the Vulkan campaign (PR
#80, merged `5397e91d`) split feature commits from record commits, so
`documentation-checkpoint` FAILED for that push range.**

The gate requires every commit touching a checkpoint path (`src/`, `tests/`,
`scripts/`, `.agents/specs/`, …) to update `docs/STATUS.md` AND
`docs/BENCHMARKS.md` **in that same commit**. Eight commits did not:
`9579f94e`, `ba5ea0cf`, `196ea46f`, `e32c5ed3`, `3bfa1f12`, `34a3efe6`,
`2c86f79e`, `f4738bb8` — each a `feat(vulkan)`/`refactor(vulkan)` touching
`src/` and `tests/`, with its doc update deferred into a following
`record(vulkan)` commit.

**Substance vs process.** The public docs on `main` ARE current — STATUS,
BENCHMARKS and FEATURES all describe the shipped 16-native-kernel /
71-on-the-tier state with the llvmpipe-only and no-speed-number caveats intact.
What was violated is the gate's per-COMMIT granularity, which exists so that a
bisect lands on a commit whose docs match its code, and so a half-landed series
cannot leave the public surface describing something that does not exist.

**Why it was not caught locally.** `scripts/check-doc-checkpoint.py` with no
arguments validates the single committed HEAD, and it was run that way and
passed. The CI job is DIFF-scoped over `before..sha` and checks EVERY commit in
the range independently. The correct local invocation for a series is
`--base origin/main --head HEAD`, which would have caught all eight.

**Not repaired by rewriting history.** `main` had already moved (#86 landed on
top) and other sessions branch from it, so a force-push to fix eight commit
messages-worth of file grouping would cost more than the defect. The gate is
diff-scoped, so the NEXT push to `main` only covers new commits — no ongoing
red. What is permanently true is that this one range is recorded as
gate-failed, and no later run re-covers it. Recorded rather than quietly left.

**Rule going forward:** a feature commit carries its own STATUS/BENCHMARKS
update. If the numbers are not yet known, the honest line is "pending/void with
the reason", which is what the gate's own error message asks for — not a
deferred record commit.

Note also that `agent-record` was ALREADY failing on `main` at `4cfeee13`
before this merge (`check-fusion-consistency`, `minimax_h3_video_vae_device`),
verified on `origin/main` itself. That one is not from this work.
12 changes: 12 additions & 0 deletions .agents/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,18 @@ Run `scripts/agent-onboard.py --probe` to see what is still unresolved.
`python3 tests/scripts/test_doc_checkpoint.py` pass: every feature or
iteration checkpoint updates the obligated public surfaces below in the
same change, even when the honest result is pending, failed, or void;
- **for a SERIES of commits, additionally
`python3 scripts/check-doc-checkpoint.py --base origin/main --head HEAD`.**
`--staged` inspects the STAGED paths only, so it is VACUOUS when nothing is
staged — running it after `git commit` (the natural order, and what
`agent-preflight.sh` does) checks nothing at all. CI is DIFF-scoped over the
whole pushed range and checks EVERY commit independently, so a branch that
defers its doc update into a following `record(...)` commit passes locally
and fails on `main`, where the range can never be re-covered. This is not
hypothetical: PR #80 (Vulkan) landed eight such commits and reddened
`documentation-checkpoint` for its range (state log, 2026-08-07). A feature
commit carries its OWN STATUS/BENCHMARKS update; if the numbers are not yet
known, write the pending/void line with its reason;
- tests green (op-parity / behavioral / e2e as applicable);
- every feature/milestone that can plausibly affect speed, latency,
scheduling, memory traffic, loading, or peak memory completes its own
Expand Down
2 changes: 1 addition & 1 deletion docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ built on it rather than keeping the flattering one.
| Memory footprint vs declared workload (`ROAD-V1-MEM`, #83) | **Never measured, and not measurable today**: there is no auto-sizing to compare against, because the KV pool is a hand-typed `--num-blocks`, so "what the run actually needed" has no number | Once M1's `MemoryBudget` lands: predicted-vs-actual bytes per allocation class, then peak footprint ours-auto vs vLLM at its 0.9 default on the same model and config |
| Startup latency (cold launch to first `/health`) | **PENDING: no number measured or claimed.** Listed in the gate protocol, never captured (readiness polled every 5 s, duration discarded). Harness landed 2026-08-06. [Detail](../.agents/specs/startup-latency-axis.md) | Run `dgx-online-serving.sh --startup-only --model 27` on dgx (needs a CUDA `server` build; the box was at 100% disk on 2026-08-06), then record ours/vLLM medians and the ratio here |
| Speculation depth (`ROAD-V1-D3-SPEC-K`, #81) | **Never measured, MTP is k=1** (our port covers vLLM's k=1 branch only), so no acceptance-vs-depth curve exists | k=2..4 three-way greedy gate, then the c1/c>1 A/B + the per-workload (prose vs code) acceptance-vs-depth curve any dynamic or adaptive depth policy needs |
| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | **NOT APPLICABLE: nothing measured, claimed or owed.** 16 NATIVE kernels; 71 on the host reference tier. opt-125m runs e2e token-exact; no speed measured. [Detail](../.agents/specs/vulkan-full-support.md) | `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, `llama-bench`, same GGUF, three columns (ours-Vulkan, llama.cpp-Vulkan, ours-CUDA). Until then: `GetReferenceTierHits()` must reach 0 |
| Vulkan vs llama.cpp Vulkan (`BENCH-VK-LLAMA`) | **NOT APPLICABLE: nothing measured, claimed or owed.** 16 NATIVE kernels; 71 on the host tier. opt-125m runs e2e token-exact on llvmpipe; no speed measured. [Detail](../.agents/specs/vulkan-full-support.md) | `VK-E`: llama.cpp `-DGGML_VULKAN=ON` at `237ad9b96` on dgx, `llama-bench`, same GGUF, three columns (ours-Vulkan, llama.cpp-Vulkan, ours-CUDA). Until then: `GetReferenceTierHits()` must reach 0 |
| SGLang floor arms | Never ran | Both arms of the SGLang comparison |
| cuBLAS invocation-parity guard | CI guard landed (CPU); `kGemvHeuristicAlgos` refactor build-verify owed | `nvcc` rebuild + SACRED gate on dgx |
| Ampere consumer (`sm_86`, RTX 3090 class) | **No number owed; no such board here.** 2026-08-06 build-verify: 7/7 FA2 TUs 0-warn, real `sm_86` SASS. [Detail](../.agents/benchmark-record.md) | External RTX 3090 report. Floor is llama.cpp on that card (GGUF, not our Blackwell-only NVFP4 grid) |
Expand Down
4 changes: 2 additions & 2 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,8 +394,8 @@ The correctness form and the full D0-D14 measured chronology live in
## Not supported yet

LoRA (W1 CPU runtime brick landed; not yet usable end-to-end), multi-GPU,
Vulkan (16 native kernels, 71 on the CPU tier; opt-125m
runs e2e token-exact;
Vulkan (16 native kernels, 71 on CPU tier; opt-125m
e2e token-exact, llvmpipe;
[campaign](../.agents/specs/vulkan-full-support.md)), and the full tool-calling
template surface. **Scale-out / distributed execution is scoped but unbuilt**
(2026-07-28): the engine is single-GPU today — no NCCL / tensor-parallel /
Expand Down
13 changes: 13 additions & 0 deletions scripts/agent-preflight.sh
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,19 @@ for suite in "${SUITES[@]}"; do
run "$suite" python3 "tests/scripts/$suite.py"
done

# The COMMITTED range, checked the way CI checks it. Deliberately OUTSIDE the
# --staged block: `--staged` inspects staged paths and is therefore VACUOUS after
# `git commit`, which is when preflight normally runs -- so the obligation went
# unchecked for a whole series and PR #80 landed eight commits that reddened
# documentation-checkpoint on main, where a diff-scoped range is never re-covered.
# Gating this on --staged would reproduce that hole exactly.
if git rev-parse --verify -q origin/main >/dev/null 2>&1 &&
[ "$(git rev-list --count origin/main..HEAD 2>/dev/null || echo 0)" -gt 0 ]; then
echo "Committed range vs origin/main:"
run "doc-checkpoint range" python3 scripts/check-doc-checkpoint.py \
--base origin/main --head HEAD
fi

if [ "$STAGED" -eq 1 ]; then
echo "Staged change:"
run "doc-checkpoint --staged" python3 scripts/check-doc-checkpoint.py --staged
Expand Down
Loading