Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .agents/NOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.

Unchanged: token-exact (or the ratified distributional gate) against the pinned
vLLM oracle, AND ≥ vLLM on every throughput axis / ≤ on latency and memory, on
both gate models, reproduced 2–3x on an idle box. See [gates.md](gates.md) and
[benchmark-protocol.md](benchmark-protocol.md). Parity pin: vLLM `555967922`
both gate models, reproduced 2–3x on an idle box. See the
[verification procedure](verification.md). Parity pin: vLLM `555967922`
(0.26.0.dev0).


Expand All @@ -63,7 +63,7 @@ both gate models, reproduced 2–3x on an idle box. See [gates.md](gates.md) and
`parity-ledger.md` LINE anchors (re-anchor by ROW ID).

**Operator/helper protocol**
([spec](specs/operator-helper-protocol.md)): roles DECLARED then MATERIALIZED
([spec](workflow.md)): roles DECLARED then MATERIALIZED
into a lock or worktree+PR; operator merges PRs first and does features only via
sub-agents; helpers use worktrees on `row/<ROW-ID>` and open a DRAFT PR at the
START, which IS the claim. **W0-W5 LANDED**; role discipline ENFORCING,
Expand Down
4 changes: 2 additions & 2 deletions .agents/backend-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Upstream references are pinned to vLLM `555967922` (0.26.0.dev0 + transformers
Build availability is not runtime support. A target remains `INVENTORIED` until
it has a native build, ported tests, hardware execution, and a trace showing the
intended dispatch. Performance claims additionally require same-workload runs
under [benchmark-protocol.md](benchmark-protocol.md).
under [verification.md](verification.md).

**How an architecture is added (the additive contract, `BACKEND-CUDA-ARCH-ADDITIVITY`).**
Since [arch-additivity](specs/cuda-arch-additivity.md) landed, arch support is
Expand Down Expand Up @@ -241,7 +241,7 @@ memory. Floating competitor versions do not count.

| ID | Item | Upstream/reference | Our code | Tests/evidence | Spike/spec | State | Owner |
|---|---|---|---|---|---|---|---|
| `BACKEND-GATE-CUDA-VLLM` | CUDA correctness and every-axis parity vs vLLM v0.25.0 | current pin `e24d1b24`; oracle `702f481`; async default `vllm/config/vllm.py:952-1043`; depth-2 engine path `vllm/v1/engine/core.py:519-607`; async output `vllm/v1/worker/gpu/async_utils.py:12-70`; [benchmark protocol](benchmark-protocol.md) | Existing schema-v5 validator/summary remain. The production [packed component driver](../scripts/dgx-gdn-packed-component.sh#L1) and fail-closed [component finalizer](../tools/bench/gdn_packed_component.py#L1) enforce exact corpus/build/oracle provenance, exact and bounded raw-metric validation, duration consistency, exact commands, pairwise stability, one-lock correctness, thermal/memory health, symlink-safe sealing and marker-last provenance | [Component/summary tests](../tests/tools/test_gdn_packed_component.py#L1) pass and the [2026-07-15 new-binding ledger row](parity-ledger.md) records the sealed evidence. **NEW BINDING `9ecd9d0`: 114/124** (full production default set = async + Triton GDN cubin + bit-identical fast RMSNorm + bit-identical fast gated-RMSNorm; supersedes `246a23c`'s 49/124, `3f256ab`'s 55/124, all retained immutable in the ledger). Mem **4/4**, c1 **20/20**, c2 **20/20**, c16 19/20, c4 & c32 18/20, c8 15/20. The bit-identical (0-ulp) fast decode-kernel stack (`348d12d`+`9ecd9d0`) closed **+62 axes** — confirming the decode deficit was norm/quant/act kernel glue. Ratified as "effective parity" via two-grid totality (114+111→115/124; 110 pass-in-both + 5 coin-flip), NOT a clean 124/124. Remaining gate: 10 axes (35B every-axis closure c1/c2 low-conc residual). 35B performance blocked until 27B reaches a clean 124/124. (2026-07-31 audit `CLAIM-ROADMAP-V1-AUDIT`: this row previously lagged the roadmap by two bindings; now synced) | [CUDA inventory](specs/cuda-architecture-inventory.md); [online gate](specs/cuda-online-serving-gate.md); [packed decode](specs/gdn-packed-decode.md); [W3-I](specs/nvfp4-fused-silu-producer.md) | `PARTIAL` | - |
| `BACKEND-GATE-CUDA-VLLM` | CUDA correctness and every-axis parity vs vLLM v0.25.0 | current pin `e24d1b24`; oracle `702f481`; async default `vllm/config/vllm.py:952-1043`; depth-2 engine path `vllm/v1/engine/core.py:519-607`; async output `vllm/v1/worker/gpu/async_utils.py:12-70`; [benchmark protocol](verification.md) | Existing schema-v5 validator/summary remain. The production [packed component driver](../scripts/dgx-gdn-packed-component.sh#L1) and fail-closed [component finalizer](../tools/bench/gdn_packed_component.py#L1) enforce exact corpus/build/oracle provenance, exact and bounded raw-metric validation, duration consistency, exact commands, pairwise stability, one-lock correctness, thermal/memory health, symlink-safe sealing and marker-last provenance | [Component/summary tests](../tests/tools/test_gdn_packed_component.py#L1) pass and the [2026-07-15 new-binding ledger row](parity-ledger.md) records the sealed evidence. **NEW BINDING `9ecd9d0`: 114/124** (full production default set = async + Triton GDN cubin + bit-identical fast RMSNorm + bit-identical fast gated-RMSNorm; supersedes `246a23c`'s 49/124, `3f256ab`'s 55/124, all retained immutable in the ledger). Mem **4/4**, c1 **20/20**, c2 **20/20**, c16 19/20, c4 & c32 18/20, c8 15/20. The bit-identical (0-ulp) fast decode-kernel stack (`348d12d`+`9ecd9d0`) closed **+62 axes** — confirming the decode deficit was norm/quant/act kernel glue. Ratified as "effective parity" via two-grid totality (114+111→115/124; 110 pass-in-both + 5 coin-flip), NOT a clean 124/124. Remaining gate: 10 axes (35B every-axis closure c1/c2 low-conc residual). 35B performance blocked until 27B reaches a clean 124/124. (2026-07-31 audit `CLAIM-ROADMAP-V1-AUDIT`: this row previously lagged the roadmap by two bindings; now synced) | [CUDA inventory](specs/cuda-architecture-inventory.md); [online gate](specs/cuda-online-serving-gate.md); [packed decode](specs/gdn-packed-decode.md); [W3-I](specs/nvfp4-fused-silu-producer.md) | `PARTIAL` | - |
| `BACKEND-BENCH-CUDA-SGLANG-PREFLIGHT` | SGLang corpus, harness, image and exact-checkpoint/token-ID preflights; P1 CPU harness is implemented/gated while P2 image/model/GPU classification remains | [SGLang v0.5.13](https://github.com/sgl-project/sglang/releases/tag/v0.5.13), commit `28b095c`; digest-pinned CUDA 13 image | [corpus](../tools/bench/make_serve_low_corpus.py#L143); [client/preflights](../tools/bench/run_serve_low.py#L124); [summary](../tools/bench/summarize_serve_low.py#L238); [memory](../tools/bench/sample_process_memory.py#L198); [driver](../scripts/dgx-sglang-low-concurrency.sh#L1) | [16 CPU contract cases](../tests/tools/test_serve_low_client.py#L103); [CTest registration](../tests/CMakeLists.txt#L12); dry-run manifest green; no image/model/GPU evidence or performance result | [SGLang low-concurrency spike](specs/cuda-sglang-low-concurrency.md) | `GATING` | - |
| `BACKEND-GATE-CUDA-SGLANG` | Binding CUDA low-concurrency serving vs SGLang | same pinned SGLang/checkpoints as `BACKEND-BENCH-CUDA-SGLANG-PREFLIGHT` | [bench main:1](../examples/bench/main.cpp#L1), [server main:1](../examples/server/main.cpp#L1) | no binding run; HTTP TTFT/ITL cannot be measured honestly yet | [SGLang low-concurrency spike](specs/cuda-sglang-low-concurrency.md) | `BLOCKED` on `SERVE-ASYNC-LLM` and successful exact-equivalence preflight | - |
| `BACKEND-GATE-CUDA-SGLANG-PREFIX` | Binding deterministic shared-prefix cache-on serving vs the faster equivalent vLLM/SGLang floor, separate from cache-neutral serving | SGLang v0.5.15 `f63458b`; digest `d0a667e`; [DGX recipe correction/results](https://github.com/Weschera/qwen-sglang-dgx-spark/tree/03253ef98c01de59a21c85b9a5cc6a27a871c383); vLLM v0.25.0 explicit Qwen hybrid `mamba_cache_mode=align` | current prefix hashes/managers [kv_cache_utils.cpp:259](../src/vllm/v1/core/kv_cache_utils.cpp#L259), [kv_cache_manager.cpp:124](../src/vllm/v1/core/kv_cache_manager.cpp#L124); no local GDN/Mamba aligned-state retention or binding long-prefix harness yet | Source/config audit rejects the original cache-asymmetric 10--40x claim and treats the reported residual 25--45% SGLang lead as unproven: cache-on vLLM is 0.23.1, KV dtypes/memory differ, MTP is enabled, only 35B is measured, and full axes/hits/memory/traces/repetitions are absent. PX1/PX2 implementation, exact 27B equivalence/hit proof and all performance evidence remain pending | [SGLang shared-prefix extension](specs/cuda-sglang-low-concurrency.md#shared-prefix-extension-2026-07-12) | `READY` for PX1 harness/counter work; PX2 begins with the `KV-MAMBA-ALIGN` leaf spike, and binding execution also needs exact v0.5.15 equivalence plus `SERVE-ASYNC-LLM` | - |
Expand Down
4 changes: 2 additions & 2 deletions .agents/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,12 @@ upstream platform PR ports mechanically.

## Non-negotiables

- The NVIDIA gate (see [gates.md](gates.md)) is not delayed by portability
- The NVIDIA gate (see [verification.md](verification.md)) is not delayed by portability
work: M0.2 bakes the interface requirements above (cheap now, unpayable
later); backend implementations are post-MVP.
- A new backend lands like any port: parity harness (same golden dumps —
they are backend-independent), behavioral suites unchanged, benchmark
honesty per [benchmark-protocol.md](benchmark-protocol.md). `DONE` additionally
honesty per [verification.md](verification.md). `DONE` additionally
requires match-or-beat against the applicable native floor: llama.cpp for
CPU/GGUF and Vulkan, oMLX/MLX-LM for Apple MLX, production vLLM for its native
platforms, plus the SGLang low-concurrency CUDA sweep. The canonical status
Expand Down
3 changes: 1 addition & 2 deletions .agents/benchmark-record.md
Original file line number Diff line number Diff line change
Expand Up @@ -10688,7 +10688,7 @@ scripts/dgx-online-serving.sh --execute --model 27 \
performance or broader roadmap execution.

The complete contract is in the
[benchmark protocol](benchmark-protocol.md) and
[benchmark protocol](verification.md) and
[online serving gate spec](specs/cuda-online-serving-gate.md).

**Breadth sweep note (2026-07-21):** the active phase is model-architecture breadth (recent-first), each held to token-exact + vLLM-speed on every axis. Ranked queue + CUDA-arch additivity audit: [.agents/specs/breadth-sweep-plan.md](specs/breadth-sweep-plan.md). CUDA archs beyond same-family sm_120 are HW-blocked (only GB10 testable).
Expand Down Expand Up @@ -15124,7 +15124,6 @@ llama.cpp's Vulkan" when it is really "our CPU tier vs llama.cpp's Vulkan". The
comparison becomes meaningful when native coverage closes — the progress metric is
`vt::GetReferenceTierHits()` reaching 0, and the ops that matter for this model are
the RoPE table build, the sampler tail, and the remaining norm/glue set.
>>>>>>> 814230a0 (bench(vulkan): VK-E unblocked with identical weights; ours quoted as NO RATIO)

#### CORRECTION (2026-08-07, same session): the vllm.cpp Vulkan arm is GPU-BOUND, not CPU-bound

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ an unreviewed change. If you cannot explain and defend a line, do not submit it.

Every commit MUST carry the trailer `FOLLOWING_AGENTS_PROTOCOL`. This is a
deliberate speed-bump: it asserts the contributor has **read
[AGENTS.md](../AGENTS.md)** and follows this project's protocol (mirror vLLM,
[AGENTS.md](../../AGENTS.md)** and follows this project's protocol (mirror vLLM,
ground every check in vLLM source, compare vs the vLLM oracle on the identical
workload, keep the parity ledger updated, etc.). CI (`commit-protocol-tag` in
`.github/workflows/ci.yml`) rejects any commit lacking it, so a contributor who
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ it becomes binding once the spike proves the workload/features are equivalent.
output lengths, sampling, cache/prefix state, concurrency and serving features.
If conversion is necessary, record it and do not turn the converted result into
a binding floor until correctness/quality equivalence is established. Full
matrix: [specs/competitive-benchmarks.md](specs/competitive-benchmarks.md).
matrix: [specs/competitive-benchmarks.md](../specs/competitive-benchmarks.md).

**Cache policy is part of the workload, never an incidental default.** CUDA
serving has at least two independent gates: a cache-neutral/cache-off workload
Expand Down Expand Up @@ -87,8 +87,8 @@ measured separately. Missing hardware leaves the row in `GATING` with an exact
handoff recipe; it never converts an unmeasured milestone into `DONE`.

Every feature/iteration checkpoint also updates the public
[`README.md`](../README.md) stage and
[`docs/BENCHMARKS.md`](../docs/BENCHMARKS.md) disposition **in the same
[`README.md`](../../README.md) stage and
[`docs/BENCHMARKS.md`](../../docs/BENCHMARKS.md) disposition **in the same
change**. This includes an attempted benchmark that fails closed: record it as
`FAILED` or `VOID`, name why no ratio is binding, and retain the exact next
reproduction command. A CPU-only/non-performance feature records
Expand All @@ -110,10 +110,10 @@ checkpoint.
serving, include the low-concurrency sweep when latency/scheduling could
change. For new backends, the area spike fixes representative models that fit
the hardware without pretending a small-model gate proves 27B/35B scale.
5. **Record every axis + every ratio** in [parity-ledger.md](parity-ledger.md),
5. **Record every axis + every ratio** in [parity-ledger.md](../parity-ledger.md),
then refresh the concise accepted/pending/failed/void summary in
[`docs/BENCHMARKS.md`](../docs/BENCHMARKS.md) and the current stage in
[`README.md`](../README.md). A ledger row that leaves ANY axis below an
[`docs/BENCHMARKS.md`](../../docs/BENCHMARKS.md) and the current stage in
[`README.md`](../../README.md). A ledger row that leaves ANY axis below an
applicable floor is an open gap.

## Reproduction is a GATE
Expand Down Expand Up @@ -162,7 +162,7 @@ rumor; a reproducible run is evidence.
server files may instead be evicted with `POSIX_FADV_DONTNEED` only when a
retained report hashes that inventory and `mincore(2)` proves **zero resident
pages** afterward; an unverified/best-effort advisory call does not qualify.
See [specs/competitive-benchmarks.md](specs/competitive-benchmarks.md)
See [specs/competitive-benchmarks.md](../specs/competitive-benchmarks.md)
§ "Folded: spark-bench".

If a result cannot be reproduced on demand under these rules, it does not count
Expand All @@ -181,5 +181,5 @@ toward the gate.
every axis, both models**. Post-MVP backend rows additionally remain open while
below their applicable native floor.

See also: [gates.md](gates.md) (the MVP gates), [parity-lever-protocol.md](parity-lever-protocol.md)
See also: [gates.md](mvp-gates-legacy.md) (the MVP gates), [parity-lever-protocol.md](../parity-lever-protocol.md)
(how to find the levers to close a below-vLLM axis).
4 changes: 2 additions & 2 deletions .agents/gates.md → .agents/completed/mvp-gates-legacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

1. **Throughput parity vs vLLM and every equivalent faster floor** on
the release-gate environment (`${GATE_HOST}`; currently the DGX Spark GB10
profile in [environment.md](environment.md)): serve
profile in [environment.md](../environment.md)): serve
**Qwen3.6-35B-A3B (NVFP4)** and **Qwen3.6-27B (NVFP4)** with prefill AND
decode throughput matching vLLM at large concurrency (request-rate sweeps,
measured with our `bench serve` equivalent, same box, same models).
Expand Down Expand Up @@ -50,7 +50,7 @@

vLLM is the parity oracle for BOTH correctness and performance. Every change
that could affect either MUST be compared against vLLM, apples-to-apples, and
BOTH numbers recorded in the [parity ledger](parity-ledger.md):
BOTH numbers recorded in the [parity ledger](../parity-ledger.md):

- **Correctness:** op dumps + model logits/greedy vs the pinned pip-vLLM oracle
(`${VLLM_ORACLE}`, forward-math-identical to the pin). A new
Expand Down
Loading
Loading