docs(dspark): storage refuted; ratio stable at ~0.966 over three within-session pairs (#442) - #550
Merged
Merged
Conversation
…ions (#442) The developer asked whether the weights sitting on NAS, or not being fully resident, could be distorting these measurements. Tested and refuted: the weights are on local NVMe (no NAS mount exists on the box), a run reads 22.06 GB once at load, decode-time RSS is 4.8 GB because the mapping is released after upload, and 8 warm reps hold a 0.5% spread. File-backed weights could not produce that stability, and on GB10 the failure mode would be loud anyway -- host pages reach the GPU through ATS at a measured 20-30% per-GEMM penalty. Recorded alongside it: that NVMe is 98% full (76 GB free). This repo has already lost a gate run to ENOSPC reporting green over work that never ran. Also records a third WITHIN-session paired ratio, ours -> oracle -> ours at free clocks with drift bracketed at -0.89%: 140.98 vs a modal 147.32, RATIO 0.9569. With the two pinned-clock pairs (0.9757, 0.9646) that puts the gap at ~0.966 +/- 0.01, consistently below 1.0, while the ABSOLUTE numbers move up to 5% between sessions for the same binary because GB10's memory clock cannot be pinned. Only the within-session ratio is quotable, and all three agree. The oracle's draws remain bimodal (~147.3 vs ~155.6) from the same one-extra- accepted-token effect, so its modal draws stay the honest denominator. No code change; this is the record catching up with the measurements. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Picks up the hf_snapshot.h build repair (#556) so this record-only PR builds. Its CPU, sanitizer and Windows-compile jobs were red for that reason alone -- the branch changes four markdown files and cannot affect a build. No overlap: #556 touches tests/parity/hf_snapshot.h, this branch touches the DSpark spec, the benchmark record, STATUS and BENCHMARKS. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 13, 2026
#550 landed squashed, so its content arrives on main as a new commit while this branch already carried it plus everything since. All three conflicts are that shape -- HEAD is the superset in every hunk -- so HEAD is taken wholesale, which is what the record-conflict rule asks for. Verified rather than assumed: all 317 `## ` entries on main's .agents/benchmark-record.md are present here, so the union lost nothing. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Record-only change closing out the measurement question the developer raised: could the weights being on NAS, or not fully resident, be distorting the DSpark numbers?
Tested and refuted. The weights are on local NVMe (
/dev/nvme0n1p2ext4 — no NAS mount exists on the box). A run reads 22.06 GB total, i.e. one full model read at load. Decode-time RSS is 4.8 GB because the mapping is released after upload, and 8 warm reps hold a 0.5% spread (146.0–147.6). File-backed weights could not produce that stability; on GB10 the failure mode would be loud anyway, since host pages reach the GPU through ATS at a measured 20–30% per-GEMM penalty.Recorded alongside it, because it is its own risk: that NVMe is 98% full (76 GB free), and this repo has already lost a gate run to ENOSPC reporting green over work that never ran.
Third within-session paired ratio, ours → oracle → ours at free clocks with drift bracketed at −0.89%: 140.98 vs a modal 147.32, RATIO 0.9569.
That puts the gap at ~0.966 ± 0.01, consistently below 1.0, while the absolute numbers move up to 5% between sessions for the same binary because GB10's memory clock cannot be pinned — which is exactly why only the within-session ratio is quotable, and all three agree. The oracle's draws remain bimodal (~147.3 vs ~155.6) from the same one-extra-accepted-token effect, so its modal draws stay the honest denominator.
No code change. Issue: #442.
Gate:
scripts/agent-preflight.sh --staged— All gates green. (test_cpu_x86_llamacpp_floorfailed once mid-session at loadavg 151 from another session's ctest storm — it is the harness's own contended-leg test — and passes 10/10 standalone.)FOLLOWING_AGENTS_PROTOCOL