record(intake): place the recipes.vllm.ai sweep and its six issues (#605-#610) by localai-bot · Pull Request #612 · mudler/vllm.cpp

localai-bot · 2026-08-13T10:27:32Z

Places the recipes.vllm.ai audit in roadmap_v1.md: the six issues into the
intake table, and a Recipe-surface sweep section alongside the 2026-07-28
vLLM feature-gap sweep.

Issues: #605, #606, #607, #608, #609, #610.

Why a second sweep

The feature-gap sweep asked what vLLM has that we lack. This asks whether a user
can run the published recipe for a model we already ship. Where they disagree,
this one is sharper: a model gated token-exact whose own recipe command aborts at
argument parsing is not usable, however good the kernels are.

Grounding

vllm-project/recipes @ 86c7777aa699482ef1ebd0c5da9fc540ccc00a40 — 157 model
recipes, 70 distinct CLI flags, 15 env vars. Architectures resolved from each
model's HF config.json (137/157 — the remainder are gated repos or diffusion
pipelines with no top-level architectures), joined against model-matrix.md, and
checked against vLLM's registry at the pin 5559679 and on main.

What it found

Model coverage is broad — 79 of the unstarted recipes already have rows, so they
are scoped rather than missing. The serve surface is where it breaks:

vllm-serve aborts on an unknown argument, so --enable-auto-tool-choice (89
recipes) and --trust-remote-code (82) — both no-ops for us — stop the
server before model load.
--reasoning-parser resolves 10 of upstream's 28 names: 61 of 76 recipe uses
throw, including qwen3, the corpus's most common value, which the published
Qwen3.5/3.6 recipes pass to models we gate token-exact.
Tool parsers are the healthy axis: 41 names, 82/90 uses.

Placement decisions

#605 goes on SAMPLE-REASONING rather than opening a track — that spec already
enumerates these names in W2/W3. What the recipe data adds is ordering, and it
inverts the numbering: W3 covers 43/76 uses, W2 covers 18 and four of its names
have zero recipe demand.

#606, #607, #609 and #610 are left as em dashes. No row owns serve CLI recipe
compatibility or model-matrix intake, and per the convention this table documents,
a wrong row id in a keyed record is worse than an honest blank.

Also recorded

A measurement trap: --language-model-only, --async-scheduling, --enforce-eager
and --mamba-ssm-cache-dtype appear in this repo only in tools/bench/ scripts
that drive the oracle. Grepping the flag name reads as coverage and is the
opposite of it.

And taxonomy.yaml carries a restricted dgx_spark_gb10 profile that 8 recipes
declare — upstream has published GB10-validated configurations for our exact gate
box, six for rows already active.

Records only, no product code touched. scripts/agent-preflight.sh --staged clean.

🤖 Generated with Claude Code

-#610) FOLLOWING_AGENTS_PROTOCOL The 2026-07-28 feature-gap sweep asked what vLLM has that we lack. This one asks a different question -- can a user run the PUBLISHED recipe for a model we already ship -- and the answers disagree. Read vllm-project/recipes at 86c7777a: 157 model recipes, 70 distinct CLI flags, 15 env vars. Architectures resolved from each model's HF config.json (137/157; the rest are gated repos or diffusion pipelines with no top-level architectures) and joined against model-matrix.md and the registry at the pin. Model coverage is broad -- 79 of the unstarted recipes already have rows, so they are scoped, not missing. The serve surface is where it breaks. vllm-serve aborts on an unknown argument, so an unmapped flag is not a degraded run but no run: --enable-auto-tool-choice (89 recipes) and --trust-remote-code (82) are both no-ops for us and both abort startup. --reasoning-parser resolves 10 of upstream's 28 names, so 61 of 76 recipe uses throw -- including qwen3, the corpus's single most common value, which the published Qwen3.5/3.6 recipes pass to models we gate token-exact. Tool parsers are the healthy axis at 82/90. #605 is placed on SAMPLE-REASONING rather than opening a track: that spec already enumerates these names in W2/W3. What the recipe data adds is the ordering, and it inverts the numbering -- W3 covers 43 of 76 uses, W2 covers 18 and four of its names have zero. #606 and #607 are left as em dashes because no row owns serve CLI recipe compatibility, and a wrong row id in a keyed record is worse than an honest blank. Two things worth recording beyond the issues. --language-model-only, --async-scheduling, --enforce-eager and --mamba-ssm-cache-dtype appear in this repo only in tools/bench/ scripts that drive the ORACLE, so grepping the flag name reads as coverage and is the opposite of it. And taxonomy.yaml carries a dgx_spark_gb10 profile that 8 recipes declare -- upstream has published GB10-validated configurations for our exact gate box, six of them for rows already active, which fixes the honest denominator for us instead of us choosing it. Records only, no product code touched. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

FOLLOWING_AGENTS_PROTOCOL The em dash was honest when written -- no row owned serve CLI recipe compatibility. The spec PR creates one, so the keyed record is updated in place rather than left understating where the work lives. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…row exists" FOLLOWING_AGENTS_PROTOCOL This reverts 84beead. The row exists on row/serve-recipe-args, not here, so on THIS branch the placement was a record pointing at nothing -- check-agent-record caught it exactly right, twice over: "dangling link specs/serve-recipe-args.md" and "references unknown stable row SERVE-RECIPE-ARGS". The em dash goes back, and it is not a placeholder for laziness: the table documents that convention precisely because a row id that does not resolve is worse than an honest blank. #606 gets placed once its row lands on main, in a change that can actually see it. Grouping the placement with the intake was the mistake -- two branches, one keyed record, and the ordering between them is not something either branch can assume. python3 scripts/check-agent-record.py: 0 errors. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…#609, #610) FOLLOWING_AGENTS_PROTOCOL The sweep counted 11 unrowed RECIPES and then reported them as if they were rows. They are not: 11 recipes resolve to 8 distinct architectures, because MossTTSDelayModel alone is reached by four of them (MOSS-TTS, MOSS-TTSD-v1.0, MOSS-SoundEffect, MOSS-VoiceGenerator). #610's headline said seven while its own table enumerated six. A row is an architecture, so 8 is the number that matters, and the issue title is corrected to match its table. Caught by the implementer re-deriving the audit independently rather than taking the brief's count on faith -- which is what the brief asked for and the reason the wrong number did not reach the matrix. The rollup moved 314->322 / 362->370. Also records what this sweep CANNOT see. 20 of the 157 recipe configs were unreadable (gated or 404: Voxtral-4B-TTS-2603, GLM-TTS, IndexTTS-2.5, stable-audio-open-1.0, several meta-llama), so those architectures are unverified and 8 is a FLOOR. And any vllm-omni architecture no recipe references is invisible to a recipe-driven sweep by construction. Stating the blind spot beats letting the number read as a total. python3 scripts/check-agent-record.py: 0 errors. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ce (#620) FOLLOWING_AGENTS_PROTOCOL This branch appended #609 and #610 to the intake table with em-dash rows, because when it was written neither issue had a row to point at. #620 has since added the eight matrix rows AND placed both issues in place, keyed to MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm and MODEL-MM-moss-tts-moss-tts-delay-talker-for-generation. Two branches, one keyed record, entries at different line positions: git merges both without a conflict and the table ends up defining #609 and #610 twice. That is the move-plus-add duplicate shape -- it does not announce itself, because a clean merge is exactly what it produces. The append is the weaker of the two (em dash vs a real row), so it goes; #620 owns those keys. This branch keeps #605-#608, which #620 does not touch. Not a merge-order fix: removing the duplicate here is correct whichever PR lands first. python3 scripts/check-agent-record.py: 0 errors. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…CausalLM FOLLOWING_AGENTS_PROTOCOL The sweep reported 8 unrowed architectures. One of them, Qwen3_5MoeForCausalLM, was already owned by #490 and its open PR #601 -- which does not merely row it but REGISTERS it alongside Qwen3_5ForCausalLM, resolves the backbone prefix, refuses unsupported MoE layouts, and carries CPU 404/404 plus SACRED GPU 315/315, 235/235 and 138/138, reviewed twice over five reproduced mutations. #601 also adds a row this sweep never saw, MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm. So 7 are owed: BailingMoeV3ForCausalLM plus the six vllm-omni architectures. AGENTS.md says to re-verify a gap against issues AND PRs before claiming. I grepped issue titles with terms that did not match #490's wording and did not check open PRs at all. The verification that should have run first was cheap: diff every open PR against main for each architecture string. Run afterwards, it shows Qwen3_5MoeForCausalLM in two branches and the other seven in one -- which is exactly the answer needed before dispatching anyone. This matters beyond one wasted row. Two branches ADDING the same keyed row merge without a conflict and define it twice; nothing announces it, because a clean merge is the symptom. Same shape as the #609/#610 duplication removed two commits ago, from the same root cause: assuming a keyed record has one writer. Both counting lessons are recorded in the sweep section rather than just fixed, since the next sweep will face both. python3 scripts/check-agent-record.py: 0 errors. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…622) FOLLOWING_AGENTS_PROTOCOL NOW.md said "all 362 archs have rows". That number is owned by model-matrix.md, whose rollup is CI-enforced, and #625 moves it to 369 -- so the line was about to be wrong, as it had been before. Bumping it would be the wrong fix. AGENTS.md is explicit: never store a measurement of one file inside another, because a number that moves on every edit couples every PR to lines it does not own. NOW.md is authored at operator cadence and is "never a per-row lifecycle write", so a row-adding PR could not satisfy both rules at once -- it had to leave NOW.md wrong or touch a file outside its scope. That is a records defect, not a stale line, and it is #622. The claim survives without the figure: "every arch in the pinned registry has a row" is what the sentence was actually asserting, and it stays true as rows are added. The count is one link away in the file that owns it. Removing the number removes the failure mode instead of guarding it -- and nothing checks NOW.md's figure against the matrix, deliberately, since that would be the second description this protocol exists to avoid. NOW.md stays inside budget: 73 lines / 3,822 chars against 100 / 6,000. scripts/agent-preflight.sh --staged: exit 0. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ment and three anchors that disagree with the tree (#606, #631) Repairs the fresh scoped review of #624. The review's verdict was FAIL on the RECORD and explicitly not on the code: the seam, the mirrored cli_args.py:395 validation, the notice and the four tests all reproduced exactly, mutation included. Nothing in `src/` or `tests/` changes here, by design. 1. AN UNNAMED RED LANE (the one that matters). The spec's `## Now` attributed the red CI to the two `windows-msvc-*` lanes and #584, so a reader was told the CPU lane was green. It is not: `build-test-cpu` fails on `test_cpu_threadpool`, 1 of 404, at `ratio 101.567` against a fixed `< 100.0`. AGENTS.md § Gates wants exactly one result per applicable rule, and a lane omitted reads as a lane satisfied. It is now named, with the same attribution rigor already spent on #584 rather than a bare "unrelated": the guard divides two wall-clock medians, the 2-core runner collapsed the DENOMINATOR to 0.48 us, and argument parsing inside `ParseArgs` has no path to a threadpool dispatch ratio. Already filed as #631 while the review was running, so this references that issue rather than opening a second one. The evidence is three observations of the same code, and the third is the informative one. The 2-core runner read 101.567; #631's table read 2.699 on a 20-core box; rebuilding the reviewed head here read 0.990872 on a 20-core box (9 cases / 9 passed, 19602 assertions, `Status: SUCCESS!`). Two same-class boxes 2.7x apart is the finding: the statistic will not carry a fixed threshold. Recorded rather than smoothed, and the middle row is kept attributed to #631 rather than restated as mine. Also recorded because it falsifies the test's own claim at `test_cpu_threadpool.cpp:496` that it can "be trusted ... never to fail spuriously on a busy one". NOT fixed in flow, deliberately. Repairing that guard changes a gate's semantics, which AGENTS.md routes through its own spec and red-before evidence; and raising 100 to a larger number would be widening a scope to turn a red gate green, which is the specific thing forbidden. 2. A COMMENT FALSE ABOUT ITS OWN TREE. The `ENGINE_ROWS` 152 -> 153 justification in `check-agent-record.py` said the row is `SPIKE` and that "no seam, no test, and no `docs/USAGE.md` entry exists yet". All four were true at the spec commit `0d8274ac3` and none is true at this head. `main` is squash-only, so the single landing commit would have shipped a comment contradicted by files in the same commit — AGENTS.md § History is git: the record cannot disagree with the tree. Corrected to state the tree rather than framed "as of the spec commit". The as-of framing would also have been accurate, and is rejected on purpose: it dates a justification to a commit that squash-merging deletes, so the reader who meets the line has no way to check it. A comment that describes the file it sits in is verifiable by looking; one that describes a vanished intermediate is not. 3. THREE STALE LINE ANCHORS, in `.agents/engine-matrix.md` and the spec. `b7eaadd07` inserted a 4-line comment at `server_main.cpp:299-302` and shifted everything below it. Re-derived from the file rather than from the review's table, and the table was right: `FindAcceptedInertArg` 308 -> 312, the parse branch 501 -> 505, the mirrored validation 557 -> 560. `kAcceptedInertArgs:289` sits above the insertion and all four test anchors (`:141`, `:165`, `:188`, `:212`) are exact; those five are untouched. 4. A FORWARD REFERENCE, made legible instead of resolved. The spec cited `roadmap_v1.md` § Recipe-surface sweep, which does not exist there yet — it arrives with #612, along with the intake rows for #605-#608. The citation now says so. #606 is deliberately NOT added to the intake table: #612 owns those keys, and two branches adding the same keyed record merge cleanly and then define it twice, which has already happened twice in this campaign. 5. `InertArg::takes_value` recorded as a named cold path. Both shipped entries are `kNoValue`, so the value-consuming branch is unexercised and cannot be tested without inventing an entry. The obligation moves to the spec: the first entry with `takes_value == true` owes a test in the same change. No speculative code and no speculative test. check-pr-size requires paired mutation evidence for any change to `scripts/check-agent-record.py`. This PR already carries it — `test_agent_record.py`'s `test_serve_recipe_args_row_is_inside_the_engine_ratchet` is the semantic evidence for the 152 -> 153 bump, and the checker verifies the pair over the whole PR range. The edit above is comment-only and changes no behavior that any test could newly pin, so no test is added: a test written to satisfy a counter rather than to defend a guarantee is the thing that rule exists to prevent. Verification, all foreground and observed: `scripts/agent-preflight.sh --staged` exit 0, "All gates green"; `scripts/check-windows-portability.py` exit 0, "Windows portability contract OK" (it is Windows-only and not part of preflight, which is how the earlier `open (` trap got through); each corrected anchor read back from the file it points into. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

FOLLOWING_AGENTS_PROTOCOL main advanced while this branch was open: #616's spec amendment squash-landed, along with the four other recipe-sweep PRs. Two conflicts, resolved deliberately. reasoning-parsers.md: this branch carries #616's amendment (it was based on that branch) PLUS its own W3 status -- nano_v3 added to the wave list and the "Brick 1 DONE" record. Taking this side is correct because it is a strict superset; verified by confirming the amendment section and all names survive, not by assuming supersetness from the diff shape. roadmap_v1.md: the check-agent-record gate refused with "issue #605 listed twice" -- and it was RIGHT. #612 landed an intake row for #605 while this branch carried its own, so the merge would have defined one key twice. That is the fourth time this campaign that two branches wrote the same keyed record; it is the first time a gate caught it rather than a human, which is worth recording. Resolved per AGENTS.md: kept main's row in main's position and folded this branch's content into it as a scoped update, rather than appending a second row. Not resolved by taking either side wholesale: the surviving row now states both the original defect AND that brick 1 landed, because after this merge both are true and a keyed record must describe its key's current state. scripts/agent-preflight.sh --staged: exit 0, all gates green. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

FOLLOWING_AGENTS_PROTOCOL main advanced to b81a5e4 (#641's LTX-2.5, then #650's oracle policy) and roadmap_v1.md's Open-issues table conflicted. That table is a KEYED RECORD, so AGENTS.md forbids resolving it by automatic three-way merge: took main's file WHOLESALE and reapplied this branch's two scoped edits. The two edits are NOT the same operation, and treating them alike is the trap I walked into first. #608 already exists on main -- it landed via #612 -- so it is an UPDATE IN PLACE. #649 is new, so it appends. My first attempt appended both and produced a duplicate #608, caught by counting rather than by any gate: the uniqueness check the checker does run would have caught it too, but only after I had already written the wrong thing. Verified rather than asserted: diff against main is +2/-1 -- one row replaced, one added -- and grep gives exactly one #608 row and one #649 row. One gate needed a second look. test_cpu_x86_llamacpp_floor's contended-leg case failed once here at load 26.97, then passed twice consecutively in this same worktree, and passes on clean main at the same load; the test file and tools/bench are byte-identical to main. That is #618's known flakiness, not this resolution. Re-ran the full staged gate afterwards for a clean result rather than reasoning past the red. scripts/agent-preflight.sh --staged: exit 0, all gates green. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

mudler added 7 commits August 13, 2026 10:27

localai-bot mentioned this pull request Aug 13, 2026

feat(SERVE-RECIPE-ARGS): accept two inert recipe flags so a published vllm serve line reaches model load (#606) #624

Merged

localai-bot merged commit da7dff5 into main Aug 13, 2026
21 of 23 checks passed

localai-bot deleted the row/recipes-intake branch August 13, 2026 14:28

Navigation Menu

record(intake): place the recipes.vllm.ai sweep and its six issues (#605-#610) - #612

record(intake): place the recipes.vllm.ai sweep and its six issues (#605-#610)#612
localai-bot merged 7 commits into
mainfrom
row/recipes-intake

localai-bot commented Aug 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-bot commented Aug 13, 2026

Why a second sweep

Grounding

What it found

Placement decisions

Also recorded

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants