policy(oracle): admit a pinned secondary oracle where vLLM implements nothing (#647)

[localai-bot]

Closes #647.

AGENTS.md said comparisons run against "the pinned oracle" and pointed at
upstream-sync.md, which pins exactly one thing: vLLM. This repository already
compares itself against five other upstreams, and every one of their pins lives
somewhere else or nowhere.

Upstream	Pin before this PR
vLLM-Omni	none at all (#633) — H3 W3+, LTX-2.5 and ~40 omni-only architectures have no oracle
SGLang	v0.5.15/f63458b5 in sglang-matrix.md and v0.5.13/28b095c0 in cuda-sglang-low-concurrency.md, with nothing making the difference visible from either
llama.cpp	237ad9b96 (b9892), repeated across six CPU specs
transformers	5.14.1, recorded only as a component of the vLLM env line
tt-forge	nowhere, though BACKEND-TENSTORRENT is ACTIVE on real Blackhole and porting-inventory.md:1004 already states vLLM has no Tenstorrent platform anywhere

So the rule had no answer for a model vLLM does not implement at all.
MiniMax-Music3 (minimax_music3, served by SGLang-Omni and diffusers) is
the case that surfaced it: scoping stalled on a policy question, not an
engineering one.

What changes

Policy. AGENTS.md gains §"When vLLM has no implementation": vLLM stays
primary and wins every disagreement; a secondary oracle is admissible only from
a fixed table and only at a recorded pin; it never becomes the mirror source;
the row reconciles onto vLLM when vLLM implements the path. Gateability keeps
the primary's rule — gateable once it demonstrably builds and runs the model.

Pin surface. .agents/oracles/<id>.md, one file per oracle, read by
glob. A shared table would be a lock: two rows advancing two pins in one week
conflict on lines neither owns. vllm.md deliberately does not restate the
parity pin's meaning — upstream-sync.md keeps it, because a value transcribed
twice drifts, which is the defect this registry removes rather than re-imports.

Enforcement. scripts/check-oracle-pins.py + --self-test + a unit suite,
wired into agent-preflight.sh and CI. It requires all nine keys non-empty with
no undefined key, exactly one role = primary and that one vllm, and the
AGENTS.md table and the directory to name the same ids in both directions.
gateable = yes demands a non-UNPINNED revision and evidence that is a path
existing in the tree; an issue number is refused there and required in the other
direction.

Evidence

Three mutations performed against the live registry, not asserted about:

Mutation	Result
flip sglang to gateable = yes	red — "an issue is a promise of a measurement, not one"
delete the tt-forge row from the AGENTS.md table	red — "pinned in .agents/oracles/ but absent from the AGENTS.md table"
repoint llama-cpp evidence at a nonexistent path	red — "evidence path does not exist in this tree"

Tree restores byte-for-byte after each. The unit suite performs the same drop
for every required key one at a time, so the key list cannot outgrow its
enforcement; --self-test sweeps 11 fixtures in both directions. 24 tests pass.

The live checker found a real defect during development — the id regex matched
the table's own backticked header cell — which is why the header now reads
Registry id.

Honest limits

The checker is network-free and never verifies a pin exists upstream (a
fabricated 40-hex passes shape and fails review). It does not reconcile
vllm.md against upstream-sync.md, since teaching it to sync two
transcriptions would bless the duplication. The **Secondary oracle:** check is
opt-in by syntax — a checker firing on the word "SGLang" would red hundreds of
innocent specs and be silenced within a week.

Five of seven records say gateable = no, which is the honest reading, not
a gap in the work: vLLM-Omni and diffusers have never been executed here,
SGLang has been read but never run (its parity spec is explicitly "no
measurement taken here"), and tt-forge needs Blackhole silicon. Only vLLM and
llama.cpp carry measured evidence; transformers carries the AUDIO A0 goldens
captured from it. Those debts stay open on #647.

tt-forge scan, since admitting an oracle is a claim: it executes models end
to end over TT-Metalium across three frontends with 800+ variants in CI, so it
is a reference to run, not merely a stack to read. tt-forge-models gets no
record — it supplies ForgeModel loaders and publishes no reference outputs and
no PCC comparison, so it is a source of subjects, named in tt-forge's scope.
ttnn is not an oracle either: our backend calls it, and an oracle is something
we compare against.

Exception argued, not hidden: this skipped the fresh-reviewer round trip on
explicit developer direction to add the policy immediately and merge it
directly. Compensating evidence is the mutation set plus full preflight; nothing
here touches src/ or a gate's numeric semantics.

Gate

scripts/agent-preflight.sh — all checkers and all suites ok except
test_cpu_x86_llamacpp_floor, a contention-sensitive harness that failed
identically (NO_QUIET_WINDOW, exit 4) on an unmodified tree at base SHA
cefacd2d0 under load average 35. Pre-existing and environmental, verified
rather than assumed.

🤖 Generated with Claude Code