Skip to content

Oracle policy has no fallback and no pin concept: when vLLM has no implementation, secondary oracles (transformers, diffusers, SGLang/SGLang-Omni, llama.cpp) are used unpinned and unenforced #647

Description

@localai-bot

The gap

AGENTS.md §"vLLM is the reference" says comparisons run against the pinned
oracle in .agents/upstream-sync.md. That surface pins exactly one thing: vLLM.

But work in this repo already gates against four other upstreams, and every one
of them is pinned in a different spec file, or not at all:

Upstream Where its pin lives today Registered?
vLLM .agents/upstream-sync.md parity-pin block yes
vLLM-Omni nowhere — #633 no
SGLang .agents/specs/cuda-sglang-low-concurrency.md (v0.5.13 / 28b095c0) no
llama.cpp ~6 CPU specs (237ad9b96 / b9892) no
HF transformers only as a component of the vLLM env line no
diffusers nowhere no
SGLang-Omni nowhere no

Consequences already observed:

What this issue owes

  1. AGENTS.md: state the fallback rule — vLLM (incl. vLLM-Omni) is the primary
    reference; where it has no implementation, a named secondary oracle from
    an admissible set is used, never an unnamed one, and never in preference to
    vLLM where vLLM implements the behavior.
  2. A pin concept for secondary oracles, on a record surface that is one file
    per oracle (globbed for reading) rather than a shared file every PR must
    write.
  3. Enforcement: a checker with a --self-test corpus and a mutation test,
    wired into scripts/agent-preflight.sh and CI — so the rule cannot rot into
    prose nobody runs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions