Skip to content

The recipe sweep could not read 20 of 157 configs, so its architecture counts are a floor presented as a total #626

Description

@localai-bot

What

The recipes.vllm.ai sweep (#605#610) resolved recipe models to architectures by
reading each model's HF config.json. 20 of the 157 could not be read — gated
repos returning 401, or 404. Their architectures are therefore UNVERIFIED, not
confirmed absent, and every count that sweep produced is a floor rather than a
total
.

Unreadable at the time of the sweep (vllm-project/recipes @ 86c7777a):

Reason Models
401 gated meta-llama/Llama-3.1-8B-Instruct, meta-llama/Llama-3.3-70B-Instruct, meta-llama/Llama-4-Scout-17B-16E-Instruct, google/translategemma-27b-it, inclusionAI/Ring-1T-FP8, pfnet/plamo-3-nict-31b-base, stabilityai/stable-audio-open-1.0, stabilityai/stable-diffusion-3.5-medium, zai-org/GLM-GA
404 IndexTeam/IndexTTS-2.5, mistralai/Mistral-Large-3-675B-Instruct-2512, mistralai/Voxtral-4B-TTS-2603, zai-org/GLM-TTS
no top-level architectures (diffusion pipelines / omni) MiniMaxAI/MiniMax-H3, Qwen/Qwen-Image, Wan-AI/Wan2.2-T2V-A14B-Diffusers, zai-org/GLM-Image, meituan-longcat/LongCat-Image-Edit, openbmb/VoxCPM2, fishaudio/s2-pro

Several are almost certainly already rowed (the Llama ones resolve to
LlamaForCausalLM), but "almost certainly" is not the standard for an inventory
that claims exhaustiveness.

The second, larger hole

A recipe-driven sweep can only see architectures some recipe references. Any
vllm-omni architecture with no recipe is invisible to it by construction —
vllm-omni's own docs/models/supported_models.md lists TTS/audio architectures
beyond the six rowed in #610.

Why it matters

.agents/model-matrix.md presents itself as the exhaustive architecture
inventory, and its rollup is CI-enforced — which makes the count look
authoritative regardless of how it was derived. A floor presented in a surface
that reads as a total is the failure mode worth closing.

Scope

Resolve the 20 by a route that does not depend on anonymous HF reads — the
transformers/vllm config machinery in the pinned oracle venv, an authenticated
fetch, or the recipe guide: prose where it names the architecture. Then sweep
vllm-omni's supported-models doc directly rather than through recipes, and row
whatever is missing. Records-only unless something surprising turns up.

Not urgent — but the sweep's own numbers should not be quoted as totals until it
is done.

Found while auditing recipes.vllm.ai coverage; flagged by the implementer on
#625, who declined to infer coverage it could not verify.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions