Skip to content

vllm-serve aborts on --enable-auto-tool-choice and --trust-remote-code, so 89 of 157 official recipe commands fail to start #606

Description

@localai-bot

What

vllm-serve hard-errors on an unrecognized argument
(src/vllm/entrypoints/openai/server_main.cpp:440). Two flags that appear in
almost every official recipe are unrecognized, so copy-pasting a published
vllm serve line aborts before the model loads
.

Measured over vllm-project/recipes @ 86c7777a (157 model recipes):

Flag Recipes using it Our engine
--enable-auto-tool-choice 89 / 157 unrecognized → abort
--trust-remote-code 82 / 157 unrecognized → abort

Both are no-ops for us, which is what makes this cheap:

  • --enable-auto-tool-choice is upstream's opt-in for automatic tool choice. We
    already behave as if it is always set whenever --tool-call-parser resolves to
    a parser, so accepting it changes nothing.
  • --trust-remote-code authorizes executing Python from the checkpoint. We have
    no Python, so there is nothing to authorize — it is N/A by construction,
    not unimplemented.

A recipe's own example is the reproducer:

# recipes/models/Qwen/Qwen3.5-27B.yaml, features.tool_calling
vllm serve Qwen/Qwen3.5-27B-FP8 --enable-auto-tool-choice --tool-call-parser qwen3_coder
#                               ^ server: unknown argument '--enable-auto-tool-choice'

Why not just ignore every unknown flag

We should not. Rejecting unknown arguments is correct — silently ignoring
--tensor-parallel-size would let a user believe they got TP. The fix is an
explicit accepted-and-inert list with a one-line reason per entry, plus a
startup notice naming what was accepted and why it does nothing. Anything genuinely
unimplemented keeps failing loudly.

Scope

Add the accepted-and-inert seam to server_main.cpp with per-flag justification,
a notice on use, and docs/USAGE.md entries stating explicitly that these are
accepted for recipe compatibility and have no effect. RED-first: a test that
asserts each listed flag starts the server and each unlisted unknown flag still
aborts.

Needs a row — no engine-matrix row covers serve CLI recipe compatibility today.
Found while auditing recipes.vllm.ai coverage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions