feat(vllm): add grammar and structured output support

[fury0928]

Description

This PR fixes #6857

Adds grammar and structured output support to the vLLM backend, enabling users to enforce structured outputs via JSON schema, JSON object, and BNF/GBNF grammar constraints.

Problem

The vLLM backend ignored all structured output parameters:

• The Grammar field from the proto was never read
• A phantom GuidedDecoding mapping referenced a non-existent proto field
• response_format with json_schema or json_object had no effect on vLLM
• Missing import time caused a runtime crash on video input

Solution

Proto (backend.proto):

• Added JSONSchema (field 52) and ResponseFormat (field 53) to PredictOptions, allowing backends to receive the raw JSON schema and format type natively

Go endpoints (chat.go, completion.go):

• Extract the raw JSON schema string from response_format: {type: "json_schema", ...} and store it on config.JSONSchema
• Set config.ResponseFormat to the format type (json_object / json_schema)
• GBNF grammar is still generated in parallel for llama.cpp compatibility
• Added json_schema grammar support to the completion endpoint (was missing)

Go backend (options.go, model_config.go):

• Pass JSONSchema and ResponseFormat through gRPCPredictOpts to backends

vLLM backend (backend.py):

• Support both StructuredOutputsParams (vLLM latest) and GuidedDecodingParams (vLLM <=0.8.x) with graceful import fallback
• Handle three structured output modes with priority: JSONSchema > json_object > Grammar
• Fix missing import time and import json
• Remove phantom GuidedDecoding mapping

Docs (constrained_grammars.md):

• Updated compatibility notice to include vLLM
• Added vLLM section with examples for JSON schema, JSON object, and grammar

How It Works

User request (response_format or grammar parameter)
  → chat.go/completion.go: extracts raw JSON schema → config.JSONSchema
  → also generates GBNF grammar → config.Grammar (for llama.cpp compat)
  → options.go: passes both via gRPC PredictOptions
  → vLLM backend: uses StructuredOutputsParams/GuidedDecodingParams
  → llama.cpp backend: uses Grammar (GBNF) as before — no regression

Verification

• xgrammar (used by vLLM) explicitly follows the GBNF spec from llama.cpp, so grammar passthrough works
• GuidedDecodingParams.json and StructuredOutputsParams.json both accept JSON strings
• grammar_is_likely_lark() in vLLM correctly identifies GBNF as non-Lark (via ::= detection)
• No conflict with existing config.ResponseFormat usage (image endpoint b64_json goes through a different code path)

Notes for Reviewers

• The proto generated Go files are gitignored and built at compile time — only backend.proto is committed
• Compatible with both vLLM <=0.8.x (GuidedDecodingParams / guided_decoding) and latest (StructuredOutputsParams / structured_outputs)
• No regression for llama.cpp or other backends

Signed commits

• Yes, I signed my commits.

@mudler

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	278e7e2
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/69b454077c0aee00078d12a0
😎 Deploy Preview	https://deploy-preview-8806--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[fury0928]

GM @mudler
Would you review my PR?
Thank you for your time

[fury0928]

Hi @localai-bot would you review this PR?
Thank you

[fury0928]

Hi @richiejp @sozercan would you review my PR if you have a moment?
Thank you

[richiejp]

Thanks, see comments.

[richiejp]

Do we need a fallback? We usually pin the upstream version.

[fury0928]

Good point. I checked and vLLM is actually not pinned to a specific version — requirements-after.txt just lists vllm with no version constraint, and different platform builds (CPU/CUDA/ROCm) may end up with different vLLM versions.

That said, if the project plans to pin vLLM to a specific version, I'm happy to drop the fallback and target whichever API is current. Let me know which you'd prefer.

[richiejp]

OK, when you say newer versions, how new? If it's a very recent change then maybe we need this, otherwise we probably don't

[fury0928]

The rename happened in vLLM v0.8.x → latest. GuidedDecodingParams was renamed to StructuredOutputsParams and the corresponding SamplingParams field changed from guided_decoding to structured_outputs.

Since vLLM isn't pinned (requirements-after.txt just says vllm), builds can land on either version depending on when/how the image is built. If we pin to a specific version, I can drop the fallback and target that API directly — let me know which version to target.

Also in the latest push: I've refactored to use the Metadata map instead of new proto fields, as discussed.

[richiejp]

Just remove the fallback and if it passes CI we don't need to worry

[richiejp]

Also can you create e2e tests for this?

[fury0928]

Also can you create e2e tests for this?

Would you review again?

[detain]

this sounds like an important/cool feature

[richiejp]

Also can you create e2e tests for this?

Would you review again?

I still don't see any tests? You also need to rebase on master and see my other comment. Possibly related #9328 @mudler.

[mudler]

where this is consumed?

[mudler]

ditto here - I'm not sure what are we doing here, just importing , adding to a var.. and not used?

[detain]

this is irrelevant now since daa0272 and d676232 i think

[pos-ei-don]

Heads-up from testing on vLLM 0.23.0 (arm64 / CUDA 13): I hit the same GuidedDecodingParams → StructuredOutputsParams rename this PR handles. The existing request.Grammar path is actually silently broken on >= 0.23 — the from vllm.sampling_params import GuidedDecodingParams import fails, so HAS_GUIDED_DECODING is False and the whole guided-decoding block is skipped (response_format ignored, unconstrained output).

I opened #10339 with a minimal fix that adapts the existing path to the renamed class/field (with a fallback for older vLLM). It looked complementary to the native-json_schema-via-metadata direction here — I noticed this PR adds _structured_output_cls / _structured_output_field but doesn't consume them yet, so on 0.23 it wouldn't enforce on its own. Happy to consolidate however you'd prefer.

[richiejp]

I'm not sure if we need this one after #10343 ?

[pos-ei-don]

Hi Richie — these two PRs address different layers of the same area, but #10343 is functionally sufficient on its own:

#10343 is a compatibility hotfix. The import failure on vLLM ≥0.23 meant the existing request.Grammar path was silently bypassed and response_format: json_schema returned unconstrained text. Just verified on production (v4.5.0, vLLM 0.23.0): the request now returns schema-conformant JSON correctly. Six-line patch, kept the existing GBNF-conversion route alive.

#8806 adds a native JSON-Schema pass-through (new proto fields JSONSchema + ResponseFormat, Go endpoints fill them, backend uses them directly). That's a quality/efficiency improvement over the GBNF detour, plus it adds the missing import time for the video path — but it's not required for functional structured-output support.

So: if the goal is "structured outputs work" → #10343 already does that. If the goal is "vLLM's native structured_outputs gets the raw JSON-Schema instead of a GBNF translation" → #8806 is the right architectural step. That's a maintainer call — I don't have a stake either way. Just flagging that #8806 will need a rebase since backend.py has moved (my #10343/#10351 and the related volume patches are now upstream).

— Poseidon

[richiejp]

There is a regression risk here, but it's the same whenever we expose more backend functionality changing the semantics of requests.

[localai-bot]

Thanks for this PR, and sorry it sat for so long. There is real value in here (two genuine endpoint gaps you found and fixed), but the PR has drifted from its description after the rebase, and a large part of it is now dead code because master evolved in the meantime. Detailed review below.

Description vs. diff

The description talks about backend.proto fields 52/53, backend.py changes, and a missing import time fix. None of that is in the current diff, which touches 5 files: options.go, chat.go, completion.go, responses.go, and docs. Since this PR was opened, #10343 landed StructuredOutputsParams/GuidedDecodingParams handling in the vLLM backend on master, which covered the original core intent: vLLM already consumes request.Grammar (JSON schema or GBNF) today. Please rewrite the description (and probably the title, since the diff no longer touches any vLLM code) to match what the PR does now.

Issues

1. The new metadata keys are write-only (dead code). config.RequestMetadata["json_schema"] (chat.go, completion.go, responses.go) and metadata["response_format"] (options.go) are consumed by no backend. The vLLM backend only reads request.Grammar (backend.py:558-564), and llama.cpp's grpc-server only reads chat_template_kwargs and the reasoning keys from metadata (its data.contains("json_schema") at grpc-server.cpp:1881 reads the predict-options JSON, not metadata). So with this diff, vLLM still receives the GBNF-converted grammar exactly as on master, not a native JSON schema.

2. The keys leak into jinja templates. gRPCPredictOpts folds every metadata key (including RequestMetadata) into the chat_template_kwargs blob via ResolveChatTemplateKwargs (core/config/model_config.go:828). With this PR, every json_schema request to llama.cpp injects the entire raw schema string plus response_format as chat template kwargs. Related nit: config.RequestMetadata aliases the client's input.Metadata map (core/http/middleware/request.go:346, no copy), so the endpoints end up mutating the parsed client request in place.

3. The "raw JSON schema" is lossy. json.Marshal(d.JsonSchema.Schema) round-trips the schema through functions.Item, which only has Type and Properties (pkg/functions/function_structure.go:9-12). required, enum, additionalProperties, $defs, and everything else is silently dropped; your own doc example uses "required": ["name", "age"], which would be lost. If this plumbing stays, the schema must be marshaled from the raw config.ResponseFormatMap instead.

4. The Open Responses docs example is wrong. convertTextFormatToResponseFormat (responses.go:801) expects the flat Responses-API form ({"type": "json_schema", "name": ..., "schema": ...}) and wraps it under a json_schema key. The new docs show the nested chat-style form; feed that in and the JsonSchemaRequest unmarshal lands on the wrong level, Schema comes out empty, and a grammar is silently generated from an empty schema. Either fix the example or make the conversion accept both shapes.

What I'd like to keep

• completion.go: /v1/completions only handled json_object on master; adding json_schema grammar generation there is a real gap fix.
• responses.go: on master, text_format is converted into openAIReq.ResponseFormat and then nothing constrains generation (that internal request never passes back through the middleware). Generating cfg.Grammar there is a real fix, and the ordering is right: the function-calling grammar block runs later and correctly wins when tools are in play.
• The docs coverage for structured output on the Responses API, once the example is corrected and the vLLM wording is adjusted ("via GBNF/xgrammar" rather than "native JSON schema enforcement").

Smaller points

• The ~20-line marshal/unmarshal/grammar block is now copy-pasted in three endpoints; please extract a shared helper.
• The nested if err == nil chains silently swallow failures; a schema that fails to parse should at least log, since the user asked for constrained output and won't get it.
• completion.go re-marshals config.ResponseFormatMap although dat from the enclosing scope already holds exactly that.
• Overloading config.ResponseFormat (today an image-endpoint string, b64_json) to carry chat modes works because the config is a per-request copy, but it muddies the field's meaning. If the metadata plumbing goes, this should go too.
• No tests cover the two genuine fixes; a small Ginkgo test exercising json_schema on /v1/completions and text_format on /v1/responses would lock them in.

Suggested path forward

Trim the PR down to the completion.go and responses.go grammar fixes plus corrected docs, and drop the RequestMetadata["json_schema"] / metadata["response_format"] / config.ResponseFormat plumbing (or, if you want native schema pass-through for vLLM, add the consumer in backend.py in the same PR and pass the truly raw schema). CI is green and DCO is signed, so once trimmed this can move quickly.