fix(openresponses): populate Content and accept bare {role,content} items (#10039) - #10040
Merged
mudler merged 3 commits intoMay 28, 2026
Conversation
…tems (mudler#10039) Fixes mudler#10039 — `/v1/responses` silently returned empty output on any model whose YAML doesn't include a Go-side `template.chat_message` block. Three cooperating bugs: * `convertORInputToMessages` populated only `StringContent` for string input and for the `input.Instructions` system message, leaving the `Content` (any) field nil. * `TemplateMessages` gated all fallback content-rendering branches on `Content != nil && StringContent != ""` — but every branch in that function consumes `StringContent`, not `Content`. The `&&` silently dropped messages that had StringContent set and Content nil, producing an empty prompt that the 5× empty-retry guard then turned into a 200 OK with `output: []`. * The array-input branch of `convertORInputToMessages` dispatched on `itemMap["type"]` with no default, dropping bare `{role, content}` items emitted by the OpenAI Python SDK helper `client.responses.create(input=[{...}])`. Fix: * Set both `Content` and `StringContent` in the two openresponses message-construction sites that only set one. * Treat a bare `{role, content}` item (no `type`) as `type: "message"` for OpenAI-SDK compatibility. * Gate `TemplateMessages` fallback rendering on `StringContent != ""`, which is what every downstream branch in that function actually reads. Regression test added to `evaluator_test.go` covering the fallback path (no `ChatMessage` template) with a StringContent-only message, both with and without a role mapping.
…r#10039) Add regression tests for the two seams the original fix touched but left uncovered: * convertORInputToMessages must populate both Content and StringContent for plain string input and for bare {role, content} array items (the OpenAI SDK shape that omits the type discriminator). Both are functional reds against the pre-fix code. * Messages.ToProto reads Content, not StringContent — this is the path UseTokenizerTemplate backends (imported GGUFs) take. The cases pin that contract so a future regression on the producer side is caught. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-7 [Claude Code]
mudler
enabled auto-merge (squash)
May 28, 2026 07:02
…tmessage-template
mudler
approved these changes
May 28, 2026
Owner
|
thanks @Anai-Guo , looking good here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #10039.
/v1/responsessilently returned empty output on any model whose YAML didn't include a Go-sidetemplate.chat_messageblock — the handler loggedPrompt (before templating) prompt="", hit the 5× empty-retry guard incore/http/endpoints/openai/inference.go:129, and returned a 200 OKresponse.completedwithoutput: [].Three cooperating bugs, all small and orthogonal:
core/http/endpoints/openresponses/responses.go:97,:302convertORInputToMessagespopulated onlyStringContentfor the string-input case and for theinput.Instructionssystem message —Content(theanyfield) stayednil.core/templates/evaluator.go:114contentExists := i.Content != nil && i.StringContent != ""silently dropped messages with StringContent set but Content nil. Every fallback branch in this function readsi.StringContent, noti.Content.core/http/endpoints/openresponses/responses.go:312itemMap["type"]with no default. The OpenAI Python SDK helperclient.responses.create(input=[{"role":"user","content":"…"}])sends items without atypediscriminator — those fell through and were dropped.Fix
ContentandStringContentin the two openresponses sites that only set one.{role, content}item (notype) astype: "message"for OpenAI-SDK compatibility. Items that are neither known typed items nor a{role, content}shape still fall through unchanged (no behavior change for current valid inputs).TemplateMessagesfallback rendering onStringContent != "". This is the most defensive form because every downstream branch in that function (fmt.Sprint(r, i.StringContent),content = fmt.Sprint(i.StringContent), therole == "system"suppression check) readsStringContent. Switching to||would have introduced a regression for the (currently impossible) shapeContent!=nil && StringContent==""since the role prefix would then render with empty content.Notes for review
evaluator_test.gocases all populateStringContentonly (Content stays nil), but they pass because they configureTemplateConfig.ChatMessage— which is an earlier branch that consumesStringContentdirectly and short-circuits beforecontentExistsis read. The new regression test exercises the fallback path (noChatMessagetemplate configured) with a StringContent-only message — the exact shape/v1/responsesproduces./v1/chat/completions— its messages already go through middleware that populates bothContentandStringContent, so neither fix B nor fix C affects that path.Test plan
go test ./core/templates/...(regression tests added —fallback path with StringContent-only message, both with and without role mapping)Templates / chat message ChatMLandchat message llama3suites still pass (touched line uses the sameStringContentsource).template.chat_message) — out of scope for this PR's CI sandbox, but the unit test pins the exact branch the bug travelled through.🤖 Generated with Claude Code