Skip to content

fix(openai): merge custom-provider system prompts#2382

Open
pandego wants to merge 1 commit into
docker:mainfrom
pandego:fix/2327-merge-system-messages
Open

fix(openai): merge custom-provider system prompts#2382
pandego wants to merge 1 commit into
docker:mainfrom
pandego:fix/2327-merge-system-messages

Conversation

@pandego
Copy link
Copy Markdown
Contributor

@pandego pandego commented Apr 11, 2026

Summary

  • merge consecutive system and user messages for custom openai_chatcompletions providers
  • align custom OpenAI-compatible providers with the existing DMR prompt-normalization behavior
  • add regression coverage so custom providers collapse multiple system prompts while the native OpenAI path stays unchanged

Testing

  • docker run --rm -v "$PWD":/src -w /src golang:1.26 sh -lc '/usr/local/go/bin/gofmt -w pkg/model/provider/openai/client.go pkg/model/provider/openai/client_test.go && /usr/local/go/bin/go test ./pkg/model/provider/openai ./pkg/model/provider/oaistream -run "TestConvertMessages_MergesConsecutiveSystemMessagesForCustomProviders|TestConvertMessages_PreservesConsecutiveSystemMessagesForOpenAIProvider|TestConvertMessages|TestMergeConsecutiveMessages" -count=1'

Closes #2327

@pandego pandego requested a review from a team as a code owner April 11, 2026 16:23
aheritier
aheritier previously approved these changes May 6, 2026
Copy link
Copy Markdown
Contributor

@aheritier aheritier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Clean, minimal fix that reuses existing infrastructure (isCustomProvider, MergeConsecutiveMessages). Both paths are well-tested and CI passes.

One nit: the em dash → hyphen change on line 51 of the test file is unrelated noise — consider keeping commits atomic in the future.

@aheritier aheritier added kind/fix PR fixes a bug (maps to fix: commit prefix) area/providers/openai For features/issues/fixes related to the usage of OpenAI models priority:medium Normal priority, standard sprint work labels May 6, 2026
@pandego pandego force-pushed the fix/2327-merge-system-messages branch from 23681e3 to 578a48d Compare May 7, 2026 11:23
@aheritier aheritier added effort:small Isolated change, clear solution, single area go Pull requests that update go code and removed priority:medium Normal priority, standard sprint work labels May 7, 2026
@aheritier
Copy link
Copy Markdown
Contributor

needs to be rebased (at the minimum). Moving it to draft

@aheritier aheritier added the pr:needs-rebase PR has merge conflicts or is out of date with main label May 12, 2026
@aheritier aheritier marked this pull request as draft May 12, 2026 21:09
@aheritier aheritier removed effort:small Isolated change, clear solution, single area go Pull requests that update go code labels May 12, 2026
Signed-off-by: pandego <7780875+pandego@users.noreply.github.com>
@pandego pandego force-pushed the fix/2327-merge-system-messages branch from 578a48d to ccf7977 Compare May 14, 2026 11:58
@pandego pandego marked this pull request as ready for review May 14, 2026 11:58
@pandego
Copy link
Copy Markdown
Contributor Author

pandego commented May 14, 2026

Rebased this on current main and resolved the conflicts.

Focused validation passes:

go test ./pkg/model/provider/openai ./pkg/model/provider/oaistream -run 'TestConvertMessages_MergesConsecutiveSystemMessagesForCustomProviders|TestConvertMessages_PreservesConsecutiveSystemMessagesForOpenAIProvider|TestConvertMessages|TestMergeConsecutiveMessages' -count=1
go test ./pkg/model/provider/openai ./pkg/model/provider/oaistream -count=1

@aheritier
Copy link
Copy Markdown
Contributor

/review

@aheritier aheritier requested review from dgageot and rumpl May 17, 2026 19:03
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The fix correctly introduces a convertMessages wrapper that applies MergeConsecutiveMessages only for custom OpenAI-compatible providers, aligning them with the existing DMR prompt-normalization behaviour. Logic is clean and tests cover both the merging path and the native-OpenAI preservation path.

No high- or medium-severity issues found. Three minor (low) observations are attached as inline comments.

// the DMR provider behavior.
func convertMessages(ctx context.Context, cfg *latest.ModelConfig, id modelsdev.ID, store *modelsdev.Store, messages []chat.Message) []openai.ChatCompletionMessageParamUnion {
openaiMessages := oaistream.ConvertMessages(ctx, messages, id, store)
if isCustomProvider(cfg) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] isCustomProvider covers all api_type values, including openai_responses

isCustomProvider returns true for any config whose ProviderOpts carries an api_type field — including openai_responses. This means MergeConsecutiveMessages is technically reachable for Responses-API custom providers too, even though the Responses-API path (CreateResponseStream) never calls convertMessages today.

This is harmless at runtime, but if a future refactor routes Responses-API providers through convertMessages the merging will kick in unexpectedly. Consider restricting the guard to Chat Completions api_type values only, or add a comment explaining the intentional scope.


result := convertMessages(t.Context(), &latest.ModelConfig{}, modelsdev.ID{}, nil, messages)
require.Len(t, result, 3)
assert.NotNil(t, result[0].OfSystem)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Test only asserts OfSystem != nil, not the actual content of each preserved message

TestConvertMessages_PreservesConsecutiveSystemMessagesForOpenAIProvider correctly asserts that three messages are returned and that the first is a system message, but it never checks the text of result[0] or result[1]. If the messages were reordered the test would still pass. Consider adding:

assert.Equal(t, "System 1", result[0].OfSystem.Content.OfString.Value)
assert.Equal(t, "System 2", result[1].OfSystem.Content.OfString.Value)

func convertMessages(ctx context.Context, cfg *latest.ModelConfig, id modelsdev.ID, store *modelsdev.Store, messages []chat.Message) []openai.ChatCompletionMessageParamUnion {
openaiMessages := oaistream.ConvertMessages(ctx, messages, id, store)
if isCustomProvider(cfg) {
return oaistream.MergeConsecutiveMessages(openaiMessages)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Merged system messages joined with \n; consider \n\n for markdown fidelity

When MergeConsecutiveMessages concatenates consecutive system-message strings it uses a single \n as the separator. The test fixture includes a message with ## Custom Shell Tools\n\n### execute_command — merging with a single newline can collapse the blank line that separates markdown sections, potentially mangling the structure the model sees.

Using \n\n as the separator would preserve the original paragraph breaks. This is a design trade-off, not a hard bug, but worth an explicit decision and a comment if single-newline is intentional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/openai For features/issues/fixes related to the usage of OpenAI models kind/fix PR fixes a bug (maps to fix: commit prefix) pr:needs-rebase PR has merge conflicts or is out of date with main

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jinja Exception: System message must be at the beginning with custom provider, docker model runner and qwen 3.5 family models

3 participants