.NET: AG-UI: Multi-turn tool calls replay produces invalid OpenAI tool_call history

[emilmuller]

A prompt triggering multiple tool calls at once, produces a history from the frontend like:

assistant(tool_call: call_1)
assistant(tool_call: call_2)
assistant(tool_call: call_3)
tool_result(call_1)
tool_result(call_2)
tool_result(call_3)

When this history is sent back to the model on the next turn (additional user message), the OpenAI tools protocol rejects it:

System.ClientModel.ClientResultException: HTTP 400
(invalid_request_error: )
Parameter: messages.[3].role

An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_8oepC2nxsf2P1h7CYhIlBBmK at
OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options)
at OpenAI.Chat.ChatClient.CompleteChatAsync(BinaryContent content, RequestOptions options)
at OpenAI.Chat.ChatClient.<>c__DisplayClass19_0.
<<CompleteChatStreamingAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---

In other words, each assistant message that performs a tool call must be immediately followed by its matching tool result.

The correct sequence should look like:

assistant(tool_call: call_1)
tool_result(call_1)
assistant(tool_call: call_2)
tool_result(call_2)
assistant(tool_call: call_3)
tool_result(call_3)

[javiercn]

@emilmuller thanks for contacting us.

Yes, this is an issue I think with ParallelToolCalls. It can be workaround by disabling them (I think we do that in one of our samples).

Thanks for submitting a fix for it too. One question that I have is, are you seeing the behavior when you use another client or when you use the C# client?

one thing that I'm wondering here is if we can either fix the issue at the time we generate events/tool calls or if this is something we need to fix on the C# client (and other clients handle correctly).

I'm hesitant about making changes that require us copying the entire list of messages to process it as that becomes expensive quickly. If this is just a matter of us generating the wrong message list on the .NET client, I'd rather fix that to make it compliant and just continue failing if someone sends a non-compliant payload.

If we must fix this on the server-side, I think we want to do something more scoped that iterates over the existing list and only starts copying messages if it detects this scenario.

[emilmuller]

Hi @javiercn and thank you for looking into this.

Sorry, I left out some details. I'm actually using the @ag-ui/client npm pkg. This is the culprit library causing the re-ordering.

I think it's a difficult question to decide where the fix should be (on the server or client side). Both sides should respect the protocol.

While there's clear specification of how messages and tool calls/results should be represented, there is no explicit text that mandates that implementations must preserve exact chronological ordering of messages in the messages array.
However, the practical example they provide shows exactly the interleaved ordering: https://docs.ag-ui.com/concepts/messages#practical-example that .NET assumes.

Since ordering is not strictly mandated, the frontend implementation that reorders messages is not explicitly disallowed by the spec, though it becomes incompatible with downstream tools protocols (OpenAI tools).

Maybe an issue should be raised with https://github.com/ag-ui-protocol/ag-ui to include this in the specs, and then fix it in @ag-ui/client.

Until then, I find it ambiguous whether the .NET version is in violation or not.

[FanyuYe]

Dear developers,

I am observing the same issue on my blazor server app that integrats the MAF AG-UI package. When using a model with parallel tool calling capability, from the aspire tracing, I can see the single tool_calls (requires calling 2 tools) emits from the model has been transformed into 2 saparate tool_calls request in AG-UI events (which is expected), however I assume the implementation should map the tool call results back to llm endpoint as a single response containing both results to fix this issue? Which is not the case right now.

[gsubiran]

Facing the same issue with the latest version of MAF (1.0.0-rc3) when using AG-UI with parallel tool calls.

As a workaround we can disable ParallelToolCalls, but that obviously removes a useful capability.

@markwallace-microsoft @javiercn
Do you think this issue is still related to the AG-UI message replay logic discussed here?

Also wondering if the fix proposed in PR #2700 is still the intended direction, or if there is another recommended workaround / configuration for now.

Happy to provide a minimal repro if that would help.

Edit:
This issue only occurs with ChatCompletion. I've switched to OpenAI Responses, and it works fine without forcing sequential tool calls. This makes sense because, with Responses, the chat history is managed server-side by the model, unlike ChatCompletion where it is managed by the framework.