Python: [Bug]: AG-UI history replay can send invalid assistant/tool sequence to OpenAI (tool_calls without matching tool messages)

[adityanile]

Description

When using agent_framework.ag_ui with persisted thread history, later requests can intermittently fail with OpenAI/Azure validation errors:

An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'.

The issue appears related to replay/history reconstruction logic where assistant tool-call messages and tool-result messages become inconsistently paired during outbound payload generation.

Code Sample

Error Messages / Stack Traces

{
  "error": {
    "message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_xxx",
    "type": "invalid_request_error",
    "param": "messages.[N].role"
  }
}

Package Versions

agent_framework_ag_ui: 1.0.0b260507

Python Version

No response

Additional Context

Observed History Behavior

Persisted history files may contain both:

• assistant tool-call messages
• matching tool-result messages

However, during replay/reconstruction, outbound model payloads can still contain:

• unmatched assistant tool_calls
• reordered tool messages
• duplicated call entries
• missing tool-result messages

This eventually causes provider-side validation failures.

---

Expected Behavior

Framework-generated outbound payloads should always satisfy provider message contracts:

1. Every assistant tool_call must have a matching tool-result message.
2. Tool messages must appear immediately after their originating assistant tool-call message.
3. No orphan tool messages should exist.
4. History replay and deduplication should remain deterministic under:
   • retries
   • interruptions
   • resumed execution flows

---

Actual Behavior

Under some replay/history scenarios, invalid message sequences are sent to the model provider, resulting in hard 400 failures and unusable conversation threads.

---

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: _sanitize_tool_history() in python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py has two defects. (1) Line 76 unconditionally overwrites pending_tool_call_ids when a new assistant tool_call message is encountered, so consecutive assistant messages with tool_calls leave the first one's calls orphaned without synthetic results. (2) The function returns at line 197 without injecting synthetic results for any remaining pending_tool_call_ids, so history ending with an assistant tool_call message produces orphaned calls. Minimal repro: call _sanitize_tool_history with [assistant(tool_calls=[call_1]), assistant(tool_calls=[call_2]), user("hi")] and verify call_1 has no matching tool result in the output.

• Failing test: python/packages/ag-ui/tests/ag_ui/test_issue_5855_tool_history_replay.py
• Files examined: python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py, python/packages/ag-ui/tests/ag_ui/test_message_adapters.py, python/packages/ag-ui/pyproject.toml
• Tests run: python/packages/ag-ui/tests/ag_ui/test_issue_5855_tool_history_replay.py
• Reported version: 1.0.0b260507
• Current version: 1.0.0rc1

[norika1207-lab]

The history sanitizer should probably enforce the provider message contract as an invariant, not as a best-effort cleanup.

Before any outbound OpenAI/Azure payload is sent, the reconstructed sequence should be validated as:

• every assistant tool_call.id has exactly one following tool message
• tool messages are adjacent to the assistant message they answer
• consecutive assistant tool-call messages are either merged intentionally or closed with synthetic/error tool results before the next assistant/user message
• history ending with pending tool calls is represented as suspended/incomplete, not replayed as a valid conversation

The dangerous case is synthetic repair hiding a real incomplete run. It can prevent a 400, but the runtime should still expose that a tool result was missing or reconstructed. Otherwise the next model call may continue from a transcript that is schema-valid but operationally false.

[lordlinus]

Still reproduces on the latest release agent-framework-ag-ui==1.0.0rc4 (with agent-framework-openai==1.8.1) — _message_adapters._sanitize_tool_history() is unchanged from the beta this was filed against, so a replayed assistant function_call with no matching tool result still produces:

400 - An assistant message with 'tool_calls' must be followed by tool messages
responding to each 'tool_call_id'. The following tool_call_ids did not have
response messages: call_xxx

Two observations from testing against Azure OpenAI Chat Completions on rc4:

1. The simple single-orphan case (one interrupted assistant tool_call followed by a user message) now passes in my runs — so partial protection seems to be working for that shape.
2. The failure persists for the multi/edge shapes described in the triage (consecutive assistant tool_call messages; history ending on an assistant tool_call), matching the two defects noted at _sanitize_tool_history() (~line 76 unconditional overwrite of pending_tool_call_ids, and ~line 197 early return without injecting synthetic results).

Related but distinct: the HITL approve → replay variant (an approved approval_mode="always_require" tool whose executed result isn't journaled back into the snapshot) is #6513.

Happy to help validate a fix.