Nest handoff history by default - #1996
Conversation
rm-openai
left a comment
There was a problem hiding this comment.
All of kaz's feedback seems right. In addition, I'd say two major things:
- This setting should be on the
Handoffobject - Since you're updating the default, this is a breaking change, so please update the changelog.
jhills20
left a comment
There was a problem hiding this comment.
moved to Handoff, added handoff history .py file for non extension parts of handoff filter, added changelog
| - By default handoff history is now packaged into a single assistant message instead of exposing the raw user/assistant turns, giving downstream agents a concise, predictable recap | ||
| - The existing single-message handoff transcript now by default starts with "For context, here is the conversation so far between the user and the previous agent:" before the `<CONVERSATION HISTORY>` block, so downstream agents get a clearly labeled recap |
There was a problem hiding this comment.
we;ve release 0.5.0 so you'll need to move this to a 0.6.0 section
| filter_name = getattr(input_filter, "__qualname__", repr(input_filter)) | ||
| from_agent = getattr(agent, "name", agent.__class__.__name__) | ||
| to_agent = getattr(new_agent, "name", new_agent.__class__.__name__) | ||
| logger.debug( | ||
| "Filtering handoff inputs with %s for %s -> %s", | ||
| filter_name, | ||
| from_agent, | ||
| to_agent, | ||
| ) |
There was a problem hiding this comment.
weird syntax - also can you use f-strings and get rid of getattr?
| __all__ = [ | ||
| "remove_all_tools", | ||
| "nest_handoff_history", | ||
| "default_handoff_history_mapper", | ||
| ] |
There was a problem hiding this comment.
can remove this, not super useful
|
Are you aware that this is duplicating everything by default? For example this code: import asyncio
from agents import Agent, Runner, function_tool
@function_tool
def random_number() -> int:
return 4
agent2 = Agent(name='agent2', instructions='Return double the number')
agent1 = Agent(name='agent1', tools=[random_number], handoffs=[agent2])
asyncio.run(Runner.run(agent1, input='Generate a random number then, hand off to agent2.'))sends all of this: [
{
"role": "assistant",
"content": "For context, here is the conversation so far between the user and the previous agent:\n<CONVERSATION HISTORY>\n1. user: Generate a random number then, hand off to agent2.\n2. function_call: {\"arguments\": \"{}\", \"call_id\": \"call_Wsu7AtBXA186HIw3qOLST3KJ\", \"name\": \"random_number\", \"id\": \"fc_0001f0f3cb0a689000691db920bd54819d805de7a682ee3d08\", \"status\": \"completed\"}\n3. function_call_output: {\"call_id\": \"call_Wsu7AtBXA186HIw3qOLST3KJ\", \"output\": \"4\"}\n4. function_call: {\"arguments\": \"{}\", \"call_id\": \"call_9xPh03L6GqirBstlCgFE9Oqn\", \"name\": \"transfer_to_agent2\", \"id\": \"fc_0001f0f3cb0a689000691db922b564819dbf86d6da3014bb36\", \"status\": \"completed\"}\n5. function_call_output: {\"call_id\": \"call_9xPh03L6GqirBstlCgFE9Oqn\", \"output\": \"{\\\"assistant\\\": \\\"agent2\\\"}\"}\n</CONVERSATION HISTORY>"
},
{
"arguments": "{}",
"call_id": "call_Wsu7AtBXA186HIw3qOLST3KJ",
"name": "random_number",
"type": "function_call",
"id": "fc_0001f0f3cb0a689000691db920bd54819d805de7a682ee3d08",
"status": "completed"
},
{
"call_id": "call_Wsu7AtBXA186HIw3qOLST3KJ",
"output": "4",
"type": "function_call_output"
},
{
"arguments": "{}",
"call_id": "call_9xPh03L6GqirBstlCgFE9Oqn",
"name": "transfer_to_agent2",
"type": "function_call",
"id": "fc_0001f0f3cb0a689000691db922b564819dbf86d6da3014bb36",
"status": "completed"
},
{
"call_id": "call_9xPh03L6GqirBstlCgFE9Oqn",
"output": "{\"assistant\": \"agent2\"}",
"type": "function_call_output"
}
]Is it better in other cases? |
|
I've tested this in our environment and it looks good to me. |
|
Ever since this change, I keep getting this error: When I check traces, the tool calls are still there but with empty responses. Any idea what gives? Does the For reference this is my setup: The error occurs only when a previous response id is provided. |
|
Hi James, This PR is a good example of a "threshold evals still pass, behavior distribution shifts" change. Making handoff-history nesting the default is the right call architecturally, but it changes what every downstream agent sees as context on every handoff. I actually ran this end-to-end against the real SDK on both sides. The agents.RunConfig() on version 0.5.1 has no nest_handoff_history field at all, whereas on 0.6.0 it defaults to True. I then built a real triage-to-specialist handoff with a manufactured 3-turn prior conversation and called Runner.run() for real (using one gpt-4o-mini call per side). Pre-fix (0.5.1): The result.to_input_list() keeps the 3 prior turns as 3 separate flat items. Post-fix (0.6.0): Those same 3 turns collapse into a single role: "assistant" item with a summary block. They are genuinely nested rather than just reordered. The final output was identical on both sides, so this is a clean, isolated internal-representation change exactly as advertised. As a minor aside, your merge commit calls it a "developer-role" summary, but the shipped code and what I actually observed both use role: "assistant". This is probably just stale commit-message wording. That is the exact class of change agent-eval (agent_regress) is built to catch at scale. Instead of asserting individual outputs pass or fail, it runs an agent N times before and after a change and performs a real statistical comparison for a REGRESSED, STABLE, or IMPROVED verdict on the distribution. This would be useful here since two people (alexmojaki and KthProg) already hit real downstream regressions from this default flip before reaching for nest_handoff_history=False. My OpenAI Agents SDK integration used to be flatly broken. It called a nonexistent agent.run() and crashed on the first call. That is fixed now, and I verified it against the real installed SDK for this check. I am genuinely curious whether a distributional before-and-after comparison on a handoff-nesting toggle specifically would have caught anything useful pre-merge. Let me know what you think if you get a chance to try it. You can find the repository here: https://github.com/RudrenduPaul/agent-eval Best, |
Summary
nest_handoff_historyflag toRunConfigand call a new helper that condenses the prior transcript into a developer-role summary when handing offTesting
uv run pytest tests/test_extension_filters.pyuv run pytest tests/test_agent_runner.py -k handoffuv run pytest tests/test_agent_runner_streamed.py -k handoffhttps://chatgpt.com/codex/tasks/task_i_68ff73bda0f4832496f3d1fa9103905f