Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions examples/agent_patterns/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,20 @@ See the [`input_guardrails.py`](./input_guardrails.py) and [`output_guardrails.p
You can pause runs for manual approval before executing sensitive tools. This is useful for operations like sending money, deleting data, or running destructive commands.

See [`human_in_the_loop.py`](./human_in_the_loop.py) for the base approval flow and [`human_in_the_loop_custom_rejection.py`](./human_in_the_loop_custom_rejection.py) for run-level tool error formatting when approvals are rejected.

## Handling ModelBehaviorError

`ModelBehaviorError` is raised when the model does something unexpected — calling a tool that doesn't exist, returning malformed JSON, or failing schema validation. Rather than always letting the exception crash your run, you have several options.

See [`model_behavior_error_handling.py`](./model_behavior_error_handling.py) for three patterns:
1. **Return a safe fallback value** — useful for best-effort tasks.
2. **Log and re-raise with enriched context** — useful in production pipelines that need structured error reporting.
3. **Retry with a fallback agent** — hand the same task to a more reliable model when the primary one misbehaves.

## Self-healing agent

A more advanced pattern: when `ModelBehaviorError` is caught, the error description is appended to the conversation as a correction and the run is retried, giving the model a chance to fix its own mistake without a full restart.

This is useful when the task is long-running, restarting from scratch is expensive, or you are using a smaller model that occasionally calls wrong tools.

See [`self_healing_agent.py`](./self_healing_agent.py) for an example.
158 changes: 158 additions & 0 deletions examples/agent_patterns/model_behavior_error_handling.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
"""Handling ModelBehaviorError — three patterns.

``ModelBehaviorError`` is raised when the model does something the SDK cannot
handle: calling a tool that does not exist, returning malformed JSON for a
structured output, or producing an output that fails schema validation.

This file demonstrates three practical ways to deal with it, in increasing
order of complexity.

Pattern 1 — Ignore and return a fallback value.
Useful when the task is best-effort and a graceful default is acceptable.

Pattern 2 — Log and re-raise with context.
Useful in production pipelines where you want structured error reporting
without losing the original exception.

Pattern 3 — Retry with a fallback agent.
When the primary model misbehaves, hand the same task to a more reliable
(often larger) fallback model.

Run with:
python -m examples.agent_patterns.model_behavior_error_handling
"""

import asyncio
from dataclasses import dataclass

from agents import Agent, ModelBehaviorError, Runner, function_tool


@function_tool
def get_stock_price(ticker: str) -> str:
"""Return a (mocked) stock price for the given ticker symbol.

Args:
ticker: Stock ticker symbol, e.g. AAPL.
"""
prices = {"AAPL": "$189.30", "GOOGL": "$175.20", "MSFT": "$415.50"}
return prices.get(ticker.upper(), f"Ticker '{ticker}' not found.")


primary_agent = Agent(
name="Stock Assistant",
instructions=(
"You are a stock price assistant. Use the get_stock_price tool to answer questions. "
"Do not call any other tools."
),
tools=[get_stock_price],
)

# A more conservative fallback agent using explicit instructions to reduce errors.
fallback_agent = Agent(
name="Stock Assistant (fallback)",
instructions=(
"You are a stock price assistant. You have ONE tool: get_stock_price(ticker). "
"Call it exactly once with the ticker symbol the user mentioned. "
"Do not call any other tool. Do not make up data."
),
tools=[get_stock_price],
)


# ---------------------------------------------------------------------------
# Pattern 1 — Return a safe default on error
# ---------------------------------------------------------------------------


async def pattern_1_fallback_value(task: str) -> str:
"""Run the agent; return a safe default string if ModelBehaviorError occurs."""
try:
result = await Runner.run(primary_agent, task)
return result.final_output
except ModelBehaviorError as exc:
print(f"[pattern-1] ModelBehaviorError: {exc.message}")
return "Sorry, I could not retrieve that information right now."


# ---------------------------------------------------------------------------
# Pattern 2 — Log and re-raise with enriched context
# ---------------------------------------------------------------------------


@dataclass
class EnrichedModelError(Exception):
original: ModelBehaviorError
task: str
agent_name: str

def __str__(self) -> str:
return (
f"Agent '{self.agent_name}' failed on task: {self.task!r}\n"
f"Reason: {self.original.message}"
)


async def pattern_2_log_and_reraise(task: str) -> str:
"""Run the agent; re-raise ModelBehaviorError with structured context."""
try:
result = await Runner.run(primary_agent, task)
return result.final_output
except ModelBehaviorError as exc:
raise EnrichedModelError(
original=exc,
task=task,
agent_name=primary_agent.name,
) from exc


# ---------------------------------------------------------------------------
# Pattern 3 — Retry with a fallback agent
# ---------------------------------------------------------------------------


async def pattern_3_fallback_agent(task: str) -> str:
"""Try the primary agent; fall back to a more reliable agent on error."""
try:
result = await Runner.run(primary_agent, task)
print("[pattern-3] Primary agent succeeded.")
return result.final_output
except ModelBehaviorError as exc:
print(f"[pattern-3] Primary agent failed ({exc.message}), trying fallback agent...")
result = await Runner.run(fallback_agent, task)
print("[pattern-3] Fallback agent succeeded.")
return result.final_output


# ---------------------------------------------------------------------------
# Demo
# ---------------------------------------------------------------------------


async def main() -> None:
task = "What is the current price of AAPL stock?"

print("=" * 60)
print("Pattern 1 — fallback value")
print("=" * 60)
output = await pattern_1_fallback_value(task)
print(f"Output: {output}\n")

print("=" * 60)
print("Pattern 2 — log and re-raise")
print("=" * 60)
try:
output = await pattern_2_log_and_reraise(task)
print(f"Output: {output}\n")
except EnrichedModelError as exc:
print(f"Caught enriched error:\n{exc}\n")

print("=" * 60)
print("Pattern 3 — fallback agent")
print("=" * 60)
output = await pattern_3_fallback_agent(task)
print(f"Output: {output}\n")


if __name__ == "__main__":
asyncio.run(main())
141 changes: 141 additions & 0 deletions examples/agent_patterns/self_healing_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
"""Self-healing agent pattern.

When a model calls a tool that does not exist, or produces malformed JSON for
a structured output, the SDK raises ``ModelBehaviorError``. The default
behaviour is to let that exception propagate and crash the run.

This example shows a loop that catches ``ModelBehaviorError``, appends the
error text to the conversation as a correction, and retries — giving the
model a chance to fix its own mistake before giving up.

Crucially, the retry resumes from ``exc.run_data.new_items``, which contains
all the turns the agent completed before the error occurred. This avoids
re-running prior work (and re-triggering tool side effects) on every heal.

The pattern is useful when:
- You are using a smaller/cheaper model that occasionally calls wrong tools.
- You want resilience without replacing the model or hard-coding fallbacks.
- The task is long-running and restarting from scratch is expensive.

Run with:
python -m examples.agent_patterns.self_healing_agent
"""

import asyncio
from typing import Union

from agents import Agent, ModelBehaviorError, Runner, TResponseInputItem, function_tool

MAX_SELF_HEALS = 3


@function_tool
def add(a: int, b: int) -> int:
"""Return the sum of two integers.

Args:
a: First integer.
b: Second integer.
"""
return a + b


@function_tool
def multiply(a: int, b: int) -> int:
"""Return the product of two integers.

Args:
a: First integer.
b: Second integer.
"""
return a * b


agent = Agent(
name="Math Assistant",
instructions=(
"You are a math assistant. Use the available tools to answer questions. "
"Only call tools that exist: add, multiply."
),
tools=[add, multiply],
)


async def run_with_self_healing(task: str) -> str:
"""Run an agent task, retrying up to MAX_SELF_HEALS times on ModelBehaviorError.

On each retry the completed turns from ``exc.run_data.new_items`` are
preserved and a correction note is appended, so the model continues from
where it left off rather than restarting the full task.

Args:
task: The initial user message to send to the agent.

Returns:
The final output string from the agent.

Raises:
ModelBehaviorError: If the model keeps misbehaving after all retries.
"""
input: Union[str, list[TResponseInputItem]] = task
heals_remaining = MAX_SELF_HEALS

while True:
try:
result = await Runner.run(agent, input)
return result.final_output
except ModelBehaviorError as exc:
if heals_remaining <= 0:
print(f"[self-heal] Giving up after {MAX_SELF_HEALS} attempts.")
raise

heals_remaining -= 1
print(
f"[self-heal] ModelBehaviorError caught "
f"({MAX_SELF_HEALS - heals_remaining}/{MAX_SELF_HEALS}): {exc.message}"
)

# Build the correction message to append.
correction: TResponseInputItem = {
"role": "user",
"content": (
f"Your previous response caused an error: {exc.message}\n"
"Please correct your approach and try again using only the tools available."
),
}

# Resume from the completed turns rather than restarting from scratch.
# exc.run_data.new_items holds every RunItem produced before the error,
# so prior tool calls and their results are preserved and not re-run.
completed_items: list[TResponseInputItem] = (
[item.to_input_item() for item in exc.run_data.new_items]
if exc.run_data is not None
else []
)

if completed_items:
input = completed_items + [correction]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the original run input on retries

When a failure happens after at least one completed turn, new_items contains only the items produced by this run while RunErrorDetails.input holds the user input separately, so this retry starts the next run with assistant/tool items plus the correction but without the original task. In this version the fresh issue is that the attempted fix now preserves generated turns but drops the run input, and on later heals can also drop prior retry context; build the retry from exc.run_data.input plus the converted new_items before appending the correction.

Useful? React with 👍 / 👎.

print(
f"[self-heal] Resuming from {len(completed_items)} completed turn(s) "
"with correction appended..."
)
else:
# No completed turns to preserve — fall back to the original task.
input = f"{task}\n\n{correction['content']}"
print("[self-heal] No prior turns to resume from. Retrying from scratch...")


async def main() -> None:
task = "What is (7 + 3) multiplied by 4?"

print(f"Task: {task}")
print("-" * 60)

output = await run_with_self_healing(task)

print("-" * 60)
print(f"Answer: {output}")


if __name__ == "__main__":
asyncio.run(main())