Skip to content

Python: Allow branching from hosted Foundry conversations #7526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Chinedum Echeta (cecheta) wants to merge 4 commits into
base: main
Choose a base branch
from
+360 −94
Open

Python: Allow branching from hosted Foundry conversations #7526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6aae03d
Python: Allow branching from hosted Foundry conversations
cecheta Aug 5, 2026
02c34af
Fix formatting in README.md for clarity on snapshot filenames
cecheta Aug 5, 2026
95eb679
Reject previous response ID when a conversation ID is provided
cecheta Aug 5, 2026
e4d58cb
Python: Enhance error handling and checkpoint management in Responses…
cecheta Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 21 additions & 13 deletions python/packages/foundry_hosting/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@ This package provides the integration of Agent Framework agents and workflows wi
agents in addition to the Responses provider's message history. By default it
uses the experimental `FoundrySessionStore` under `/.sessions` when hosted and
an in-memory `SessionStore` locally. Hosted snapshots are partitioned by the
Agent Server request context's platform user ID. Snapshot filenames use the
Responses `conversation_id` or `response_id`, depending on the continuation
mode.
Agent Server request context's platform user ID. Snapshot filenames use
Responses `response_id` values, with an additional `conversation_id` snapshot
that points to the latest state of each stored conversation.

Foundry's session file API exposes the hosted `$HOME` directory as `/`, so the
API path `/.sessions` is stored on disk at `$HOME/.sessions`.

Workflow agents continue to use their existing checkpoint storage layout.
Workflow agents use the same continuation model for their checkpoints: every
turn is stored under its `response_id`, and stored conversations also maintain
a `conversation_id` checkpoint alias for their latest turn.

## Foundry session isolation

Expand All @@ -33,11 +35,14 @@ A Foundry session controls hosted compute and filesystem lifetime and may host
multiple users and Responses conversations. The Foundry session ID is not used
as the MAF session identifier.

When `conversation_id` is used, the host reads and writes the same snapshot
under that ID. When `previous_response_id` is used, the host reads that response
snapshot, runs the loaded MAF session, and writes the updated snapshot under the
current response's `response_id`. Multiple responses can therefore branch from
one prior response without overwriting its snapshot.
When `conversation_id` is used, the host reads the latest snapshot under that
ID, then writes the updated state under both the current `response_id` and the
conversation ID. This preserves every turn while keeping conversation
continuation pointed at the latest state. When `previous_response_id` is used,
the host reads that response snapshot, runs the loaded MAF session, and writes
the updated snapshot under the current response's `response_id`. Responses can
therefore branch from any prior turn without overwriting its snapshot,
including turns originally created through a conversation.

Foundry does not infer the hosted `agent_session_id` from
`previous_response_id`. Callers using response chains must also reuse the
Expand All @@ -46,15 +51,18 @@ same sandbox and `$HOME/.sessions` filesystem. Conversation objects bind to a
stable hosted session automatically.

Workflow checkpoints and function approvals preserve the existing Foundry
Hosting layout. Hosted paths insert the validated raw platform user ID:
Hosting roots. Hosted paths insert the validated raw platform user ID:

```text
/.checkpoints/<user-id>/<context-id>/
/.checkpoints/<user-id>/<response-id>/
/.checkpoints/<user-id>/<conversation-id>/
/.function_approvals/<user-id>/approval_requests.json
```

Local workflow checkpoints use `{cwd}/.checkpoints/<context-id>/`, and local
function approvals remain in memory.
The conversation directory is a latest-state alias. Each response directory
retains the final checkpoint selected for that turn, allowing a later request
to branch from it. Local workflow checkpoints use the same layout without
`<user-id>`, and local function approvals remain in memory.

Hosted requests require container protocol `2.0.0`. The v2-only request
`call_id` is checked before session, checkpoint, or approval storage is used,
Expand Down
211 changes: 133 additions & 78 deletions python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -569,7 +569,9 @@ async def _handle_inner_agent(

Foundry sessions govern hosted compute and filesystem lifetime and may
serve multiple users and Responses conversations. Conversation mode
reads and writes one MAF session snapshot under ``conversation_id``.
reads the latest MAF session snapshot under ``conversation_id`` and
writes each turn under both its immutable ``response_id`` and the
conversation ID.
Response chaining reads the snapshot under ``previous_response_id`` and
writes the updated session under the current ``response_id``, allowing
branches without changing the MAF session's own identifier. The request
Comment thread
cecheta marked this conversation as resolved.
Expand All @@ -591,6 +593,8 @@ async def _handle_inner_agent(

try:
approval_storage = self._approval_storage_for_request()
if request.previous_response_id is not None and context.conversation_id is not None:
raise RuntimeError("Previous response ID cannot be used in conjunction with conversation ID.")
read_session_id = context.conversation_id or request.previous_response_id
if self._session_store is None:
if read_session_id is not None:
Expand Down Expand Up @@ -688,7 +692,9 @@ async def _handle_inner_agent(
session.state.pop(_HOSTED_RESPONSES_HISTORY_SOURCE_ID, None)
if session is not None and self._session_store is not None:
try:
await self._session_store.set(context.conversation_id or context.response_id, session)
await self._session_store.set(context.response_id, session)
if context.conversation_id is not None:
await self._session_store.set(context.conversation_id, session)
except Exception as save_error:
if request_interrupted:
logger.error(
Expand Down Expand Up @@ -808,100 +814,149 @@ async def _handle_inner_workflow(
if latest_checkpoint is not None:
latest_checkpoint_id = latest_checkpoint.checkpoint_id

# Storage that will receive checkpoints written during this turn.
# When the caller chains with previous_response_id, the next turn
# will reference the current response_id as its previous_response_id,
# so new checkpoints must land under the current response_id (or the
# conversation_id when set). When conversation_id is set, this
# matches restore_storage; when only previous_response_id was
# supplied, restore_storage points at the *prior* response's
# directory and write_storage points at the *current* response's.
write_context_id = context.conversation_id or context.response_id
# Each turn writes to response-addressed checkpoint storage.
# Conversation continuation is updated from its latest checkpoint
# after the run.
write_context_id = context.response_id

@moonbox3 Evan Mattson (moonbox3) Aug 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a conversation-backed workflow is cancelled or the client disconnects after the workflow has written a checkpoint? write_storage now points only at response_id, but _finalize_workflow_checkpoints is called only on the normal completion paths. Cancellation (CancelledError) and the failure handler therefore leave conversation_id at the previous turn. The next conversation request can restore stale state and repeat already-completed external side effects, so could we promote the latest response checkpoint during interruption or failure cleanup as well?

@cecheta Chinedum Echeta (cecheta) Aug 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The saving is in a finally block now, similar to when running a normal agent.

write_storage = _checkpoint_storage_for_context(
self._checkpoint_storage_path,
write_context_id,
user_id=user_id,
)

# Multi-turn pattern: when we have a prior checkpoint, restore it
# first (drive the workflow back to idle with prior state intact),
# then make a separate call that delivers the new user input. This
# depends on Workflow.run preserving shared state across calls. The
# restore-only call may yield events from any pending in-flight
# work in the checkpoint; we consume those internally here so they
# don't surface to the response stream as duplicates.
#
# If the restored checkpoint had pending request_info events, the
# restore-only call replays them through
# ``WorkflowAgent._convert_workflow_event_to_agent_response_updates``
# and populates ``self._agent.pending_requests``. That is the correct
# state: those requests are genuinely outstanding, and the next
# ``run(input_messages, ...)`` call may contain ``function_call_output``
# items (carried as FunctionResult/FunctionApprovalResponse content)
# that fulfill them via :meth:`WorkflowAgent._process_pending_requests`.
if latest_checkpoint_id is not None:
if is_streaming_request:
async for _ in self._agent.run(
stream=True,
checkpoint_id=latest_checkpoint_id,
checkpoint_storage=restore_storage,
):
pass
else:
await self._agent.run(
request_failure: Exception | None = None
request_interrupted = False
try:
# Multi-turn pattern: when we have a prior checkpoint, restore it
# first (drive the workflow back to idle with prior state intact),
# then make a separate call that delivers the new user input. This
# depends on Workflow.run preserving shared state across calls. The
# restore-only call may yield events from any pending in-flight
# work in the checkpoint; we consume those internally here so they
# don't surface to the response stream as duplicates.
#
# If the restored checkpoint had pending request_info events, the
# restore-only call replays them through
# ``WorkflowAgent._convert_workflow_event_to_agent_response_updates``
# and populates ``self._agent.pending_requests``. That is the correct
# state: those requests are genuinely outstanding, and the next
# ``run(input_messages, ...)`` call may contain ``function_call_output``
# items (carried as FunctionResult/FunctionApprovalResponse content)
# that fulfill them via :meth:`WorkflowAgent._process_pending_requests`.
if latest_checkpoint_id is not None:
if is_streaming_request:
async for _ in self._agent.run(
stream=True,
checkpoint_id=latest_checkpoint_id,
checkpoint_storage=restore_storage,
):
pass
else:
await self._agent.run(
stream=False,
checkpoint_id=latest_checkpoint_id,
checkpoint_storage=restore_storage,
)

if not is_streaming_request:
# Run the agent in non-streaming mode with the new user input.
response = await self._agent.run(
input_messages,
stream=False,
checkpoint_id=latest_checkpoint_id,
checkpoint_storage=restore_storage,
checkpoint_storage=write_storage,
)

if not is_streaming_request:
# Run the agent in non-streaming mode with the new user input.
response = await self._agent.run(
input_messages,
stream=False,
checkpoint_storage=write_storage,
)

async for item in _to_outputs_for_messages(
response_event_stream,
response.messages,
approval_storage=approval_storage,
):
yield item

await self._delete_not_latest_checkpoints(write_storage, self._agent.workflow.name)
yield response_event_stream.emit_completed()
return
async for item in _to_outputs_for_messages(
response_event_stream,
response.messages,
approval_storage=approval_storage,
):
yield item
else:
tracker = _OutputItemTracker(response_event_stream)

tracker = _OutputItemTracker(response_event_stream)
# Run the workflow agent in streaming mode with the new user input.
async for update in self._agent.run(
input_messages,
stream=True,
checkpoint_storage=write_storage,
):
for content in update.contents:
for event in tracker.handle(content):
yield event
if tracker.needs_async:
async for item in _to_outputs(
response_event_stream, content, approval_storage=approval_storage
):
yield item
tracker.needs_async = False

# Run the workflow agent in streaming mode with the new user input.
async for update in self._agent.run(
input_messages,
stream=True,
checkpoint_storage=write_storage,
):
for content in update.contents:
for event in tracker.handle(content):
# Close any remaining active builder
for event in tracker.close():
yield event
if tracker.needs_async:
async for item in _to_outputs(
response_event_stream, content, approval_storage=approval_storage
):
yield item
tracker.needs_async = False

# Close any remaining active builder
for event in tracker.close():
yield event

await self._delete_not_latest_checkpoints(write_storage, self._agent.workflow.name)
except asyncio.CancelledError:
request_interrupted = True
raise
except GeneratorExit:
request_interrupted = True
raise
except Exception as ex:
request_failure = ex
raise
finally:
try:
await self._finalize_workflow_checkpoints(
write_storage,
workflow_name=self._agent.workflow.name,
conversation_id=context.conversation_id,
user_id=user_id,
)
except Exception as save_error:
if request_interrupted:
logger.error(
"Failed to finalize workflow checkpoints while unwinding an interrupted request",
exc_info=(type(save_error), save_error, save_error.__traceback__),
)
elif request_failure is not None:
logger.error(
"Failed to finalize workflow checkpoints after a workflow failure",
exc_info=(type(save_error), save_error, save_error.__traceback__),
)
else:
raise
yield response_event_stream.emit_completed()
except Exception as ex:
logger.exception("Failed to produce response for workflow agent")
for event in self._emit_failure(response_event_stream, tracker, ex):
yield event

async def _finalize_workflow_checkpoints(
self,
response_storage: FileCheckpointStorage,
*,
workflow_name: str,
conversation_id: str | None,
user_id: str | None,
) -> None:
"""Keep one response checkpoint and update the conversation's latest-state alias."""
await self._delete_not_latest_checkpoints(response_storage, workflow_name)
if conversation_id is None:
return

latest_checkpoint = await response_storage.get_latest(workflow_name=workflow_name)
if latest_checkpoint is None:
return
if self._checkpoint_storage_path is None:
raise RuntimeError("Checkpoint storage path is not configured for workflow agent.")

conversation_storage = _checkpoint_storage_for_context(
self._checkpoint_storage_path,
conversation_id,
user_id=user_id,
)
await conversation_storage.save(latest_checkpoint)
await self._delete_not_latest_checkpoints(conversation_storage, workflow_name)

@staticmethod
async def _delete_not_latest_checkpoints(checkpoint_storage: FileCheckpointStorage, workflow_name: str) -> None:
"""Delete all checkpoints except the latest one.
Expand Down
6 changes: 3 additions & 3 deletions python/packages/foundry_hosting/agent_framework_foundry_hosting/_session_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ class FoundrySessionStore(FileSessionStore):
A Foundry hosted session controls platform compute and filesystem lifetime
and may host multiple users and Responses conversations. A MAF
:class:`AgentSession` contains framework context state. Snapshots are keyed
by ``conversation_id`` for stored conversations or by Responses
``response_id`` for response chains; these storage keys are independent of
the MAF session's own identifier.
by every Responses ``response_id``. Stored conversations also update a
snapshot keyed by ``conversation_id`` as an alias for the latest turn.
These storage keys are independent of the MAF session's own identifier.

This implementation currently persists through :class:`FileSessionStore`,
with each validated platform user ID as a child directory. The
Expand Down
Loading
Loading