feat(agentserver-responses): resilient/durable responses orchestration (1.0.0b10)

sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md

@@ -8,13 +8,83 @@
 - Added local `TypedDict` model contract generation for the Responses protocol, including generated type aliases, union aliases, and `py.typed` packaging support.
 - Added dict-native wire payload helpers and request validators for validating protocol payloads without depending on generated model internals.
 
+- `ResponseContext.conversation_chain_id` (and the resilient task id it backs) now follows the native id convention: `cchain_<partition><scope>` for conversation-scoped chains, `rchain_<partition><scope>` for steerable response-linkage chains, or the `response_id` verbatim for a non-steerable one-shot. The id embeds the chain's partition key for co-location and carries a deterministic `(agent, session)` scope; `task_id == conversation_chain_id` exactly. Replaces the previous opaque `resilient-resp-<32-hex>` form.
+
+- **Resilient background responses.** `ResponsesServerOptions(resilient_background=True)`
+  makes `store=true`, `background=true` responses survive process crashes:
+  the framework persists handler progress and re-invokes the registered
+  handler on the next process start when a prior attempt did not reach a
+  terminal event. Defaults to `False`.
+
+- **Steerable conversations.** `ResponsesServerOptions(steerable_conversations=True)`
+  lets clients post a new turn on an in-flight conversation; the running
+  handler is woken (via the cancellation signal, distinguished by
+  `context.pending_input_count > 0`), drains the queued input on a fresh
+  invocation, and the turns are linked in a stable conversation chain.
+  Defaults to `False`.
+
+- **`ResponseContext` resilience + steering surface.** Flat fields stamped on
+  each invocation: `context.is_recovery`, `context.is_steered_turn`,
+  `context.pending_input_count`, and `context.conversation_chain_id` (a stable
+  identifier shared by every turn of a conversation chain, usable as a key into
+  application-side session state).
+
+- **Developer checkpoints.** `yield stream.checkpoint()` persists the
+  current response snapshot at a developer-chosen boundary (gated to resilient
+  background responses; a no-op otherwise; backpressured and idempotent). On a
+  recovered entry, `context.persisted_response` exposes the last persisted
+  snapshot so the handler can seed its stream and resume — the basis of the
+  one-`OutputItem`-per-phase recovery pattern.
+
+- **`internal_metadata`.** A single-turn, platform-internal `MutableMapping[str, Any]`
+  on output items (`item.internal_metadata`) and on the response
+  (`stream.internal_metadata`). It is persisted with the response (so it is
+  available on recovery) and is always stripped before any client-facing
+  HTTP/SSE payload, and on ingress. Distinct from the public
+  `ResponseObject.metadata`.
+
+- **`context.conversation_chain_metadata`.** Cross-turn, named-scope,
+  explicit-`flush()` resilient metadata over a conversation chain, typed by the
+  public `ConversationChainMetadataNamespace` Protocol.
+
+- **`await context.exit_for_recovery()`.** A single uniform graceful-shutdown
+  recovery primitive that works in every handler shape (coroutine, async
+  generator, sync) — it raises `ResponseExitForRecovery` internally to leave
+  the response `in_progress` for next-lifetime recovery.
+
+- **Stream recovery.** SSE events are persisted incrementally; clients reconnect
+  with `GET /responses/{id}?stream=true&starting_after=<event_id>` and resume
+  from their last received event.
+
+- **Response acceptor hook.** Register `@app.response_acceptor` to customize the
+  response shape returned when a turn is queued behind an active steerable
+  conversation.
+
+- **Storage.** `FileResponseStore` is exported from
+  `azure.ai.agentserver.responses` and is the default local-development store
+  (under `${AGENTSERVER_STATE_ROOT:-~/.agentserver}/responses/`) when no `store=`
+  is supplied in a non-hosted environment; pass
+  `store=InMemoryResponseProvider()` to opt out. The `AGENTSERVER_STATE_ROOT`
+  environment variable sets the local state storage root. A typed
+  `ResponseAlreadyExistsError` is raised by the response-store providers on a
+  duplicate `create_response` (the idempotent-create signal on recovery).
+
+- **Handlers are `async def`.** `@app.response_handler` requires an async
+  handler with the `(request, context, cancellation_signal)` signature so it can
+  observe the `asyncio.Event` cancellation signal.
+
 ### Breaking Changes
 
 - Removed a-prefixed async convenience generator methods from the sync `ResponseEventStream` and sync builder classes. Use `azure.ai.agentserver.responses.aio.ResponseEventStream` for async streaming convenience methods.
 - Replaced generated model classes in `azure.ai.agentserver.responses.models` with dict-native `TypedDict` contracts. Model constructors such as `ItemMessage(...)` and `CreateResponse(...)` now produce plain dictionaries instead of generated model instances.
 - Removed runtime model-class behavior from response protocol models. Code should no longer rely on attribute access, `isinstance(..., ModelType)`, `.as_dict()`, or generated model base-class behavior.
 - Replaced most generated enum classes with string literal type aliases. Use string values directly for protocol fields, for example `"completed"`, `"message"`, or `"function_call_output"`.
 
+- The resilient-task input persisted for a `store=true` background response now
+  carries a single `user_id_key` (the durable per-user partition key) instead of
+  the previous `user_isolation_key` / `chat_isolation_key` pair; conversation
+  scoping continues to use `conversation_id`.
+
 ### Migration Guide
 
 Async response event stream helpers now live under the `aio` namespace and no longer use the `a` prefix.
@@ -77,8 +147,33 @@ After:
 status = "completed"
 ```
 
+### Bugs Fixed
+
+- **Steering now works on the first turn of a conversation.** In a
+  `steerable_conversations=true` deployment, the first turn (a request with no
+  `conversation_id` and no `previous_response_id`) is now hosted on the
+  multi-turn chain primitive instead of a one-shot task. Because all turns of a
+  chain share a stable `conversation_chain_id` — and therefore the same backing
+  task — a steered turn posted onto an in-flight first turn previously queued
+  onto a one-shot task that completed and auto-deleted before draining the
+  queued input, leaving the steered turn stuck `in_progress`. The first turn now
+  suspends between turns and drains queued steering inputs correctly.
+
+- **`context.conversation_chain_id` is now stable across every turn of a
+  conversation.** Previously it returned the raw `previous_response_id` (the
+  immediate predecessor), so it shifted on every turn after the second — breaking
+  its use as a stable per-conversation key (e.g. an upstream SDK session id). It
+  is now derived from the partition key embedded in the chain's response IDs
+  (which every turn shares), so all turns of a chain — and the resilient task
+  that backs them — resolve to the same identity. The value is now an opaque,
+  agent/session-scoped hash rather than a raw id. (Known limitation: a client
+  that supplies its own `response_id` with a mismatched embedded partition can
+  shift the chain identity for later turns.)
+
 ### Other Changes
 
+- Bumped the minimum `azure-ai-agentserver-core` dependency to `>=2.0.0b9`.
+- Reworked the resilient responses samples: `sample_21_resilient_langgraph` is now a real-time streaming LangGraph agent that composes LangGraph's checkpointer with the framework's response checkpoints (see the "Composing an External Durable Engine" section of the handler guide), added real crash-harness e2e coverage for samples 19–22, and removed the copilot sample.
 - Updated response hosting, persistence, streaming, validation, samples, and tests to operate on JSON-compatible wire dictionaries.
 - Updated model generation tooling to use TypeSpec Python `models-mode=typeddict` and removed generated model shim files.
 
@@ -150,7 +245,7 @@ status = "completed"
 
 ### Bugs Fixed
 
-- `DELETE /responses/{id}` no longer returns intermittent 404 when the background task's eager eviction races with the delete handler. Previously, `try_evict` could remove the record from in-memory state between the handler's `get()` and `delete()` calls, causing `delete()` to return `False` and producing a spurious 404. The handler now falls through to the durable provider when the in-memory delete fails due to a concurrent eviction.
+- `DELETE /responses/{id}` no longer returns intermittent 404 when the background task's eager eviction races with the delete handler. Previously, `try_evict` could remove the record from in-memory state between the handler's `get()` and `delete()` calls, causing `delete()` to return `False` and producing a spurious 404. The handler now falls through to the resilient provider when the in-memory delete fails due to a concurrent eviction.
 - `POST /responses` with `background=true, stream=false` now correctly returns `status: "in_progress"` instead of `"completed"`. Handlers that yield events synchronously (no `await` between yields — the normal pattern with `ResponseEventStream`) would cause the background task to run to completion before `run_background` captured the initial snapshot. A cooperative yield after `response_created_signal.set()` now ensures the POST handler resumes promptly.
 - Conversation history IDs (`previous_response_id`, `conversation_id`) are now validated eagerly before the handler is invoked. A nonexistent reference now returns a 404 error to the client immediately, instead of being silently ignored or surfacing as an opaque error deep inside the handler. The prefetched IDs are reused by `ResponseContext.get_history()`, eliminating a redundant provider call.
 

sdk/agentserver/azure-ai-agentserver-responses/README.md

@@ -24,12 +24,20 @@ This automatically installs `azure-ai-agentserver-core` as a dependency.
 
 ```python
 @app.response_handler
-def my_handler(
-    request: CreateResponse, context: ResponseContext, cancellation_signal: asyncio.Event
-):
+async def my_handler(request: CreateResponse, context: ResponseContext, cancellation_signal: asyncio.Event):
     ...
 ```
 
+Handlers MUST be `async def` and take exactly three positional parameters
+(`request`, `context`, `cancellation_signal`). Sync handlers and the 2-arg
+signature `(request, context)` are hard-rejected at decoration time.
+Cancellation is observed via the `cancellation_signal` event (set on
+client cancel, `/cancel` API, or steering pressure). Server shutdown is
+a **distinct** signal observed via `context.shutdown` — shutdown does
+NOT fire the cancellation signal; handlers that care about both must
+inspect each independently. See the handler implementation guide for
+the full surface.
+
 ### Protocol endpoints
 
 | Method | Route | Description |
@@ -90,14 +98,29 @@ The `ResponseContext` provides request-scoped state:
 | Property / Method | Description |
 |---|---|
 | `response_id` | Unique ID for this response |
+| `conversation_id` / `conversation_chain_id` | Conversation identifiers; `conversation_chain_id` is the framework-computed stable id shared by every turn in a chain |
 | `is_shutdown_requested` | Whether the server is draining |
 | `platform_context` | `PlatformContext` with `user_id_key` (from `x-agent-user-id`) and `call_id` (from `x-agent-foundry-call-id`) for multi-tenant state partitioning and per-request caller-context forwarding |
 | `client_headers` | Dictionary of `x-client-*` headers forwarded from the platform (keys normalized to lowercase) |
 | `query_parameters` | Dictionary of query string parameters |
+| `shutdown` | `asyncio.Event` set on graceful server shutdown — distinct from the per-request cancellation signal |
+| `client_cancelled` | `bool` set when the cancel cause is `/cancel` endpoint or non-bg POST disconnect |
+| `is_recovery` | `bool` set on a crash-recovered re-entry |
+| `is_steered_turn` | `bool` set on the drain re-entry that follows a steering input |
+| `pending_input_count` | `int` count of queued steering inputs |
+| `conversation_chain_metadata` | `ConversationChainMetadataNamespace` for handler-managed checkpoint state |
+| `exit_for_recovery()` | `await` to opt into the graceful-shutdown recovery path |
 | `get_input_items()` | Load resolved input items as `Item` subtypes |
 | `get_input_text()` | Extract all text content from input items as a single string |
 | `get_history()` | Load conversation history items |
 
+The per-request cancellation signal is delivered as the **3rd
+positional handler argument** (`cancellation_signal: asyncio.Event`),
+not via a `ResponseContext` attribute. It fires on client cancel
+(`/cancel` API or non-bg POST disconnect) or steering pressure; it
+does NOT fire on server shutdown — `context.shutdown` is the
+independent surface for that case.
+
 ### Streaming and background modes
 
 The SDK automatically handles all combinations of `stream` and `background` flags:
@@ -113,13 +136,15 @@ The library orchestrates the complete response lifecycle: `created` → `in_prog
 
 For detailed handler implementation guidance, see [docs/handler-implementation-guide.md](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/handler-implementation-guide.md).
 
+### Resilience
+
+Crash recovery is **opt-in** via `ResponsesServerOptions(resilient_background=True)`. When opted in, background responses with `store=True` are crash-recoverable: the handler is re-invoked on restart and the recovered context exposes `context.is_recovery == True`. Stream events are persisted incrementally so clients can reconnect and resume from where they left off. Without the opt-in (the default), a crash mid-handler marks the response `failed` instead of re-invoking the handler. For advanced scenarios (metadata checkpointing, multi-turn steering), see the [Resilient Responses Developer Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/resilient-responses-developer-guide.md).
+
 ## Examples
 
 ### Echo handler
 
 ```python
-import asyncio
-
 from azure.ai.agentserver.responses import (
     CreateResponse,
     ResponseContext,
@@ -230,7 +255,7 @@ app = ResponsesAgentServerHost(options=options)
 ### Common errors
 
 - **400 Bad Request**: The request body failed validation. Check that optional fields such as `model` (when provided) are valid and that `input` items are well-formed.
-- **404 Not Found**: The response ID does not exist or has expired past the configured TTL.
+- **404 Not Found**: The response ID does not exist. In hosted deployments persisted responses live in the Foundry hosted responses store; in local development they live under `${AGENTSERVER_STATE_ROOT:-~/.agentserver}/responses/` by default. A missing record may indicate the response was never persisted or was deleted via `DELETE /responses/{id}`.
 - **400 Bad Request** (cancel): The response was not created with `background=true`, or it has already reached a terminal state.
 
 ### Reporting issues
@@ -256,6 +281,10 @@ Visit the [Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/
 | [File Inputs](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_14_file_inputs.py) | Receive files via base64 data URL, URL, or file ID |
 | [Annotations](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_15_annotations.py) | Attach file_path, file_citation, and url_citation annotations |
 | [Structured Outputs](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_16_structured_outputs.py) | Return structured JSON as a `structured_outputs` item |
+| [Resilient Streaming](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_19_resilient_streaming.py) | Framework-checkpoint handler — one item per phase + `stream.checkpoint()`, recovery via `context.persisted_response` |
+| [Resilient Steering](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_20_resilient_steering.py) | `context.is_steered_turn` on the drain re-entry with `resilient_background=True, steerable_conversations=True`; naive re-run recovery |
+| [Resilient LangGraph](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_21_resilient_langgraph.py) | Real-time streaming LangGraph agent composing `AsyncSqliteSaver` with framework `stream.checkpoint()` / `context.persisted_response`; graph checkpoint id stored in `internal_metadata` so recovery rewinds to the persisted point (no divergence window) |
+| [Resilient Multi-turn](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_22_resilient_multiturn.py) | Multi-turn conversation with `resilient_background=True, steerable_conversations=False` |
 
 - [Handler implementation guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/handler-implementation-guide.md) — Detailed reference for building handlers