added resumable streaming with minimal setup by whoiskatrin · Pull Request #673 · cloudflare/agents

whoiskatrin · 2025-11-23T20:50:07Z

something to play with https://resumable-stream-chat.katjareznikova.workers.dev

changeset-bot · 2025-11-23T20:50:11Z

🦋 Changeset detected

Latest commit: 7859d25

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
agents	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-11-23T20:51:46Z

Open in StackBlitz

npm i https://pkg.pr.new/cloudflare/agents@673

commit: 7859d25

claude · 2025-11-23T20:51:47Z

Claude Code Review

Overview: Adds resumable streaming capability to AIChatAgent with automatic stream restoration on reconnect.

Issues Found

1. Critical: Hook wrapping breaks subclass overrides (ai-chat-agent.ts:153-174)
The constructor wraps onConnect and onMessage methods, which will override any subclass implementations:

const _onConnect = this.onConnect.bind(this);
this.onConnect = async (connection: Connection, ctx: ConnectionContext) => {
  if (this._activeStreamId) {
    this._notifyStreamResuming(connection);
  }
  return _onConnect(connection, ctx);
};

Subclasses that override these methods will have their implementations replaced. Consider using a hook/event system or template method pattern instead.

2. Unbounded chunk buffer memory leak (ai-chat-agent.ts:360-377)
CHUNK_BUFFER_MAX_SIZE is checked but never enforced - chunks can accumulate indefinitely:

if (this._chunkBuffer.length >= CHUNK_BUFFER_SIZE) {
  await this._flushChunkBuffer();
} else if (this._chunkBuffer.length >= CHUNK_BUFFER_MAX_SIZE) {
  console.warn("Chunk buffer exceeded max size");
}

After warning, chunks keep accumulating. Should force flush or drop chunks.

3. Race condition in flush lock (ai-chat-agent.ts:390-399)
Non-atomic check-and-set of _isFlushingChunks:

if (this._isFlushingChunks) return;
this._isFlushingChunks = true;
// ... async operations

Multiple concurrent calls can pass the check before the flag is set, causing duplicate flushes and potential data corruption.

4. SQL injection via template literals (ai-chat-agent.ts:403)
Using template string instead of parameterized query:

this.sql`insert into cf_ai_chat_stream_chunks (id, stream_id, body, chunk_index, created_at)
  values ${values}`;

Should use proper parameterized queries with ? placeholders.

5. Stale stream threshold too aggressive (ai-chat-agent.ts:31)
5 minutes may be too short for legitimate long-running streams, especially if the LLM response is slow. Consider 15-30 minutes or make it configurable.

6. No error handling in resume flow (ai-react.tsx:236-270)
Resume logic has no try-catch around chunk parsing/processing. Corrupted chunks will crash the client with no recovery mechanism.

7. Test worker implementation is incomplete (tests/worker.ts)
The test worker stub doesn't implement the full resumable streaming flow end-to-end. Tests only verify individual methods, not the actual WebSocket message protocol.

Minor Issues

Type safety: StreamChunk and StreamMetadata types should be exported for testing/extensions
Docs: Missing error scenarios in resumable-streaming.md (what happens on error streams?)
Performance: Index on request_id would help _restoreActiveStream query (ai-chat-agent.ts:440)
Logging: No structured logging for debugging resume failures in production

Testing Coverage

✅ Good test coverage for basic flow
✅ Tests verify stale stream cleanup
❌ Missing integration tests for actual reconnection scenario
❌ No tests for error handling/recovery paths
❌ No tests for concurrent flush scenarios

Verdict

NEEDS WORK - The hook wrapping pattern (#1) and race condition (#3) need fixes before merge. The other issues should be addressed but are less critical.

threepointone

changeset

deathbyknowledge

Looks good, only had a few thoughts. This needs a test though.

deathbyknowledge · 2025-11-24T14:01:44Z

+  };
+
+  async onStart() {
+    await super.onStart();


What is this for?

not needed anymore with Sunil's suggestion

deathbyknowledge · 2025-11-25T13:44:20Z

@@ -0,0 +1 @@
+/// <reference types="vite/client" />


tbh, we don't really need it, removed

deathbyknowledge · 2025-11-25T13:49:48Z

+      // Check if stream is stale (started too long ago)
+      if (streamAge > STREAM_STALE_THRESHOLD_MS) {
+        // Mark stale stream as error and don't restore
+        this.sql`


Just a thought, do we rather delete the entry here instead? As a user I'm not sure I want to keep taking storage with older stream metadata

deathbyknowledge · 2025-11-25T13:50:53Z

+   * Stream chunks are sent only after the client acknowledges with CF_AGENT_STREAM_RESUME_ACK.
+   */
+  override async onConnect(connection: Connection, ctx: ConnectionContext) {
+    await super.onConnect(connection, ctx);


Is the super.onConnect needed?

threepointone · 2025-11-25T14:09:20Z

+   * Override onConnect to notify clients about active streams that can be resumed.
+   * Stream chunks are sent only after the client acknowledges with CF_AGENT_STREAM_RESUME_ACK.
+   */
+  override async onConnect(connection: Connection, ctx: ConnectionContext) {


instead of overriding directly here, can we do it like we dp in the base agent class? so that consumers won't have to call super.onConnect

good call, yes, we totally should do that

Added support for resumable streaming with minimal setup.

Documents the new resumable streaming feature added to AIChatAgent class in PR cloudflare/agents#673. This feature provides automatic stream resumption when clients disconnect and reconnect during AI model responses, which is especially useful for long-running reasoning models. Key additions: - New resumable-streaming.mdx page in api-reference section - Server implementation examples using AIChatAgent - Client implementation examples using useAgentChat hook - Technical details on how resumable streaming works internally - Configuration guidance for SQLite migrations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

This documentation covers the new automatic resumable streaming feature in AIChatAgent and useAgentChat, which allows AI chat responses to automatically resume when clients reconnect after disconnection. Key features documented: - Automatic stream persistence to SQLite - Client reconnection handling - Server-side and client-side implementation examples - Configuration options for disabling resume Based on PR cloudflare/agents#673 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Synced from cloudflare/agents PR #673: added resumable streaming with minimal setup This documentation covers the automatic resumable streaming feature in AIChatAgent that allows chat streams to seamlessly resume after disconnections. Source PR: cloudflare/agents#673 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>

* Add resumable streaming documentation Synced from cloudflare/agents PR #673: added resumable streaming with minimal setup This documentation covers the automatic resumable streaming feature in AIChatAgent that allows chat streams to seamlessly resume after disconnections. Source PR: cloudflare/agents#673 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * adding resumable streaming to the docs --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: katereznykova <kreznykova@cloudflare.com>

added resumable streaming with minimal setup

9094a58

whoiskatrin requested review from deathbyknowledge and threepointone November 23, 2025 20:52

agents-git-bot Bot mentioned this pull request Nov 23, 2025

added resumable streaming with minimal setup cloudflare/cloudflare-docs#26705

Merged

4 tasks

fix naming and scripts

4f4f869

whoiskatrin marked this pull request as ready for review November 24, 2025 11:51

whoiskatrin added 3 commits November 24, 2025 23:21

refactor resumable streaming implementation into AIChatAgent

6ec8e7a

made claude review happy

7acea9c

another attempt to make claude review like me

e960ab8

whoiskatrin added 2 commits November 25, 2025 09:41

minor nits

b5dcf8f

make claude review happy v2

e373e58

threepointone approved these changes Nov 25, 2025

View reviewed changes

Comment thread examples/resumable-stream-chat/wrangler.jsonc

Comment thread examples/resumable-stream-chat/wrangler.jsonc

deathbyknowledge reviewed Nov 25, 2025

View reviewed changes

threepointone reviewed Nov 25, 2025

View reviewed changes

fix some review comments

e4b095f

whoiskatrin force-pushed the resumable-stream-minimal branch from 1c2884a to e4b095f Compare November 25, 2025 14:31

Add resumable streaming feature

b6a12f9

Added support for resumable streaming with minimal setup.

whoiskatrin added 2 commits November 25, 2025 14:41

add tests

d24a8fa

fix type check

828867a

Merge main into resumable-stream-minimal

7859d25

deathbyknowledge approved these changes Nov 25, 2025

View reviewed changes

whoiskatrin merged commit 603b825 into main Nov 25, 2025
6 checks passed

whoiskatrin deleted the resumable-stream-minimal branch November 25, 2025 17:02

github-actions Bot mentioned this pull request Nov 25, 2025

Version Packages #657

Merged

jldec mentioned this pull request Dec 6, 2025

Agent SDK page: not all connected clients receive realtime streaming updates. jldec/agents-chat#23

Closed

Uh oh!

Conversation

whoiskatrin commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new Bot commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Claude Code Review

Issues Found

Minor Issues

Testing Coverage

Verdict

Uh oh!

threepointone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

deathbyknowledge left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

whoiskatrin commented Nov 23, 2025 •

edited

Loading

changeset-bot Bot commented Nov 23, 2025 •

edited

Loading

pkg-pr-new Bot commented Nov 23, 2025 •

edited

Loading

claude Bot commented Nov 23, 2025 •

edited

Loading