Fix _flushQueue stuck on callback error

[ask-bonk]

Here's a summary of the changes made to fix issue #883:

Changes

Root cause

_flushQueue() in packages/agents/src/index.ts:1555 had two bugs:

1. No try-catch around callback invocation -- if callback.bind(this)(...) threw, this.dequeue(row.id) was skipped, the error broke out of the while(true) loop, and the row permanently stayed in cf_agents_queues, creating an infinite retry loop that blocked all subsequent items.
2. _flushingQueue never reset on error -- the error propagating out of the while loop meant this._flushingQueue = false on the last line never executed, permanently locking the queue for the DO instance's lifetime.

Fix (packages/agents/src/index.ts)

• Wrapped the entire while(true) loop in a try/finally to guarantee _flushingQueue is always reset to false
• Wrapped each individual callback invocation in try/catch/finally so that:
  • Errors are logged with context (callback name and row ID)
  • this.dequeue(row.id) always runs in the finally block, removing the item regardless of success or failure
  • Subsequent items in the queue continue processing
• Missing callbacks now also trigger dequeue() instead of just continue, preventing orphaned rows

Tests

• Added packages/agents/src/tests/agents/queue.ts -- test agent with success and throwing callbacks
• Added packages/agents/src/tests/queue.test.ts -- 3 tests covering:
  • Successful queue item processing
  • Failing item gets dequeued and doesn't block subsequent items
  • Mixed batch: failing + succeeding items both get processed/dequeued

Verification

• Build passes
• All 461 tests pass (29 test files), including 3 new queue tests
• TypeScript type check passes across all 35 projects
• Formatting passes
• Changeset added for the patch release

Closes #883

github run

[changeset-bot]

🦋 Changeset detected

Latest commit: b229430

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
agents	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[threepointone]

/bonk would deploying this fix existing stuck DOs/queues?

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/cloudflare/agents@890

commit: b229430

[ask-bonk]

Here's the critical finding. Let me answer the question:

---

Short answer: Partially. It depends on which bug bit you.

This fix addresses two bugs, and they have different recovery stories:

Bug 1: _flushingQueue permanently stuck to true (in-memory flag)

If a callback threw and _flushingQueue never got reset, the queue was locked for the lifetime of that DO instance. But _flushingQueue is an in-memory boolean — it's not persisted. When the DO gets evicted (due to inactivity, redeployment, etc.) and later re-instantiated, _flushingQueue resets to false. However, _flushQueue() is only called from queue() (line 1546) — there's no call to _flushQueue() in onStart(). So a stuck DO that gets evicted and re-created won't automatically drain its orphaned rows unless something calls queue() again to enqueue a new item.

Deploying this fix alone does NOT unstick these DOs unless they receive a new queue() call after redeployment.

Bug 2: Poison rows stuck in cf_agents_queues (persisted in SQLite)

The rows that caused the error are still in the cf_agents_queues table. Even after deploying this fix, those rows will be retried on the next _flushQueue() call. With the fix, they'll now:

1. Fail again (if the callback still throws for that payload)
2. Get properly dequeued in the finally block
3. Not block subsequent items

So the fix will self-heal poison rows — but only when _flushQueue() actually runs, which requires a new queue() call.

Recommendation

If there are known stuck DOs in production, consider adding a _flushQueue() call to onStart() so that existing orphaned queue items get drained on DO initialization without needing a new enqueue:

// In the onStart wrapper (around line 978):
return _onStart(props);
// Add after:
void this._flushQueue().catch((e) => {
  console.error("Error flushing queue on start:", e);
});

This would make the fix fully self-healing — any stuck DO would drain its queue the next time it wakes up for any reason (not just a new queue() call). Without that, operators would need to either manually trigger a queue() call on affected DOs or call dequeueAll()/dequeueAllByCallback() to clear the poison rows.

github run

[threepointone]

/bonk can you add this recommendation to the changeset so people will know when they see the changelog?

[ask-bonk]

Updated the changeset to include the recovery recommendation. The added note explains:

• The fix is self-healing for poison rows once _flushQueue() runs again
• _flushQueue() is only triggered by a new queue() call, not on DO initialization
• Workarounds for existing stuck DOs: trigger a new queue() call, or use dequeueAll()/dequeueAllByCallback()
• Mentions the potential future improvement of calling _flushQueue() in onStart()

github run