Skip to content

docs(target-arch): revise coordination model to pull/work-stealing (#1081)#1086

Merged
vybe merged 1 commit into
devfrom
feature/1081-target-arch-pull-coordination
Jun 5, 2026
Merged

docs(target-arch): revise coordination model to pull/work-stealing (#1081)#1086
vybe merged 1 commit into
devfrom
feature/1081-target-arch-pull-coordination

Conversation

@vybe

@vybe vybe commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Revises docs/planning/TARGET_ARCHITECTURE.md §Coordination Model from a push actor model (backend dispatches into a per-agent Redis mailbox) to pull / work-stealing, per the 2026-06-05 design review. The backend owns one durable per-agent queue (schedule_executions); each agent pulls the next task only when it has a free worker; capacity is physical (worker count); the sole recovery primitive is lease-expiry re-delivery of the same execution_id.

Chosen over the push model for simplicity, single-source-of-truth, and ~80% reuse of existing primitives (claim_next_queued, the RELIABILITY-005 CAS, idempotency #525, the agent auto_sync.py loop).

What changed in the doc

  • Governing principle Setup improvements #5 → pull-based work-stealing.
  • Data layer — queue + execution state move to Postgres (one owner of "queued" + "running"); Redis loses the mailbox STREAM and slot ZSET, keeps only a wake-up hint.
  • Coordination Model rewritten: backend queue, pull endpoints, physical worker pool, the two operator levers, lease-expiry re-delivery, fan-out join, failure isolation without a dispatch breaker, replica groups via row-claim.
  • New side-effect idempotency contract — the hardest open problem (feat: effect-scoped idempotency keys for outbound side effects #1084): re-delivery is safe for bookkeeping but re-emits external effects; effect-scoped keys gate pull-on for side-effect agents.
  • Agent runtime / observability — pull worker pool; mailbox_depth dropped from /health; journal demoted from source-of-truth; queue-depth metrics.
  • Open questions + tracking table reconciled with the new issues.

Tracking (Epic #1045)

Issue Role
#1081 umbrella — 6-phase pull migration
#1082 / #1083 bankable wins (status-as-projection / fire-and-forget) — ship independently
#1084 effect-scoped idempotency — the gate
#1085 correlated-failure / herd controls
#946 / #429 / #300 pilot / cleanup payoff / Postgres prerequisite
#307 / #526 repurposed gate→alert under pull

Also includes the team announcement record under docs/user-docs/dev-announcements/.

Docs-only; no code changes.

🤖 Generated with Claude Code

…1081)

Replace the push actor-model with pull / work-stealing in
TARGET_ARCHITECTURE.md, per the 2026-06-05 design review. The backend
owns one durable per-agent queue (schedule_executions); agents pull when
they have free capacity; capacity is physical (worker count); recovery is
lease-expiry re-delivery of the same execution_id.

- Governing principle #5, data layer (queue→Postgres, Redis loses the
  mailbox/slot-ZSET), agent runtime, and observability updated to match.
- Adds the side-effect idempotency contract (the hardest open problem,
  #1084) and the Postgres-before-the-queue sequencing constraint (#300).
- Reconciles the tracking table + open questions with the new issues:
  #1081 umbrella, #1082 status-as-projection, #1083 fire-and-forget,
  #1084 effect-idempotency, #1085 herd controls (all under Epic #1045);
  #946 reframed as the pilot, #307/#526 as gate→alert.
- Includes the team announcement record.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vybe
vybe merged commit a1685ea into dev Jun 5, 2026
11 checks passed
@vybe vybe added type-docs Documentation theme-reliability Theme: Reliability labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

theme-reliability Theme: Reliability type-docs Documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant