Skip to content
This repository was archived by the owner on Aug 10, 2026. It is now read-only.

feat(agent): agent-to-agent callback routing (reply_to) — orchestrator substrate - #123

Merged
unforced merged 3 commits into
mainfrom
ag-agent-callbacks
Jun 20, 2026
Merged

feat(agent): agent-to-agent callback routing (reply_to) — orchestrator substrate#123
unforced merged 3 commits into
mainfrom
ag-agent-callbacks

Conversation

@unforced

Copy link
Copy Markdown
Contributor

What

Request/response between agent threads, built ON the thread-as-container (#122) + pending-inbound (#121) substrate. When an agent sends a message to ANOTHER agent and wants to know when it's done, it sets a callback address on the message. When the recipient finishes its turn, the daemon delivers a lightweight callback back to the sender's channel — a NOTIFICATION + a LINK to the result, not the full result duplicated. The sender (an orchestrator) is woken by the callback and PULLS the full result if it wants.

Summary + link, orchestrator pulls (the explicit design choice): cleaner (callback notes stay small/uniform) + a better security boundary (the orchestrator reaches the result through its own read scope; the callback carries only opaque ids).

No new transport, no new backend — additive metadata + one new daemon→registry seam.

The model

reply_to on inbound (the SEND side)

An #agent/message/inbound note MAY carry, in metadata:

  • reply_to — the sender's channel name (where to deliver the callback). Absent → no callback (ordinary turn).
  • correlation_id — (optional) opaque id the sender matches replies to requests with; echoed verbatim.
  • delegation_depth — (optional, default 0) hop count; the loop-guard counter.

These ride note.metadataingestInbound flattens into metacontextFor.emit extracts via callbackFieldsFromMeta → onto QueuedMessage → the drain. The pending buffer carries them too (a delegated request that arrives before its recipient is live still calls back on replay).

Callback on turn-completion (the core)

In the drain, after a turn completes on BOTH ok and error (the orchestrator MUST learn about failures), IF the message had reply_to, the daemon writes a NEW #agent/message/inbound note to the reply_to channel (wakes the sender through the normal vault-trigger path).

Metadata contract:

Field Value
callback "true"
status "ok" | "error"
source_channel the recipient channel/def that finished
source_thread the recipient's per-turn #agent/thread id (pull the full thread record)
source_message the recipient's OUTBOUND reply note id, when a reply landed (absent on error/empty turn)
correlation_id echoed when present
delegation_depth incoming + 1

Brief content ([callback] <ch> finished (ok) — see source_message / source_thread …), never the duplicated reply. source_message is captured by widening WriteOutbound's return to Promise<{ id?: string } | void> (back-compat — void is in the union, so every existing recorder still satisfies it).

Loop safety (3 layers, defense-in-depth)

  1. The callback note never carries reply_to (structural — no ping-pong). Enforced at the daemon wiring AND stripped defensively in VaultTransport.writeCallback.
  2. delegation_depth ceiling (MAX_DELEGATION_DEPTH = 8): a message at/past the ceiling delivers NO callback (logged loudly). Bounds any chain even if layer 1 were defeated. The turn still runs + records.
  3. Unknown / not-live reply_to channel reuses the fix(agent): pending-inbound queue + replay (#121) and thread-as-container working-ensure #122 own-it-don't-strand posture: buildWriteCallback logs + returns WITHOUT throwing — never crashes the recipient's drain.

A callback delivery failure is best-effort: logged, never thrown, never re-runs the (completed) turn.

Concurrency (verified, not re-architected)

#122 already gives it. N callbacks returning to one orchestrator channel arrive as inbound notes, queue FIFO, and drain ONE at a time via the per-channel serial #draining invariant — never two concurrent claude -p for one channel, so the orchestrator's single-threaded --resume session carries state across them, none lost. A test simulates N callbacks draining FIFO with maxConcurrent === 1.

Files

  • src/backends/registry.tsQueuedMessage gains replyTo/correlationId/delegationDepth; WriteCallback seam + CallbackMeta + MAX_DELEGATION_DEPTH; maybeDeliverCallback at all four drain terminal points; WriteOutbound returns the note id.
  • src/daemon.tscallbackFieldsFromMeta; contextFor.emit threads the fields (enqueue + pending); buildWriteCallback; buildWriteOutbound returns the note id.
  • src/transport.tsTransport.writeCallback? + CallbackMetadata.
  • src/transports/vault.tsVaultTransport.writeCallback; writeInbound optional extraMeta.
  • design/2026-06-20-agent-callbacks.md.

Tests / gate

bun run test (typecheck + bun test ./src) green — 1017 pass, 0 fail. New coverage: all four drain terminal points, the depth guard at ceiling + ceiling-1, no-reply_to → no callback (regression), unwired/throwing sink, FIFO concurrency, callbackFieldsFromMeta coercion, end-to-end contextFor.emit threading, buildWriteCallback own-it, the full vault callback-note metadata contract + reply_to-strip, and the ingestInbound flatten round-trip.

Independent reviewer pass: LGTM, no critical issues, no must-fix bugs (the two comment-clarity nits applied; the shared-type-refactor nit left as a noted follow-up since the duck-type is typecheck-guarded).

Deferred

  • A dedicated delegate MCP tool that fills in reply_to/correlation_id/delegation_depth so an orchestrator doesn't hand-stamp them (SEND-side ergonomics).
  • Single-threaded source_thread is a per-turn correlation id (the deterministic thread-note-by-stable-path linkage is the same gap that already exists for metadata.thread).

No version bump (per instructions).

🤖 Generated with Claude Code

unforced and others added 2 commits June 20, 2026 00:52
…r substrate

Request/response between agent threads. An inbound #agent/message/inbound note MAY
carry metadata.reply_to (the sender's channel), plus optional correlation_id and
delegation_depth. When the recipient's programmatic turn finishes (BOTH ok and error),
the daemon delivers a lightweight CALLBACK back to the reply_to channel — a brief
notification + LINK (source_thread / source_message) the orchestrator pulls the full
result from, NOT the duplicated reply.

Loop safety (3 layers): the callback note never carries reply_to (terminal, structural);
delegation_depth ceiling (MAX_DELEGATION_DEPTH=8) bounds runaway chains; an unknown
reply_to channel reuses the #122 own-it-don't-strand posture (log + no throw).

Concurrency: N callbacks returning to one orchestrator channel drain FIFO via the
existing per-channel serial drain (#122) — never concurrent, none lost; the orchestrator's
--resume session carries state across them.

- registry.ts: QueuedMessage gains replyTo/correlationId/delegationDepth; WriteCallback
  seam + CallbackMeta contract + MAX_DELEGATION_DEPTH; maybeDeliverCallback at all four
  drain terminal points; WriteOutbound return widened to surface the outbound note id
  for source_message (back-compat — void is in the union).
- daemon.ts: callbackFieldsFromMeta (extract + string->int coerce); contextFor.emit
  threads the fields onto the enqueue + pending paths; buildWriteCallback (resolve
  reply_to channel transport, own-it on unknown); buildWriteOutbound returns the note id.
- transport.ts: Transport.writeCallback? + CallbackMetadata.
- vault.ts: VaultTransport.writeCallback (writes a callback inbound note, strips any
  stray reply_to); writeInbound gains optional extraMeta.
- design/2026-06-20-agent-callbacks.md: the model, metadata contract, loop safety,
  concurrency story, summary+link rationale, deferred delegate MCP tool.

Gate: bun run test (typecheck + bun test ./src) green — 1017 pass, 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…wer nits)

Comment-only: callbackFieldsFromMeta notes that a literal "0" is omitted (the drain's
?? 0 fallback handles depth 0); the concurrency test comment clarifies it exercises the
drain-side FIFO property, not the real vault-IPC delivery path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>


source_thread is resolvable for multi-threaded (per-fire note leaf) but is a
per-turn correlation id for single-threaded (NOT the deterministic note leaf), so
source_message is the reliable pull-link for single-threaded recipients. Fixed the
CallbackMeta doc, the metadata-contract table, and the deferred-notes; the proper
fix (widen the writeThread seam so source_thread is the written note id for both
modes) is tracked as #124. No logic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@unforced
unforced merged commit a3dc60b into main Jun 20, 2026
@unforced
unforced deleted the ag-agent-callbacks branch June 20, 2026 07:06
unforced added a commit that referenced this pull request Jun 29, 2026
…both modes (#124) (#161)

* fix(agent): make callback source_thread a resolvable thread-note id for both modes (#124)

The agent-to-agent callback (PR #123) puts `source_thread` on the callback
metadata so an orchestrator can pull the recipient's full thread record. It
worked for multi-threaded recipients (`source_thread` = the per-fire note leaf,
resolvable at `Threads/<channel>/<source_thread>`) but NOT for single-threaded
recipients (the DEFAULT): `source_thread` was a per-turn CORRELATION id, not the
note leaf — the single-threaded note lives at the deterministic
`Threads/<safeChannel>/<safeName>`, so the orchestrator couldn't resolve the
thread from `source_thread`. The reliable single-threaded pull-link was
`source_message` (the outbound reply note id) — but that's ABSENT on an
error/empty/tool-only turn, leaving an unresolvable `source_thread` and no link.

Fix: make `source_thread` the ACTUAL written thread-note id for BOTH modes, so
it's always resolvable via `query-notes { id: source_thread }`:
- widen the `WriteThread` seam to return the written note id (`{ id } | void`);
  the daemon's `buildWriteThread` surfaces `writeThread`'s `{ sent: [id] }`.
- `recordThread` returns the written id; the drain captures it at every terminal
  point and passes it to `maybeDeliverCallback` as `source_thread` instead of
  the per-turn correlation UUID. Falls back to the per-turn id when no durable
  store / a write failure means the seam surfaced no id (never undefined).
- the thread note is written BEFORE the outbound reply, so its id is available
  even on an error/empty/tool-only turn (no `source_message`) — the narrow edge.

`source_message` is preserved as-is (still the per-reply link when present).

Tests: pin `source_thread` = the written thread-note id for single- AND
multi-threaded recipients (a faithful `WriteThread` recorder mirroring the
VaultTransport's path-as-id logic), incl. a single-threaded error turn and an
empty/tool-only turn (no `source_message`), plus the no-durable-store fallback.

Also: add `.claude/` to .gitignore (recurring reviewer nit — stray
`.claude/worktrees/` local artifact).

Bump 0.2.3-rc.9 → 0.2.3-rc.10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(agent): fold reviewer nits — pin outbound-failure re-record source_thread + de-mask multi-threaded id

Reviewer-deferred nits on #161 (inline, per PR-flow discipline):
- add a test pinning source_thread = the sameTurn re-recorded thread-note id on
  the OUTBOUND-DELIVERY-FAILURE terminal path (the `?? threadNoteId` precedence
  leg the author called out but hadn't covered).
- de-mask the multi-threaded id assertion: concrete cross-check that the per-fire
  leaf equals the recorded threadId, not just a regex shape.
- clarify the void `threadRecorder` deliberately exercises the per-turn-id
  fallback (the id-pinning tests use threadRecorderWithIds).

Gates: typecheck clean; bun test ./src 1208 pass / 0 fail; biome exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant