Skip to content

fix(acp): deliver plain-text replies when the model skips the send tool - #2681

Closed
micspiral wants to merge 2 commits into
mainfrom
micn/acp-plaintext-reply-fallback
Closed

fix(acp): deliver plain-text replies when the model skips the send tool#2681
micspiral wants to merge 2 commits into
mainfrom
micn/acp-plaintext-reply-fallback

Conversation

@micspiral

Copy link
Copy Markdown
Collaborator

What

Adds a content-delivery fallback to buzz-acp: when a managed agent answers a
conversational prompt in plain assistant text and never calls the publish tool
(buzz messages send), the harness posts that streamed content as a threaded
kind-9 reply — instead of silently dropping it.

Capable hosted models reliably publish via the send tool, but weaker local /
shared-compute models (e.g. a small Gemma/Qwen) often just answer in prose and
never call the tool, so the reply is lost. This closes that gap.

How

Delivery is confirmed from the send tool call's terminal outcome, not its
input text — intent is not delivery:

  • A tool_call whose input matches the publish signature (normalized so both
    shell strings and argv forms like subprocess.run(['buzz','messages','send',…])
    match) only registers a candidate toolCallId.
  • The candidate is resolved on its terminal tool_call_update:
    • failed / cancellednot delivered (fallback stays armed).
    • completed → inspect rawOutput.isError, the CLI's {"accepted":…}
      response envelope, and any reported exit_code. A bare completed with no
      inspectable output keeps the status-quo direction (suppress).
  • On a normal EndTurn with buffered content but no confirmed publish, post the
    content as a threaded reply (buzz_sdk::build_message, signed with the agent
    keys, best-effort — mirrors post_failure_notice). Bare acknowledgements the
    base prompt forbids publishing are skipped.

Fires only on EndTurn (not MaxTokens/MaxTurnRequests, which are
truncated/aborted turns) and only when a triggering channel event was present
(not heartbeats).

Testing

  • Unit: send-tool detection incl. read-vs-send discrimination and argv-style
    publishes; the outcome classifier matrix
    (pending/failed/cancelled/completed × envelope/isError/exit_code); bare-ack
    filtering.
  • Handler-level lifecycle tests through handle_session_update +
    take_undelivered_turn_message: success, failure, cancel, failed-then-retry,
    and argv-style publish.
  • Original fallback was live-validated against a small Gemma on a real
    community: fires on prose answers, stays dormant when the model calls the
    send tool.

Notes

Split out of #2448 so this reply-delivery fix stands on its own, independent of
that PR's shared-compute usability changes. +710 / crates/buzz-acp only — no
desktop / mesh / toggle changes.

michaelneale and others added 2 commits July 24, 2026 15:18
buzz-agent's output is its tool calls; streamed assistant content is
observability-only and normally never posted. Capable models reliably call
`buzz messages send`, but weaker local models (e.g. via Buzz shared compute)
often answer a conversational prompt in plain content and never call the send
tool, silently dropping the reply.

Add a content-delivery fallback in buzz-acp: track per-turn whether the model
published its own message and buffer streamed content; on a normal turn end
with content but no publish tool call, post that content as a threaded kind-9
reply (reusing buzz_sdk::build_message, signed with the agent keys, best-effort,
mirroring post_failure_notice). Skips bare acknowledgements the base prompt
forbids publishing.

Live-validated against a small Gemma on a real community: the fallback fires
when the model answers in prose and stays dormant when it calls the send tool.

Unit tests cover send-tool detection (incl. read-vs-send discrimination) and
bare-ack filtering.

(cherry picked from commit 1fe074b)
Review findings from #2448 (Wren + Eva): the content-delivery fallback
inferred delivery from the send tool call's INPUT, which was wrong in
both directions:

- False negative (silent drop): turn_sent_message was set the moment a
  tool_call's rawInput matched 'messages send' — intent, not delivery.
  A send that subsequently failed (relay/network/auth/CLI error) still
  suppressed the fallback, dropping the exact reply the feature exists
  to save.
- False positive (double post): the substring match missed argv-style
  publishes (subprocess.run(['buzz','messages','send',...])), so a
  capable model that narrates between tool calls and publishes via
  argv would get its whole turn narration re-posted by the fallback.

Fix: registering a publish is now two-phase.
- tool_call input matching (normalized so shell strings AND argv forms
  match) only registers a CANDIDATE toolCallId.
- Delivery is confirmed solely from the candidate's terminal
  tool_call_update outcome: failed/cancelled → not delivered (fallback
  stays armed); completed → inspect rawOutput.isError, the CLI's
  {"accepted":...} response envelope, and any reported exit_code;
  a bare completed with no inspectable output keeps the status-quo
  direction (suppress, i.e. today's behavior).

Tests: outcome classifier matrix (pending/failed/cancelled/completed x
envelope/isError/exit_code) plus handler-level lifecycle tests through
handle_session_update + take_undelivered_turn_message covering success,
failure, cancel, failed-then-retry, and argv-style publish.

Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
(cherry picked from commit e992ae7)

Copy link
Copy Markdown

Additional live validation: I tested this PR unchanged with OpenClaw 2026.6.1 acting as a plain-text ACP server behind buzz-acp.

Path: Buzz mention → buzz-acpkubectl exec -iopenclaw acp → existing OpenClaw Gateway session.

OpenClaw emitted standard agent_message_chunk text and had no Buzz CLI/MCP tools. The harness logged content-delivery fallback: posted undelivered agent content as channel reply, and the relay contained exactly one correctly threaded kind:9 reply with the requested text. The full buzz-acp test suite also passed locally (600 unit tests + 9 lifecycle tests).

This confirms the fallback covers an ACP bridge where giving the downstream agent the Buzz signing key or a general-purpose shell/MCP solely for reply delivery would be undesirable.

@micspiral

Copy link
Copy Markdown
Collaborator Author

I am going to close this as we have had a few runs at other ways of this - but if this comes back, we will bring it back

@micspiral micspiral closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants