Skip to content

Send the error text (not null) in the Anthropic tool_result when FunctionResultContent has an Error and nil Result - #756

Open
PratikDhanave wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:anthropic-tool-result-error-text
Open

Send the error text (not null) in the Anthropic tool_result when FunctionResultContent has an Error and nil Result#756
PratikDhanave wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:anthropic-tool-result-error-text

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

What

In provider/anthropicprovider/agent.go, the *message.FunctionResultContent branch of buildMessageParam derived the tool_result content solely from c.Result. When a tool result carried an Error but a nil Result, the default arm's json.Marshal(c.Result) produced the literal null, and c.Error was only used as the is_error flag:

content = append(content, anthropic.NewToolResultBlock(c.CallID, resStr, c.Error != nil))

So Anthropic received tool_result content null with is_error: true, and the model never saw why the tool failed.

This change short-circuits on the error: when c.Error != nil, the content is set to "Error: " + c.Error.Error() before building the block. The normal (success) path is unchanged.

Why

Every other Go provider surfaces the error text for the same content:

  • openaiprovider/chat.go sets the content to funcResult.Error
  • openaiprovider/responses.go sends "Error: %v"
  • geminiprovider sends {"error": c.Error.Error()}

Anthropic was the only provider dropping the diagnostic. This also matches the .NET/Python SDK semantics, which put the failure text into the tool result rather than sending null, keeping cross-SDK behavior aligned. This is reachable from in-process failed tool results (e.g. the tool-autocall loop) that set Error with an empty Result.

Testing

Added two black-box tests in the canonical agent_test.go, driven through the existing fake-transport harness (a.Run(...).Collect()):

  • TestToolResultErrorSendsErrorText: a RoleTool message whose FunctionResultContent has Error set and a nil Result. Asserts the outgoing tool_result block has is_error: true and content containing the error text, and not the literal null. This test fails before the fix (content is null) and passes after.
  • TestToolResultSuccessSendsResult: a companion case with Error nil and Result set, confirming the success path is unchanged (is_error false, content carries the result).

Copilot AI review requested due to automatic review settings July 24, 2026 04:23
@PratikDhanave
PratikDhanave requested a review from a team as a code owner July 24, 2026 04:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Anthropic provider message construction so tool failures surface diagnostic text in tool_result content (instead of serializing a nil result to null), aligning Anthropic behavior with other providers and improving model/tool debugging.

Changes:

  • In buildMessageParam, short-circuit tool result serialization to send "Error: ..." when a tool result has an error and would otherwise serialize to null.
  • Add black-box tests asserting Anthropic requests include error text for failed tool results and preserve the success path behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
provider/anthropicprovider/agent.go Adjusts Anthropic tool_result content construction to include error text instead of null on failures.
provider/anthropicprovider/agent_test.go Adds request-capture tests ensuring tool_result error/success content is serialized as intended.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread provider/anthropicprovider/agent.go
Comment thread provider/anthropicprovider/agent_test.go Outdated
@github-actions

This comment has been minimized.

@github-actions github-actions Bot added the parity-approved Go API consistency review found no parity issues label Jul 24, 2026
The FunctionResultContent branch derived the tool_result content solely
from c.Result, so a failed tool result carrying an Error but a nil Result
serialized to the literal "null" while is_error was set to true. The model
never saw why the tool failed.

Short-circuit on c.Error and send "Error: <text>" instead, matching the
other providers: OpenAI Responses sends "Error: %v", OpenAI Chat sets the
content to the error, and Gemini sends {"error": ...}. This also aligns
with the .NET/Python SDKs, which surface the failure diagnostic in the
tool result rather than dropping it.
@PratikDhanave
PratikDhanave force-pushed the anthropic-tool-result-error-text branch from 19f9db3 to 572318b Compare July 24, 2026 09:29
@github-actions

Copy link
Copy Markdown
Contributor

Parity Review — ✅ Approved

This PR fixes a behavior bug in the Anthropic provider (provider/anthropicprovider/agent.go) where FunctionResultContent with a non-nil Error and nil Result produced null in the outgoing tool_result block instead of the error text.

Scope Assessment

No exported API surface changed. The fix is confined to the unexported buildMessageParam function. No new types, functions, interfaces, or options are introduced. The public-api-change label is not warranted.

Cross-SDK Parity

This change closes a pre-existing divergence rather than introducing one:

  • Go (OpenAI provider): openaiprovider/chat.go already surfaces funcResult.Error in the result content.
  • Go (OpenAI Responses provider): openaiprovider/responses.go sends "Error: %v".
  • Go (Gemini provider): geminiprovider sends {"error": c.Error.Error()}.
  • Anthropic provider (before this fix): sent null with is_error: true — the only Go provider that dropped the diagnostic.

The PR description notes this aligns with .NET and Python SDK semantics, which surface the failure text in the tool result rather than sending null. That claim is consistent with the cross-SDK pattern visible across all Go providers.

Verdict

The fix brings the Anthropic provider to parity with all other Go providers and with the upstream .NET/Python behavioral expectation. No cross-repo consistency issues found.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent · 35.5 AIC · ⌖ 5.61 AIC · ⊞ 5.9K ·

@gdams

gdams commented Jul 28, 2026

Copy link
Copy Markdown
Member

@PratikDhanave conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parity-approved Go API consistency review found no parity issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants