Anthropic extended thinking content not captured in instrumentation

[braintrust-bot]

Summary

Anthropic's extended thinking is a stable, prominently documented feature that adds thinking content blocks (non-streaming) and thinking_delta stream events (streaming) to responses. This repo's Anthropic instrumentation does not capture thinking content in either the wrapper or the auto-instrumentation plugin, and the vendor SDK types do not model thinking blocks or deltas at all.

What is missing

Vendor types (js/src/vendor-sdk-types/anthropic.ts)

• AnthropicOutputContentBlock (line 85–93) does not include { type: "thinking"; thinking: string }.
• AnthropicStreamEvent's content_block_delta union (lines 112–114) does not include { type: "thinking_delta"; thinking: string }.
• AnthropicUsage (lines 95–100) does not include thinking-related token fields (input_tokens for thinking cache, etc.).

Wrapper streaming (js/src/wrappers/anthropic.ts)

• streamNextProxy handles text_delta and input_json_delta in content_block_delta events (lines 343–353) but has no case for thinking_delta.
• When a thinking block streams, content_block_start saves the block, but no delta text is accumulated. At content_block_stop, text is empty so nothing is logged (line 359: if (!text) break).
• Thinking content is silently dropped from the span output.

Auto-instrumentation plugin (js/src/instrumentation/plugins/anthropic-plugin.ts)

• aggregateAnthropicStreamChunks (lines 138–198) only handles text_delta in the content_block_delta case (line 161). thinking_delta events are ignored.

Non-streaming

• parseEventFromMessage returns { role: message.role, content: message.content }, so thinking blocks would be passed through if the actual SDK returns them. However, the vendor types don't model them, so TypeScript consumers lose type safety.

Upstream reference

• Anthropic Extended Thinking docs: https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking
• @anthropic-ai/sdk streaming format includes thinking_delta events: https://github.com/anthropics/anthropic-sdk-typescript/blob/main/helpers.md
• Extended thinking is stable (not beta) and works with thinking: { type: "enabled", budget_tokens: N }.
• Streaming events: content_block_start with type: "thinking", followed by content_block_delta with delta.type: "thinking_delta" and delta.thinking field.

Braintrust docs status

Braintrust Anthropic integration docs mention streaming and metric collection but do not mention extended thinking (not_found).

• https://www.braintrust.dev/docs/reference/wrappers/anthropic

Local files inspected

• js/src/vendor-sdk-types/anthropic.ts
• js/src/wrappers/anthropic.ts
• js/src/instrumentation/plugins/anthropic-plugin.ts
• js/src/instrumentation/plugins/anthropic-channels.ts
• e2e/scenarios/anthropic-auto-instrumentation-node-hook/ (no thinking test cases)
• e2e/scenarios/wrap-anthropic-message-traces/ (no thinking test cases)

[lforst]

Fixed with #1695