feat: support native tools in OpenAI Responses provider

[Star-Moon10]

Closes #9530

Modifications / 改动点

• Add dashboard configuration and defaults for native Responses API tools: web_search (context size and allowed domains), file_search (vector store IDs), code_interpreter, image_generation, and tool_choice.

• Combine configured native tools with AstrBot function schemas while preserving raw tools from custom_extra_body.

• Parse URL/file citations and image-generation results into response messages.

• Backfill defaults for existing openai_responses provider sources.

• Add unit coverage for native tool payloads and response parsing.

• This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

• uv run ruff check
• uv run pytest tests/test_openai_responses_source.py -q — 10 passed
• corepack pnpm@10.28.2 run build in dashboard/

---

Checklist / 检查清单

• 😊 This feature was discussed in Support OpenAI Responses API built-in tools ( web_search , file_search , code_interpreter ) in openai_responses provider #9530. / 新功能已在 Support OpenAI Responses API built-in tools ( web_search , file_search , code_interpreter ) in openai_responses provider #9530 中讨论。
• 👀 The changes have been tested and results are included above. / 更改已测试，结果已列于上方。
• 🤓 No new dependencies were introduced. / 未引入新依赖。
• 😮 No malicious code was introduced. / 未引入恶意代码。

Summary by Sourcery

Add configurable support for OpenAI Responses native tools and surface their outputs in parsed responses.

New Features:

• Allow configuring and enabling native OpenAI Responses tools (web_search, file_search, code_interpreter, image_generation) and tool_choice via provider and dashboard settings.
• Combine native Responses tools with AstrBot function tools while preserving any custom tools configured in extra request bodies.
• Parse URL/file citations and image-generation results from Responses API outputs into the final message chain, including inline source listings.

Enhancements:

• Backfill default configuration values for Responses-native tool settings in existing openai_responses provider templates and dashboard sources.

Tests:

• Add unit tests to verify tool combination behavior, default tool_choice handling, and parsing of citations and generated images from Responses API responses.

[sourcery-ai]

Hey - I've left some high level feedback:

• The tool_choice normalization logic is duplicated in both _query and _query_stream; consider extracting this into a small helper to keep the behavior consistent and make future changes less error-prone.
• When combining dashboard-configured native tools with custom_extra_body.tools in _build_response_tools, you may want to deduplicate tools (e.g., duplicate web_search or file_search entries) to avoid sending conflicting or redundant tools to the Responses API.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `tool_choice` normalization logic is duplicated in both `_query` and `_query_stream`; consider extracting this into a small helper to keep the behavior consistent and make future changes less error-prone.
- When combining dashboard-configured native tools with `custom_extra_body.tools` in `_build_response_tools`, you may want to deduplicate tools (e.g., duplicate `web_search` or `file_search` entries) to avoid sending conflicting or redundant tools to the Responses API.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[Copilot]

Pull request overview

This PR adds first-class support for OpenAI Responses API native tools in AstrBot’s openai_responses provider, exposing configuration via the dashboard and parsing native tool outputs (citations and generated images) into AstrBot’s message chain.

Changes:

• Add provider + dashboard configuration defaults for Responses native tools (web_search, file_search, code_interpreter, image_generation) and tool_choice.
• Combine configured native tools with AstrBot function tools while preserving raw custom_extra_body tool entries.
• Parse URL/file citations and image-generation results into the final MessageChain, with unit tests covering tool payload composition and response parsing.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
astrbot/core/provider/sources/openai_responses_source.py	Builds mixed tool payloads and parses citations + image generation output into MessageChain.
astrbot/core/config/default.py	Adds default config/template entries and dashboard metadata schema for Responses native tool settings.
dashboard/src/composables/useProviderSources.ts	Backfills defaults for existing openai_responses provider sources in the dashboard editor.
dashboard/src/i18n/locales/en-US/features/config-metadata.json	Adds English dashboard metadata labels/hints for the new Responses tool settings.
dashboard/src/i18n/locales/zh-CN/features/config-metadata.json	Adds Chinese dashboard metadata labels/hints for the new Responses tool settings.
tests/test_openai_responses_source.py	Adds coverage for tool merging/tool_choice behavior and parsing citations + generated images.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (3)

astrbot/core/provider/sources/openai_responses_source.py:158

• responses_file_search_vector_store_ids values are forwarded without trimming whitespace. Stripping IDs avoids sending malformed vector_store_ids (e.g. copy/pasted values with spaces) and prevents avoidable request failures.

            vector_store_ids = [
                vector_store_id
                for vector_store_id in vector_store_ids
                if isinstance(vector_store_id, str) and vector_store_id
            ]

dashboard/src/i18n/locales/en-US/features/config-metadata.json:1214

• New provider config keys were added here, but the repository also ships ru-RU locale files (see dashboard/src/i18n/locales/ru-RU/...). Without adding the same keys there, the UI will show missing-translation placeholders for Russian users. Please add corresponding responses_* entries to the ru-RU features/config-metadata.json to keep locales in sync.

      "responses_web_search": {
        "description": "Enable Responses web search",
        "hint": "Use the OpenAI Responses API web_search tool for online retrieval. Only applies to openai_responses providers."
      },

astrbot/core/provider/sources/openai_responses_source.py:145

• responses_web_search_allowed_domains values are forwarded without trimming whitespace. If users enter domains with leading/trailing spaces (common when copy/pasting), the provider will send invalid domain filters to the Responses API. Strip domains before including them.

                domains = [
                    domain
                    for domain in allowed_domains
                    if isinstance(domain, str) and domain
                ]

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

astrbot/core/provider/sources/openai_responses_source.py:214

• tool_choice handling has two issues: (1) if custom_extra_body.tool_choice (or a per-request payloads['tool_choice']) is a dict/object tool selector, the current tool_choice not in {..} checks will raise TypeError: unhashable type: 'dict'; (2) the default responses_tool_choice='auto' will always override a valid custom_extra_body.tool_choice, which is a behavior change for existing configs that relied on pass-through tool_choice. Consider only validating membership for string values and let custom/per-request tool_choice win unless an explicit non-default responses_tool_choice is set.

            tool_choice = self.provider_config.get("responses_tool_choice")
            if tool_choice not in {"auto", "required", "none"}:
                tool_choice = payloads.get("tool_choice", custom_tool_choice)
            if tool_choice not in {"auto", "required", "none"}:
                tool_choice = "auto"

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.