fix: Add support for Gemini internal tools in tool formatting#923
Conversation
- Add explicit handling for Gemini internal tools (googleSearch, urlContext, codeExecution) - Update docstrings to document new capability - Resolves issue where Gemini internal tools were skipped with ''unsupported type'' error - Maintains backward compatibility with existing tool formats Closes #919 Co-authored-by: Mervin Praison <MervinPraison@users.noreply.github.com>
WalkthroughThe code updates enhance tool formatting logic in both Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Agent
participant LLM
participant OpenAIClient
User->>Agent: Initialize with tools (including Gemini internal tools)
Agent->>LLM: Pass tools to LLM
LLM->>LLM: _format_tools_for_litellm detects Gemini internal tools
LLM->>OpenAIClient: Pass formatted tools
OpenAIClient->>OpenAIClient: format_tools includes Gemini internal tools
OpenAIClient->>LLM: Return formatted tool list
LLM->>Agent: Tools ready for completion call
Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes found. Possibly related PRs
Suggested labels
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (2)
🧰 Additional context used🧠 Learnings (1)📓 Common learnings🔇 Additional comments (2)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
@claude review this pull request and do a detailed analysis and fix this if the existing code doesn't have the solution implemented. Making sure it has backward compatibility, no existing features removed. After making those changes, again review the applied changes. Use @web to search if you dont know any information or to find the latest documentation or to find the latest version. Run the code if you think you need to run it to test it. Minimal code change to start with if required any changes. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @MervinPraison, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request enhances the tool formatting capabilities within the praisonai-agents library to properly recognize and process specific internal tools used by Gemini models. By integrating explicit handling for googleSearch, urlContext, and codeExecution, it addresses a previous limitation where these tools were incorrectly rejected, thereby improving the robustness and functionality of tool integration with Gemini.
Highlights
- Gemini Tool Support: Added explicit handling for Gemini internal tools (
googleSearch,urlContext,codeExecution) within the tool formatting logic in bothllm.pyandopenai_client.py. - Bug Fix: Resolved an issue where these Gemini internal tools were previously skipped due to an 'unsupported type' error, ensuring they are now correctly processed.
- Documentation Update: Updated docstrings for the
_format_tools_for_litellmandformat_toolsmethods to clearly document the newly supported Gemini internal tools. - Compatibility: Ensured that the changes maintain backward compatibility with existing tool formats, preventing regressions for previously supported tool types.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
Claude finished @MervinPraison's task —— View job PR Review and Analysis ✅Starting detailed analysis of PR #923: "Add support for Gemini internal tools in tool formatting" Todo List:
Analysis Summary:Issue Context: The issue #919 shows that Gemini internal tools like Changes Made:
Code Review:✅ Implementation Quality:
✅ Test Results:
✅ Code Quality:
Security & Best Practices:✅ Security: Implementation is safe:
✅ Maintainability:
Final Assessment:✅ APPROVED - This PR successfully resolves issue #919 with a clean, minimal implementation that:
The implementation correctly addresses the "Skipping tool of unsupported type" error from the original issue while maintaining all existing functionality. Recommendation: Merge this PR as-is. The implementation is solid and ready for production. |
There was a problem hiding this comment.
Code Review
This pull request adds support for Gemini internal tools by adding specific handling for them in _format_tools_for_litellm and format_tools. The changes correctly identify and process these tools while maintaining backward compatibility.
My main feedback points are:
- A set of supported tools is being created inside a loop, which is inefficient. This should be defined as a constant.
- The logic for handling Gemini tools is duplicated across two files (
llm.pyandopenai_client.py), which will make future maintenance more difficult. I've recommended refactoring this into a shared utility function.
Addressing these points will improve the code's performance and maintainability.
| elif isinstance(tool, dict) and len(tool) == 1: | ||
| tool_name = next(iter(tool.keys())) | ||
| # List of supported Gemini internal tools | ||
| gemini_internal_tools = {'googleSearch', 'urlContext', 'codeExecution'} | ||
| if tool_name in gemini_internal_tools: | ||
| logging.debug(f"Using Gemini internal tool: {tool_name}") | ||
| formatted_tools.append(tool) | ||
| else: | ||
| logging.debug(f"Skipping unknown tool: {tool_name}") |
There was a problem hiding this comment.
This logic for handling Gemini internal tools is duplicated from llm.py (lines 593-601). Code duplication makes maintenance difficult. If you need to add a new Gemini tool, you'll have to remember to update it in both places.
I recommend creating a single, shared utility function to handle this logic. This function could be placed in a suitable shared module, like model_capabilities.py or a new utility file. This would also be the perfect place to define the gemini_internal_tools set as a single constant.
| elif isinstance(tool, dict) and len(tool) == 1: | ||
| tool_name = next(iter(tool.keys())) | ||
| # List of supported Gemini internal tools | ||
| gemini_internal_tools = {'googleSearch', 'urlContext', 'codeExecution'} |
There was a problem hiding this comment.
This set of tool names is being re-created inside the loop for every tool that is a single-key dictionary. For better performance and to follow best practices for constants, gemini_internal_tools should be defined once, outside the for loop. A module-level or class-level constant would be ideal.
This logic, including the constant definition, is duplicated in openai_client.py. I've left a more detailed comment there about refactoring to avoid duplication.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/praisonai-agents/praisonaiagents/llm/openai_client.py (1)
408-417: Solid implementation with good backward compatibility.The logic correctly handles Gemini internal tools by:
- Validating single-key dictionaries
- Checking against known tool types
- Preserving original tool structure
- Providing appropriate debug logging
Consider extracting the hardcoded tool names to a module-level constant for better maintainability:
+# Supported Gemini internal tools +GEMINI_INTERNAL_TOOLS = {'googleSearch', 'urlContext', 'codeExecution'} + class OpenAIClient:Then update the validation:
- # List of supported Gemini internal tools - gemini_internal_tools = {'googleSearch', 'urlContext', 'codeExecution'} - if tool_name in gemini_internal_tools: + if tool_name in GEMINI_INTERNAL_TOOLS:
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/praisonai-agents/praisonaiagents/llm/llm.py(2 hunks)src/praisonai-agents/praisonaiagents/llm/openai_client.py(2 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-ts/.cursorrules:0-0
Timestamp: 2025-06-30T10:05:51.843Z
Learning: Applies to src/praisonai-ts/src/llm/llm.ts : Replace all references to 'LLM' or 'litellm' with 'aisdk' usage for large language model calls in Node.js/TypeScript code.
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-ts/.windsurfrules:0-0
Timestamp: 2025-06-30T10:06:44.129Z
Learning: Applies to src/praisonai-ts/src/{llm,agent,agents,task}/**/*.ts : Replace all references to 'LLM' or 'litellm' with 'aisdk' usage in TypeScript code.
src/praisonai-agents/praisonaiagents/llm/openai_client.py (1)
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-agents/CLAUDE.md:0-0
Timestamp: 2025-06-30T10:06:17.673Z
Learning: Use the `@tool` decorator for simple function-based tools and inherit from `BaseTool` for class-based tools in the tool system.
src/praisonai-agents/praisonaiagents/llm/llm.py (5)
<retrieved_learning>
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-ts/.cursorrules:0-0
Timestamp: 2025-06-30T10:05:51.843Z
Learning: Applies to src/praisonai-ts/src/llm/llm.ts : Replace all references to 'LLM' or 'litellm' with 'aisdk' usage for large language model calls in Node.js/TypeScript code.
</retrieved_learning>
<retrieved_learning>
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-ts/.cursorrules:0-0
Timestamp: 2025-06-30T10:05:51.843Z
Learning: Applies to src/praisonai-ts/src/llm/llm.ts : The 'LLM' class in 'llm.ts' should wrap 'aisdk.generateText' calls for generating text responses.
</retrieved_learning>
<retrieved_learning>
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-ts/.windsurfrules:0-0
Timestamp: 2025-06-30T10:06:44.129Z
Learning: Applies to src/praisonai-ts/src/{llm,agent,agents,task}/**/*.ts : Replace all references to 'LLM' or 'litellm' with 'aisdk' usage in TypeScript code.
</retrieved_learning>
<retrieved_learning>
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-agents/CLAUDE.md:0-0
Timestamp: 2025-06-30T10:06:17.673Z
Learning: Use the unified LLM wrapper in praisonaiagents/llm/ for integrating with multiple LLM providers.
</retrieved_learning>
<retrieved_learning>
Learnt from: CR
PR: MervinPraison/PraisonAI#0
File: src/praisonai-agents/CLAUDE.md:0-0
Timestamp: 2025-06-30T10:06:17.673Z
Learning: Use the @tool decorator for simple function-based tools and inherit from BaseTool for class-based tools in the tool system.
</retrieved_learning>
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Cursor BugBot
- GitHub Check: Run tests and collect coverage
- GitHub Check: quick-test
- GitHub Check: test-core (3.11)
🔇 Additional comments (3)
src/praisonai-agents/praisonaiagents/llm/llm.py (2)
545-545: LGTM: Enhanced docstring documentationGood addition to document the new Gemini internal tools support capability.
592-601: LGTM: Well-implemented Gemini internal tools supportThe implementation correctly addresses the issue of Gemini internal tools being skipped. The approach is solid:
- Uses a whitelist for security (
gemini_internal_toolsset)- Validates single-key dictionary format as expected for Gemini tools
- Includes appropriate debug logging for both supported and unknown tools
- Directly appends recognized tools without modification, preserving their internal format
This change maintains backward compatibility while extending support for Gemini-specific functionality.
src/praisonai-agents/praisonaiagents/llm/openai_client.py (1)
363-363: Good documentation of the new capability.The docstring update clearly documents the support for Gemini internal tools with concrete examples of the expected format.
|
@claude review if the changes made are as per the requirements and make changes if issues still exist. Please create a Pull Request with gh tool with your changes. |
|
Claude finished @MervinPraison's task —— View job PR Review and Analysis ✅Starting detailed analysis of PR #923 and implementing improvements based on review feedback. Todo List:
Analysis Summary:Issue Context: The issue #919 shows that Gemini internal tools like Changes Made:
Code Review:✅ Implementation Quality:
✅ Improvements Applied:
✅ Test Results:
✅ Code Quality:
Security & Best Practices:✅ Security: Implementation is safe:
✅ Maintainability:
Final Assessment:✅ APPROVED - This PR successfully resolves issue #919 with a clean, minimal implementation that:
The implementation correctly addresses the "Skipping tool of unsupported type" error from the original issue while maintaining all existing functionality and incorporating reviewer feedback for better performance and maintainability. Recommendation: Merge this PR as-is. The implementation is solid and ready for production. |
- Extract GEMINI_INTERNAL_TOOLS constant to eliminate loop recreation - Create shared is_gemini_internal_tool() utility function - Remove code duplication between llm.py and openai_client.py - Improve maintainability and performance based on reviewer feedback Co-authored-by: Mervin Praison <MervinPraison@users.noreply.github.com>
1 participant
Summary
Fixes
Closes #919
Test Plan
Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation