Skip to content

Add Google API toolset for agents to access 1000+ APIs#70144

Open
gopidesupavan wants to merge 6 commits into
apache:mainfrom
gopidesupavan:gcp-toolset
Open

Add Google API toolset for agents to access 1000+ APIs#70144
gopidesupavan wants to merge 6 commits into
apache:mainfrom
gopidesupavan:gcp-toolset

Conversation

@gopidesupavan

Copy link
Copy Markdown
Member

Add GoogleCloudToolset to the common.ai provider so agents can call explicitly allow-listed Google APIs through pydantic-ai tools.

The toolset exposes three tools for agents:

  • list_gcp_methods to show the allowed Google API methods
  • describe_gcp_method to inspect parameters and schemas before calling
  • call_gcp to execute an allowed method and return JSON

Design

The toolset is intentionally generic rather than service-specific. Instead of implementing separate wrappers for GCS, BigQuery, Pub/Sub, Cloud Run, and other Google services, it uses Google Discovery REST metadata to resolve method names, parameters, request schemas, and response schemas at runtime. This keeps the provider small while still allowing agents to work with the Google APIs that publish Discovery documents.

Access is deny-by-default. Dag authors must pass allowed_methods, for example storage/v1:objects.list or bigquery/v2:jobs.query. Method-level wildcards are supported, but the API and version must be explicit so an agent cannot move from an approved stable API to an alpha or beta surface by changing tool input.

Credentials, impersonation, and project selection come from the Airflow Google connection through GoogleBaseHook. They are not exposed as tool arguments, so the model can choose only the allowed API method and request parameters, not which credentials or project to use.

The toolset also adds guardrails around high-risk behavior:

  • Credential-returning methods are not matched by wildcards and must be allow-listed verbatim.
  • Media upload/download is rejected.
  • Pagination is bounded with max_pages.
  • Returned payload size is bounded with max_output_bytes.
  • Project parameters can be pinned to the connection project by default.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@gopidesupavan

Copy link
Copy Markdown
Member Author
Screenshot 2026-07-20 at 19 52 59 Screenshot 2026-07-20 at 20 38 16 Screenshot 2026-07-20 at 20 38 23 Screenshot 2026-07-20 at 20 39 06 Screenshot 2026-07-20 at 20 39 14

@gopidesupavan

Copy link
Copy Markdown
Member Author

follows same approach here #70087


__all__ = ["AWSToolset", "HookToolset", "MCPToolset", "SQLToolset", "airflow_toolset_to_langchain_tools"]
__all__ = [
"GoogleCloudToolset",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just have it use HookToolset with GCPHook?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HookToolset is still useful when the workflow maps to existing hook methods. The gap here is that it only exposes Python methods that a hook already wraps, while this toolset exposes allowed Google API client methods directly.

For example, an incident triage agent might need to check recent BigQuery jobs, Dataflow jobs, and GCS object arrival. With GoogleCloudToolset, the Dag author can allow-list the exact API methods, such as bigquery/v2:jobs.list, bigquery/v2:jobs.get, dataflow/v1b3:projects.locations.jobs.list, and storage/v1:objects.list.

Doing the same with HookToolset depends on each service hook having the right method and returning data in a useful shape for the agent. Some APIs also have no dedicated hook coverage. For example, Cloud Monitoring time series reads can be exposed as monitoring/v3:projects.timeSeries.list, while the existing Stackdriver hook mainly wraps alert policy and notification channel operations.

So this is not replacing HookToolset; it is for cases where the agent should call selected Google REST API methods directly, instead of being limited to the Python methods currently wrapped by Airflow hooks

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One example I tested was this prompt:

Pipeline is slow today in project `playground-s-11-af79665a`. Check active Cloud Monitoring alert policies, list recent BigQuery pipeline rows, and confirm raw files arrived in `gs://gcp-toolset-demo-af79665a/raw/`. Then compare Cloud Monitoring metrics for BigQuery query count and GCS request count for today versus yesterday.

With HookToolset, the agent handled the parts backed by existing hook methods: alert policies, BigQuery rows, and GCS object listing. But it could not answer the Monitoring time-series part because that metric-read operation is not exposed by the hook methods available to it.

With GoogleCloudToolset, the agent completed the full request by calling the relevant Google REST API methods directly, including monitoring/v3:projects.timeSeries.list.

This is just one example; in other multi-service troubleshooting questions the missing piece may be a different Google API method. ideally here it is not to replace HookToolset, but to support cases where we want to expose selected Google API methods without needing a dedicated Airflow hook wrapper for each one.

@gopidesupavan
gopidesupavan marked this pull request as ready for review July 21, 2026 17:20
@gopidesupavan
gopidesupavan requested a review from Lee-W as a code owner July 21, 2026 17:20
@gopidesupavan

Copy link
Copy Markdown
Member Author

I’d welcome feedback from the Google provider team @MaksYermak

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants