Skip to content

feat(think): add opt-in, read-only HTTP fetch tool #1821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
threepointone merged 2 commits into from
Jun 27, 2026
Merged

feat(think): add opt-in, read-only HTTP fetch tool #1821

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2e0ed6b
feat(think): add opt-in, read-only HTTP fetch tool
threepointone Jun 27, 2026
937319d
fix(think): block IPv4-mapped IPv6 addresses in SSRF guard
threepointone Jun 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/think-fetch-tool.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@cloudflare/think": minor
---

Add an opt-in, read-only HTTP fetch capability for Think agents via the new `@cloudflare/think/tools/fetch` export and a `fetchTools` property on `Think`.

`createFetchTools()` generates a generic, allowlisted `fetch_url` tool plus one `fetch_<name>` tool per named service-binding/`Fetcher` target. It is `GET`-only with Workers-grounded SSRF defenses (private/loopback/link-local/`*.internal` blocking, URL normalization, credential rejection), separate download/model/workspace size limits (`maxBytes`, `maxModelChars`, `response: "workspace"` spill), an allowlist-aware redirect policy with cross-origin header stripping, a model header allowlist, and a `tool:fetch` observability event. Disabled by default.
1 change: 1 addition & 0 deletions design/test-coverage-matrix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ Legend: ✅ covered · ⚠️ partial / indirect · — none · 🚫 gated (opt-
| Voice | ✅ | ✅ | ✅ `voice/src/tests/` (SFU 🚫 gated) | ✅ `voice/src/react-tests/` | — | — | — |
| Shell / Workspace | ✅ | ✅ | ✅ `shell/src/tests/` (git clone 🚫) | — | — | — | — |
| Codemode (dynamic worker exec) | ✅ | ✅ | ✅ | ✅ `codemode/src/tests/` (browser) | — | ⚠️ `codemode/e2e/` (**not nightly**) | — |
| Think fetch tool (allowlist, SSRF, limits, redirects, auto-merge) | ✅ | ✅ | ✅ `think/src/tests/fetch-tools.test.ts` (+ `agents/fetch-tools.ts` fixture) | — | — | — | — |
| Shared-engine genericity (pi / tanstack adapters) | ✅ | ✅ | ✅ | — | ✅ `experimental/pi-recovery`, `tanstack-recovery` (workers-ai leg 🚫) | — | ✅ tanstack leg |

### Accepted coverage gaps
Expand Down
16 changes: 16 additions & 0 deletions design/think.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,21 @@ const executeTool = createExecuteTool({

The LLM writes JavaScript code. The tool sends it to a dynamic Worker isolate via `DynamicWorkerExecutor`. The sandbox can call workspace tools via `codemode.*` and optionally the full `state.*` filesystem API (`readFile`, `writeFile`, `glob`, `searchFiles`, `planEdits`, etc.). Fully isolated: no network access by default, configurable timeout.

### Fetch tools (`@cloudflare/think/tools/fetch`)

Opt-in, read-only HTTP reads. `createFetchTools()` generates a generic `fetch_url` tool (when a public `allowlist` is set) plus one `fetch_<name>` per binding target. Wired into Think via the `fetchTools` property, which auto-merges the generated tools between workspace tools and `getTools()` and adds a capability-prompt line; it injects `this.workspace` and a `tool:fetch` observability emit automatically.

```typescript
fetchTools = {
allowlist: ["https://developers.cloudflare.com/**"],
bindings: {
docsApi: { binding: this.env.DOCS_API, allowlist: ["/v1/docs/**"] }
}
};
```

Key decisions: named tools (not one polymorphic tool) so per-target policy is baked in; `GET`-only (mutations belong in approval-gated actions, and recovery replays a turn so non-idempotent egress would be unsafe); Workers-grounded SSRF defenses (private/loopback/link-local/`*.internal` blocked for the public path, credentials rejected, IPv4 shorthand normalized by the WHATWG URL parser); three size knobs (`maxBytes` download cap, `maxModelChars` text truncation, `response: "workspace"` spill); allowlist-aware redirect policy with cross-origin header stripping; and a markdown-first default `Accept`. Results are returned as structured `{ ok, ... }` values, never thrown.

### Browser tools (`@cloudflare/think/tools/browser`)

Two AI SDK tools for CDP-based browser automation:
Expand Down Expand Up @@ -532,6 +547,7 @@ Tests in `packages/think/src/tests/`, running inside the Workers runtime via `@c
| `@cloudflare/think/extensions` | `src/extensions/index.ts` | ExtensionManager, HostBridgeLoopback |
| `@cloudflare/think/tools/workspace` | `src/tools/workspace.ts` | File operation tool factories (for custom backends) |
| `@cloudflare/think/tools/execute` | `src/tools/execute.ts` | Sandboxed code execution tool |
| `@cloudflare/think/tools/fetch` | `src/tools/fetch.ts` | Opt-in allowlisted, read-only HTTP fetch tools |
| `@cloudflare/think/tools/browser` | `src/tools/browser.ts` | CDP browser automation tools (search + execute) |
| `@cloudflare/think/tools/extensions` | `src/tools/extensions.ts` | Extension management AI tools |

Expand Down
52 changes: 52 additions & 0 deletions docs/think/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -852,6 +852,7 @@ path.
| `getSkills()` | `[]` | Return Agent Skills sources for on-demand skill activation |
| `getSkillScriptRunner()` | `null` | Enable the optional `run_skill_script` tool |
| `workspaceBash` | `true` | Include or configure the default workspace `bash` tool |
| `fetchTools` | `false` | Opt-in allowlisted, read-only HTTP fetch tools (`fetch_url` + per-binding `fetch_<name>`). Set to a config object; see [Fetch tool](#fetch-tool) |
| `messageConcurrency` | `"queue"` | How overlapping submits behave — see [Client Tools](./client-tools.md) |
| `waitForMcpConnections` | `false` | Wait for MCP servers before inference |
| `chatRecovery` | `true` | Wrap turns in `runFiber` for durable execution, including sub-agent turns. Set to `{ maxAttempts, stableTimeoutMs, terminalMessage, onExhausted }` to tune bounded recovery. |
Expand Down Expand Up @@ -1189,6 +1190,56 @@ handler action before the failure is logged. The next occurrence is still
scheduled after the action succeeds or exhausts its retries, so failed
occurrences do not block future runs.

## Fetch tool

Think can give the model a conservative, **read-only** way to read HTTP resources. It is **off by default**. Set the `fetchTools` property for static config, or call `createFetchTools()` inside `getTools()` for per-tenant/dynamic allowlists (it runs every turn). When configured, Think registers a generic `fetch_url` tool (when a public `allowlist` is provided) plus one `fetch_<name>` tool per binding target, and advertises the capability in the system prompt.

```typescript
export class DocsAgent extends Think<Env> {
getModel() {
/* ... */
}

fetchTools = {
allowlist: ["https://developers.cloudflare.com/**"],
bindings: {
docsApi: {
binding: this.env.DOCS_API, // a service binding / Fetcher
description: "Internal docs search API.",
allowlist: ["/v1/docs/**"],
headers: { "x-agent": "think" } // fixed, server-side, never model-set
}
}
};
}
```

The model sees named tools — `fetch_url({ url, response?, headers? })` and `fetch_docsApi({ path, response?, headers? })` — rather than one polymorphic tool, so per-target policy is baked into each tool.

**Safety model.** The threat surface is the Workers runtime: reaching loopback/`.internal`/internally bound targets, allowlist-bypass tricks, prompt-injected URLs, credential leakage, and context/storage bloat.

- **Read-only** — `GET` only. Mutations belong in explicit, approval-gated [Actions](#actions), not here.
- **Allowlisted** — every request must match the configured allowlist. URLs are normalized (host lowercased, credentials rejected, paths resolved) before matching, and private/loopback/link-local/`*.internal` targets are blocked for `fetch_url` even if the allowlist is misconfigured.
- **Bounded** — `maxBytes` caps the download, `maxModelChars` truncates the model-facing text (`truncated: true`), and `response: "workspace"` (or `spillToWorkspace: true` in auto mode) writes large or binary bodies to a workspace file so the transcript stays small.
- **Header-safe** — only headers in `modelHeaderAllowlist` (default `accept`, `accept-language`, `range`) may be set by the model; fixed binding headers are server-side only and are stripped on cross-origin redirects.
- **Markdown-first** — a weighted default `Accept` header (`text/markdown` → `text/plain` → `application/json` → `text/html` → `*/*;q=0.1`) nudges content-negotiating endpoints (docs platforms, `llms.txt`-style endpoints) to return clean markdown instead of HTML, while still accepting anything so a strict server never returns `406`. Override per call (the model can set `accept`) or globally via `defaultAccept` (`""` disables it).
- **Redirects** — `followRedirects` (`allowlisted` by default) follows a redirect only when the final URL is still allowlisted; binding targets never follow cross-origin redirects.

Results are structured values (never thrown). Success carries `{ ok: true, status, finalUrl, contentType, bytes, truncated, response, body?/json?/path? }`; failure carries `{ ok: false, code, message, status?, finalUrl? }` where `code` is one of `disallowed_url`, `disallowed_redirect`, `timeout`, `aborted`, `non_2xx`, `unsupported_content_type`, `invalid_json`, `too_large`, `request_failed`. A `tool:fetch` observability event fires on every call, including blocked attempts, for audit.

**Allowlist semantics.** A bare origin (`https://example.com`) matches that origin and every subpath. Patterns are globs — `**` matches any characters (including `/`) and `*` matches any character except `/`; a pattern with an explicit path and no glob matches that path literally (`https://x.com/v1` matches only `/v1`). Matching ignores the query string and fragment (only scheme + host + port + path are compared), though the original query/fragment are still sent. Binding allowlists should be path-based (`/v1/docs/**`). Note that `json` responses are bounded by `maxBytes` (only `text` is truncated by `maxModelChars`), so for large JSON APIs lower `maxBytes` or use `response: "workspace"`.

You do not need new machinery to gate egress per call: `beforeToolCall` can `block` or `substitute` a fetch, and channel `tools(...)` policy can narrow which fetch tools are available.

**When to use what:**

| Capability | Use it for |
| -------------------------------- | ---------------------------------------------------------------------------------------- |
| Fetch tool | Reading a known, allowlisted URL or service binding; no code generation |
| `createExecuteTool()` (codemode) | Composing/transforming several calls in sandboxed code (`globalOutbound`) |
| Browser Run (`tools/browser`) | Rendered pages, auth flows, screenshots, CDP automation |
| Typed tools / `agentTool()` | Calling a `WorkerEntrypoint`/DO method with a typed schema, or delegating to a sub-agent |

## Session Integration

Think uses [Session](https://github.com/cloudflare/agents/blob/main/docs/agents/sessions.md) for conversation storage. Override `configureSession` to add persistent memory, compaction, search, and skills:
Expand Down Expand Up @@ -1228,6 +1279,7 @@ Think's `this.messages` getter reads directly from Session's tree-structured sto
| `@cloudflare/think/messengers/telegram` | Telegram messenger provider and delivery helpers |
| `@cloudflare/think/workflows` | `ThinkWorkflow`, `step.prompt()` — Workflow prompts |
| `@cloudflare/think/tools/workspace` | `createWorkspaceTools()` — for custom storage backends |
| `@cloudflare/think/tools/fetch` | `createFetchTools()` — opt-in allowlisted HTTP reads |
| `@cloudflare/think/tools/execute` | `createExecuteTool()` — sandboxed code execution via codemode |
| `@cloudflare/think/tools/extensions` | `createExtensionTools()` — LLM-driven extension loading |
| `@cloudflare/think/extensions` | `ExtensionManager`, `HostBridgeLoopback` — extension runtime |
Expand Down
1 change: 1 addition & 0 deletions examples/assistant/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ the sub-agent routing primitive from `agents`.
- **Sandboxed code execution** — `createExecuteTool` lets the LLM write and run JavaScript in a Dynamic Worker via `@cloudflare/codemode`
- **Browser automation** — the `BROWSER` binding gives the execute sandbox a `cdp.*` connector: a real browser driven over the Chrome DevTools Protocol, with durable sessions the model can promote and reuse across messages
- **Stateless browsing (Quick Actions)** — `createQuickActionTools` adds `browser_markdown`, `browser_extract`, `browser_links`, and `browser_scrape` for one-shot page reads (no CDP session or sandbox); the model uses these for simple reads and `cdp.*` for interactive automation
- **HTTP fetch** — the `fetchTools` property registers a read-only `fetch_url` tool; this demo allows any public URL (`http(s)://**`, though private/loopback targets are always refused), and large/binary responses spill into the shared workspace (`spillToWorkspace`) so they show up in the file browser instead of bloating the transcript
- **Self-authored extensions** — `extensionLoader` + `createExtensionTools` let the agent create new tools at runtime
- **Persistent memory** — context blocks (`soul`, `memory`) the model can read and write across sessions
- **Non-destructive compaction** — older messages summarized when context overflows, originals preserved
Expand Down
Loading
Loading