Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Let the user interact with or evaluate the AI response. This interaction will in

## 2. Send Human Feedback

Use the [`patchLog()`](/ai-gateway/integrations/worker-binding-methods/#31-patchlog-send-feedback) method to provide feedback for the AI evaluation.
Use the [`patchLog()`](/ai-gateway/integrations/worker-binding-methods/#patchlog) method to provide feedback for the AI evaluation.

```javascript
await env.AI.gateway("my-gateway").patchLog(myLogId, {
Expand Down
47 changes: 35 additions & 12 deletions src/content/docs/ai-gateway/features/unified-billing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,33 @@ When your balance falls below the set threshold, AI Gateway will automatically a

## Use Unified Billing

Call any supported provider without passing an API Key. The request will automatically use Cloudflare's key and deduct credits from your account.
Unified Billing works in two ways: through the AI binding or through the HTTP API. Both deduct credits from your account automatically without requiring provider API keys.

:::note
Workers AI models (models prefixed with `@cf/`) routed through AI Gateway are not charged via Unified Billing. These models are billed through [Workers AI pricing](/workers-ai/platform/pricing/) instead. Unified Billing only applies to third-party provider models (such as OpenAI, Anthropic, and Google AI Studio).
:::

### AI binding

Call any model listed in the [model catalog](https://developers.cloudflare.com/ai/models/) using `env.AI.run()`. This includes both Workers AI models and third-party models from providers like OpenAI, Anthropic, and Google.

```typescript
const resp = await env.AI.run(
"openai/gpt-4.1-mini",
{
messages: [{ role: "user", content: "What is Cloudflare?" }],
},
{
gateway: { id: "my-gateway" },
},
);
```

Refer to the [binding reference](/ai-gateway/integrations/worker-binding-methods/) for the full API surface.

### HTTP API

For example, you can use the Unified API:
Call a supported provider through the AI Gateway REST API without passing a provider API key. Use the `cf-aig-authorization` header to authenticate with your Cloudflare API token.

```bash
curl -X POST https://gateway.ai.cloudflare.com/v1/$CLOUDFLARE_ACCOUNT_ID/default/compat/chat/completions \
Expand All @@ -70,6 +94,15 @@ curl -X POST https://gateway.ai.cloudflare.com/v1/$CLOUDFLARE_ACCOUNT_ID/default

The `default` gateway is created automatically on your first request. Replace `default` with a specific gateway ID if you have already created one.

The HTTP API supports the following providers through [provider-native endpoints](/ai-gateway/usage/providers/) and the [Unified API (chat completions)](/ai-gateway/usage/chat-completion/):

- [OpenAI](/ai-gateway/usage/providers/openai/)
- [Anthropic](/ai-gateway/usage/providers/anthropic/)
- [Google AI Studio](/ai-gateway/usage/providers/google-ai-studio/)
- [Google Vertex AI](/ai-gateway/usage/providers/vertex/)
- [xAI](/ai-gateway/usage/providers/grok/)
- [Groq](/ai-gateway/usage/providers/groq/)

### Spend limits

Set spend limits to prevent unexpected charges on your loaded credits. You can define daily, weekly, or monthly limits. When a limit is reached, the AI Gateway automatically stops processing requests until the period resets or you increase the limit.
Expand Down Expand Up @@ -129,13 +162,3 @@ curl -X POST https://gateway.ai.cloudflare.com/v1/$CLOUDFLARE_ACCOUNT_ID/{gatewa
]
}'
```

### Supported providers

Unified Billing supports the following AI providers:

- [OpenAI](/ai-gateway/usage/providers/openai/)
- [Anthropic](/ai-gateway/usage/providers/anthropic/)
- [Google AI Studio](/ai-gateway/usage/providers/google-ai-studio/)
- [xAI](/ai-gateway/usage/providers/grok/)
- [Groq](/ai-gateway/usage/providers/groq/)
3 changes: 1 addition & 2 deletions src/content/docs/ai-gateway/integrations/agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ pcx_content_type: navigation
title: Agents
external_link: /agents/
sidebar:
order: 10
order: 3
head: []
description: Build AI-powered Agents on Cloudflare
---

Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: Workers AI
title: Set up Workers AI with AI Gateway
pcx_content_type: tutorial
reviewed: 2024-10-17
sidebar:
order: 4
description: >-
This guide will walk you through setting up and deploying a Workers AI project. You will use Workers, an AI Gateway binding, and a large language model (LLM) to deploy your first AI-powered application on the Cloudflare global network.
---
Expand Down Expand Up @@ -56,8 +57,8 @@ To bind Workers AI to your Worker, add the following to the end of your [Wrangle
```jsonc
{
"ai": {
"binding": "AI"
}
"binding": "AI",
},
}
```

Expand Down Expand Up @@ -120,12 +121,7 @@ You will be prompted to log in after you run `wrangler dev`. When you run `npx w

```json
{
"response": "A fascinating question!\n\nThe phrase \"Hello, World!\" originates from a simple computer program written in the early days of programming. It is often attributed to Brian Kernighan, a Canadian computer scientist and a pioneer in the field of computer programming.\n\nIn the early 1970s, Kernighan, along with his colleague Dennis Ritchie, were working on the C programming language. They wanted to create a simple program that would output a message to the screen to demonstrate the basic structure of a program. They chose the phrase \"Hello, World!\" because it was a simple and recognizable message that would illustrate how a program could print text to the screen.\n\nThe exact code was written in the 5th edition of Kernighan and Ritchie's book \"The C Programming Language,\" published in 1988. The code, literally known as \"Hello, World!\" is as follows:\n\n```
main()
{
printf(\"Hello, World!\");
}
```\n\nThis code is still often used as a starting point for learning programming languages, as it demonstrates how to output a simple message to the console.\n\nThe phrase \"Hello, World!\" has since become a catch-all phrase to indicate the start of a new program or a small test program, and is widely used in computer science and programming education.\n\nSincerely, I'm glad I could help clarify the origin of this iconic phrase for you!"
"response": "A fascinating question!\n\nThe phrase \"Hello, World!\" originates from a simple computer program written in the early days of programming. It is often attributed to Brian Kernighan, a Canadian computer scientist and a pioneer in the field of computer programming.\n\nIn the early 1970s, Kernighan, along with his colleague Dennis Ritchie, were working on the C programming language. They wanted to create a simple program that would output a message to the screen to demonstrate the basic structure of a program. They chose the phrase \"Hello, World!\" because it was a simple and recognizable message that would illustrate how a program could print text to the screen.\n\nThe exact code was written in the 5th edition of Kernighan and Ritchie's book \"The C Programming Language,\" published in 1988. The code, literally known as \"Hello, World!\" is as follows:\n\n main()\n {\n printf(\"Hello, World!\");\n }\n\nThis code is still often used as a starting point for learning programming languages, as it demonstrates how to output a simple message to the console.\n\nThe phrase \"Hello, World!\" has since become a catch-all phrase to indicate the start of a new program or a small test program, and is widely used in computer science and programming education.\n\nSincerely, I'm glad I could help clarify the origin of this iconic phrase for you!"
}
```

Expand Down Expand Up @@ -154,3 +150,7 @@ https://hello-ai.<YOUR_SUBDOMAIN>.workers.dev
Your Worker will be deployed to your custom [`workers.dev`](/workers/configuration/routing/workers-dev/) subdomain. You can now visit the URL to run your AI Worker.

By completing this tutorial, you have created a Worker, connected it to Workers AI through an AI Gateway binding, and successfully ran an inference task using the Llama 3.1 model.

## Next steps

- [Workers bindings](/ai-gateway/integrations/worker-binding-methods/) — Call third-party models, access gateway methods, and integrate with AI SDKs.
44 changes: 32 additions & 12 deletions src/content/docs/ai-gateway/integrations/vercel-ai-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,10 @@
title: Vercel AI SDK
pcx_content_type: configuration
sidebar:
order: 3
order: 2
---

import {
Details,
Tabs,
TabItem
} from "~/components";
import { Details, Tabs, TabItem } from "~/components";
import CodeSnippets from "~/components/ai-gateway/code-examples.astro";

The [Vercel AI SDK](https://sdk.vercel.ai/) is a TypeScript library for building AI applications. The SDK supports many different AI providers, tools for streaming completions, and more.
Expand All @@ -21,20 +17,44 @@ To use Cloudflare AI Gateway with Vercel AI SDK, you will need to use the `ai-ga
npm install ai-gateway-provider
```

## Examples
## Examples

<CodeSnippets forceClient="aisdk" />

### AI binding with third-party models

If you are already using the [`workers-ai-provider`](https://www.npmjs.com/package/workers-ai-provider) package, you can route requests through AI Gateway to call third-party models without needing separate provider SDKs. Pass a `gateway` option with your gateway ID to `createWorkersAI`:

```ts
import { createWorkersAI } from "workers-ai-provider";
import { streamText } from "ai";

export default {
async fetch(request, env) {
const workersai = createWorkersAI({
binding: env.AI,
gateway: { id: "my-gateway" },
});

const result = streamText({
model: workersai("openai/gpt-4o"),
messages: [{ role: "user", content: "Write a short story" }],
});

return result.toTextStreamResponse();
},
} satisfies ExportedHandler<Env>;
```

This works with any [supported provider and model](/ai-gateway/usage/providers/) available through AI Gateway.

### Fallback Providers

To specify model or provider fallbacks to handle request failures and ensure reliability, you can pass an array of models to the `model` option.

```js title=""
const { text } = await generateText({
model: aigateway([
openai.chat("gpt-5.1"), anthropic("claude-sonnet-4-5")
]),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
model: aigateway([openai.chat("gpt-5.1"), anthropic("claude-sonnet-4-5")]),
prompt: "Write a vegetarian lasagna recipe for 4 people.",
});
```

Loading
Loading