Skip to content

Commit 56f4f92

Browse files
authored
Add ai gateway binding methods (#19484)
1 parent 3f312bb commit 56f4f92

File tree

1 file changed

+122
-0
lines changed

1 file changed

+122
-0
lines changed
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
---
2+
title: AI Gateway Binding Methods
3+
pcx_content_type: tutorial
4+
updated: 2025-01-28
5+
---
6+
7+
import { Render, PackageManagers } from "~/components";
8+
9+
This guide provides an overview of how to use the latest Cloudflare Workers AI Gateway binding methods. You will learn how to set up an AI Gateway binding, access new methods, and integrate them into your Workers.
10+
11+
## Prerequisites
12+
13+
- Ensure your Worker project is configured with an AI Gateway binding in `wrangler.json`.
14+
- Install and use the `@cloudflare/workers-types` library, version `4.20250124.3` or above.
15+
16+
## 1. Add an AI Binding to your Worker
17+
18+
To connect your Worker to Workers AI, add the following to your `wrangler.toml` file:
19+
20+
import { WranglerConfig } from "~/components";
21+
22+
<WranglerConfig>
23+
24+
```toml title="wrangler.toml"
25+
[ai]
26+
binding = "AI"
27+
```
28+
29+
</WranglerConfig>
30+
31+
This configuration sets up the AI binding accessible in your Worker code as `env.AI`.
32+
33+
## 2. Basic Usage with Workers AI + Gateway
34+
35+
To perform an inference task using Workers AI and an AI Gateway, you can use the following code:
36+
37+
```typescript title="src/index.ts"
38+
const resp = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
39+
prompt: "tell me a joke"
40+
}, {
41+
gateway: {
42+
id: "my-gateway"
43+
}
44+
});
45+
```
46+
47+
Additionally, you can access the latest request log ID with:
48+
49+
```typescript
50+
const myLogId = env.AI.aiGatewayLogId;
51+
```
52+
53+
## 3. Access the Gateway Binding
54+
55+
You can access your AI Gateway binding using the following code:
56+
57+
```typescript
58+
const gateway = env.AI.gateway("my-gateway");
59+
```
60+
61+
Once you have the gateway instance, you can use the following methods:
62+
63+
### 3.1. `patchLog`: Send Feedback
64+
65+
The `patchLog` method allows you to send feedback, score, and metadata for a specific log ID. All object properties are optional, so you can include any combination of the parameters:
66+
67+
```typescript
68+
gateway.patchLog('my-log-id', {
69+
feedback: 1,
70+
score: 100,
71+
metadata: {
72+
user: "123"
73+
}
74+
});
75+
```
76+
77+
- **Returns**: `Promise<void>` (Make sure to `await` the request.)
78+
- **Example Use Case**: Update a log entry with user feedback or additional metadata.
79+
80+
### 3.2. `getLog`: Read Log Details
81+
82+
The `getLog` method retrieves details of a specific log ID. It returns an object of type `Promise<AiGatewayLog>`. You can import the `AiGatewayLog` type from the `@cloudflare/workers-types` library.
83+
84+
```typescript
85+
const log = await gateway.getLog("my-log-id");
86+
```
87+
88+
- **Returns**: `Promise<AiGatewayLog>`
89+
- **Example Use Case**: Retrieve log information for debugging or analytics.
90+
91+
### 3.3. `run`: Universal Requests
92+
93+
The `run` method allows you to execute universal requests. Users can pass either a single universal request object or an array of them. This method supports all AI Gateway providers.
94+
95+
Refer to the [Universal endpoint documentation](/ai-gateway/providers/universal/) for details about the available inputs.
96+
97+
```typescript
98+
const resp = await gateway.run({
99+
provider: "workers-ai",
100+
endpoint: "@cf/meta/llama-3.1-8b-instruct",
101+
headers: {
102+
authorization: "Bearer my-api-token"
103+
},
104+
query: {
105+
prompt: "tell me a joke"
106+
}
107+
});
108+
```
109+
110+
- **Returns**: `Promise<Response>`
111+
- **Example Use Case**: Perform a universal AI request to any supported provider.
112+
113+
## Conclusion
114+
115+
With the new AI Gateway binding methods, you can now:
116+
117+
- Send feedback and update metadata with `patchLog`.
118+
- Retrieve detailed log information using `getLog`.
119+
- Execute universal requests to any AI Gateway provider with `run`.
120+
121+
These methods offer greater flexibility and control over your AI integrations, empowering you to build more sophisticated applications on the Cloudflare Workers platform.
122+

0 commit comments

Comments
 (0)