You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/changelog/ai-gateway/2025-02-05-aig-request-handling.mdx
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,10 +8,10 @@ date: 2025-02-06T11:00:00Z
8
8
9
9
AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries), making it easier to keep your applications responsive and reliable.
10
10
11
-
Timeouts and retries can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/).
11
+
Timeouts and retries can be used on both the [Universal Endpoint](/ai-gateway/universal) or directly to a [supported provider](/ai-gateway/providers/).
12
12
13
13
**Request timeouts**
14
-
A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond.
14
+
A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond.
15
15
16
16
To set a request timeout directly to a provider, add a `cf-aig-request-timeout` header.
A [request retry](/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without intervening.
27
28
28
29
To set up request retries directly to a provider, add the following headers:
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/configuration/fallbacks.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ sidebar:
7
7
8
8
import { Render } from"~/components";
9
9
10
-
Specify model or provider fallbacks with your [Universal endpoint](/ai-gateway/providers/universal/) to handle request failures and ensure reliability.
10
+
Specify model or provider fallbacks with your [Universal endpoint](/ai-gateway/universal/) to handle request failures and ensure reliability.
11
11
12
12
Cloudflare can trigger your fallback provider in response to [request errors](#request-failures) or [predetermined request timeouts](/ai-gateway/configuration/request-handling#request-timeouts). The [response header `cf-aig-step`](#response-headercf-aig-step) indicates which step successfully processed the request.
13
13
@@ -38,7 +38,7 @@ You can add as many fallbacks as you need, just by adding another object in the
38
38
39
39
## Response header(cf-aig-step)
40
40
41
-
When using the [Universal endpoint](/ai-gateway/providers/universal/) with fallbacks, the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number. This header provides visibility into whether a fallback was triggered and which model ultimately processed the response.
41
+
When using the [Universal endpoint](/ai-gateway/universal/) with fallbacks, the response header `cf-aig-step` indicates which model successfully processed the request by returning the step number. This header provides visibility into whether a fallback was triggered and which model ultimately processed the response.
42
42
43
43
-`cf-aig-step:0` – The first (primary) model was used successfully.
44
44
-`cf-aig-step:1` – The request fell back to the second model.
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/configuration/request-handling.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ A timeout is set in milliseconds. Additionally, the timeout is based on when the
28
28
29
29
#### Universal Endpoint
30
30
31
-
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request timeout specifies the timeout duration for requests and triggers a fallback.
31
+
If set on a [Universal Endpoint](/ai-gateway/universal/), a request timeout specifies the timeout duration for requests and triggers a fallback.
32
32
33
33
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property within the provider-specific `config` object. Each provider can have a different `requestTimeout` value for granular customization.
34
34
@@ -123,7 +123,7 @@ On the final retry attempt, your gateway will wait until the request completes,
123
123
124
124
#### Universal endpoint
125
125
126
-
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request retry will automatically retry failed requests up to five times before triggering any configured fallbacks.
126
+
If set on a [Universal Endpoint](/ai-gateway/universal/), a request retry will automatically retry failed requests up to five times before triggering any configured fallbacks.
127
127
128
128
For a Universal Endpoint, configure the retry settings with the following properties in the provider-specific `config`:
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/glossary.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ AI Gateway supports a variety of headers to help you configure, customize, and m
16
16
Settings in AI Gateway can be configured at three levels: **Provider**, **Request**, and **Gateway**. Since the same settings can be configured in multiple locations, the following hierarchy determines which value is applied:
17
17
18
18
1.**Provider-level headers**:
19
-
Relevant only when using the [Universal Endpoint](/ai-gateway/providers/universal/), these headers take precedence over all other configurations.
19
+
Relevant only when using the [Universal Endpoint](/ai-gateway/universal/), these headers take precedence over all other configurations.
Additionally, you can access the latest request log ID with:
@@ -64,12 +68,12 @@ Once you have the gateway instance, you can use the following methods:
64
68
The `patchLog` method allows you to send feedback, score, and metadata for a specific log ID. All object properties are optional, so you can include any combination of the parameters:
The `run` method allows you to execute universal requests. Users can pass either a single universal request object or an array of them. This method supports all AI Gateway providers.
143
150
144
-
Refer to the [Universal endpoint documentation](/ai-gateway/providers/universal/) for details about the available inputs.
151
+
Refer to the [Universal endpoint documentation](/ai-gateway/universal/) for details about the available inputs.
145
152
146
153
```typescript
147
154
const resp =awaitgateway.run({
148
-
provider: "workers-ai",
149
-
endpoint: "@cf/meta/llama-3.1-8b-instruct",
150
-
headers: {
151
-
authorization: "Bearer my-api-token"
152
-
},
153
-
query: {
154
-
prompt: "tell me a joke"
155
-
}
155
+
provider: "workers-ai",
156
+
endpoint: "@cf/meta/llama-3.1-8b-instruct",
157
+
headers: {
158
+
authorization: "Bearer my-api-token",
159
+
},
160
+
query: {
161
+
prompt: "tell me a joke",
162
+
},
156
163
});
157
164
```
158
165
159
166
-**Returns**: `Promise<Response>`
160
-
-**Example Use Case**: Perform a [universal request](/ai-gateway/providers/universal/) to any supported provider.
167
+
-**Example Use Case**: Perform a [universal request](/ai-gateway/universal/) to any supported provider.
161
168
162
169
## Conclusion
163
170
@@ -168,4 +175,4 @@ With these AI Gateway binding methods, you can now:
168
175
- Get gateway URLs for direct API access with `getUrl`, making it easy to integrate with popular AI SDKs.
169
176
- Execute universal requests to any AI Gateway provider with `run`.
170
177
171
-
These methods offer greater flexibility and control over your AI integrations, empowering you to build more sophisticated applications on the Cloudflare Workers platform.
178
+
These methods offer greater flexibility and control over your AI integrations, empowering you to build more sophisticated applications on the Cloudflare Workers platform.
0 commit comments