Skip to content

Commit fed42cf

Browse files
authored
Update request-handling.mdx
slight updates re: -what to use when using universal -wording
1 parent c5af8b4 commit fed42cf

File tree

1 file changed

+6
-11
lines changed

1 file changed

+6
-11
lines changed

src/content/docs/ai-gateway/configuration/request-handling.mdx

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ A timeout is set in milliseconds. Additionaly, the timeout is based on when the
3030

3131
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request timeout specifies the timeout duration for requests and triggers a fallback.
3232

33-
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property either as a universal attribute or within the provider-specific `config` object.
33+
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property within the provider-specific `config` object. Each provider can have a different `requestTimeout` value for granular customization.
3434

3535
```bash title="Provider-level config" {12-14} collapse={15-48}
3636
curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
@@ -41,8 +41,7 @@ curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
4141
"endpoint": "@cf/meta/llama-3.1-8b-instruct",
4242
"headers": {
4343
"Authorization": "Bearer {cloudflare_token}",
44-
"Content-Type": "application/json",
45-
"cf-aig-request-timeout": "2000"
44+
"Content-Type": "application/json"
4645
},
4746
"config": {
4847
"requestTimeout": 1000
@@ -55,7 +54,7 @@ curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
5554
},
5655
{
5756
"role": "user",
58-
"content": "What is Cloduflare?"
57+
"content": "What is Cloudflare?"
5958
}
6059
]
6160
}
@@ -86,10 +85,6 @@ curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
8685
]'
8786
```
8887

89-
To further customize request handling, you can include unique `RequestTimeout` values for each provider and a default across your Universal Endpoint.
90-
91-
In this case, the most specific value takes precedence. A timeout value on a specific provider would take precedence over the one on the endpoint itself.
92-
9388
#### Direct provider
9489

9590
If set on a [provider](/ai-gateway/providers/) request, request timeout specifies the timeout duration for a request and - if exceeded - returns an error.
@@ -130,7 +125,7 @@ On the final retry attempt, your gateway will wait until the request completes,
130125

131126
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request retry will automatically retry failed requests up to five times before triggering any configured fallbacks.
132127

133-
For a Universal Endpoint, configure the timeout value by setting the following properties in the overall or provider-specific `config`:
128+
For a Universal Endpoint, configure the retry settings with the following properties in the provider-specific `config`:
134129

135130
```json
136131
config:{
@@ -140,11 +135,11 @@ config:{
140135
}
141136
```
142137

143-
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), the values can interact with each other to provide more customized logic.
138+
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), each provider can have a different retry settings for granular customization.
144139

145140
#### Direct provider
146141

147-
If set on a [provider](/ai-gateway/providers/) request, request timeout specifies the timeout duration for a request and - if exceeded - returns an error.
142+
If set on a [provider](/ai-gateway/providers/) request, a request retry will automatically retry failed requests up to five times. On the final retry attempt, your gateway will wait until the request completes, regardless of how long it takes.
148143

149144
For a provider-specific endpoint, configure the timeout value by adding different header values:
150145

0 commit comments

Comments
 (0)