Skip to content

Commit 9cb76f6

Browse files
committed
Added example
1 parent 01021cf commit 9cb76f6

File tree

1 file changed

+59
-2
lines changed

1 file changed

+59
-2
lines changed

src/content/docs/ai-gateway/configuration/request-handling.mdx

Lines changed: 59 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ A timeout is set in milliseconds. Additionaly, the timeout is based on when the
3030

3131
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request timeout specifies the timeout duration for requests and triggers a fallback.
3232

33-
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property within the provider-specific `config` object. Each provider can have a different `requestTimeout` value for granular customization.
33+
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property within the provider-specific `config` object. Each provider can have a different `requestTimeout` value for granular customization.
3434

3535
```bash title="Provider-level config" {12-14} collapse={15-48}
3636
curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
@@ -135,7 +135,64 @@ config:{
135135
}
136136
```
137137

138-
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), each provider can have a different retry settings for granular customization.
138+
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), each provider can have a different retry settings for granular customization.
139+
140+
```bash title="Provider-level config" {11-15} collapse={16-55}
141+
curl 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}' \
142+
--header 'Content-Type: application/json' \
143+
--data '[
144+
{
145+
"provider": "workers-ai",
146+
"endpoint": "@cf/meta/llama-3.1-8b-instruct",
147+
"headers": {
148+
"Authorization": "Bearer {cloudflare_token}",
149+
"Content-Type": "application/json"
150+
},
151+
"config": {
152+
"maxAttempts": 2,
153+
"retryDelay": 1000,
154+
"backoff": "constant"
155+
},
156+
"query": {
157+
"messages": [
158+
{
159+
"role": "system",
160+
"content": "You are a friendly assistant"
161+
},
162+
{
163+
"role": "user",
164+
"content": "What is Cloudflare?"
165+
}
166+
]
167+
}
168+
},
169+
{
170+
"provider": "workers-ai",
171+
"endpoint": "@cf/meta/llama-3.1-8b-instruct-fast",
172+
"headers": {
173+
"Authorization": "Bearer {cloudflare_token}",
174+
"Content-Type": "application/json"
175+
},
176+
"query": {
177+
"messages": [
178+
{
179+
"role": "system",
180+
"content": "You are a friendly assistant"
181+
},
182+
{
183+
"role": "user",
184+
"content": "What is Cloudflare?"
185+
}
186+
]
187+
},
188+
"config": {
189+
"maxAttempts": 4,
190+
"retryDelay": 1000,
191+
"backoff": "exponential"
192+
},
193+
}
194+
]'
195+
```
139196

140197
#### Direct provider
141198

0 commit comments

Comments
 (0)