Skip to content

Commit 1a12194

Browse files
kathaylkodster28
andauthored
Create 2025-02-05-aig-request-handling.mdx (#19748)
* Create 2025-02-05-aig-request-handling.mdx added copy * Update 2025-02-05-aig-request-handling.mdx wording updated * Update 2025-02-05-aig-request-handling.mdx * Update 2025-02-05-aig-request-handling.mdx typo --------- Co-authored-by: kodster28 <[email protected]>
1 parent c332b45 commit 1a12194

File tree

1 file changed

+31
-0
lines changed

1 file changed

+31
-0
lines changed
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
title: Request timeouts and retries with AI Gateway
3+
description: AI Gateway has added additional request handling, with request timeouts and request retries
4+
products:
5+
- ai-gateway
6+
date: 2025-02-06T11:00:00Z
7+
---
8+
9+
AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries), making it easier to keep your applications responsive and reliable.
10+
11+
Timeouts and retries can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/).
12+
13+
**Request timeouts**
14+
A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond.
15+
16+
To set a request timeout directly to a provider, add a `cf-aig-request-timeout` header.
17+
18+
```bash title="Provider-specific endpoint example" {4}
19+
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/workers-ai/@cf/meta/llama-3.1-8b-instruct \
20+
--header 'Authorization: Bearer {cf_api_token}' \
21+
--header 'Content-Type: application/json' \
22+
--header 'cf-aig-request-timeout: 5000'
23+
--data '{"prompt": "What is Cloudflare?"}'
24+
```
25+
**Request retries**
26+
A [request retry](/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without intervening.
27+
28+
To set up request retries directly to a provider, add the following headers:
29+
- cf-aig-max-attempts (number)
30+
- cf-aig-retry-delay (number)
31+
- cf-aig-backoff ("constant" | "linear" | "exponential)

0 commit comments

Comments
 (0)