From 53405c2f2a2d17f16bfe9efd9dd77c7a99057a03 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Tue, 4 Feb 2025 17:13:37 -0800 Subject: [PATCH 1/4] Create 2025-02-05-aig-request-handling.mdx added copy --- .../2025-02-05-aig-request-handling.mdx | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 src/content/changelogs-next/2025-02-05-aig-request-handling.mdx diff --git a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx new file mode 100644 index 000000000000000..648063b31f78ce2 --- /dev/null +++ b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx @@ -0,0 +1,31 @@ +--- +title: Request timeouts and Request retries with AI Gateway +description: AI Gateway has added additional request handing, with request timeouts and request retries +products: + - ai-gateway +date: 2025-02-05T11:00:00Z +--- + +AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries). This helps allowing you to manage AI interactions effectively and ensure your applications remain responsive and reliable. + +These can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/). + +**Request timeouts** +- A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond. + +To set a request timeout directly to a provider, add a `cf-aig-request-timeout` header. + +```bash title="Provider-specific endpoint example" {4} +curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/workers-ai/@cf/meta/llama-3.1-8b-instruct \ + --header 'Authorization: Bearer {cf_api_token}' \ + --header 'Content-Type: application/json' \ + --header 'cf-aig-request-timeout: 5000' + --data '{"prompt": "What is Cloudflare?"}' +``` +**Request retries** +- A [request retry](/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without manual intervention. + +To set up request retries directly to a provider, add the following headers: +- cf-aig-max-attempts (number) +- cf-aig-retry-delay (number) +- cf-aig-backoff ("constant" | "linear" | "exponential) From a42139462565cf709ce930510c695e0d4a25ed37 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Wed, 5 Feb 2025 11:17:55 -0800 Subject: [PATCH 2/4] Update 2025-02-05-aig-request-handling.mdx wording updated --- .../changelogs-next/2025-02-05-aig-request-handling.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx index 648063b31f78ce2..b6add1f39e8dfff 100644 --- a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx +++ b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx @@ -1,14 +1,14 @@ --- -title: Request timeouts and Request retries with AI Gateway +title: Request timeouts and retries with AI Gateway description: AI Gateway has added additional request handing, with request timeouts and request retries products: - ai-gateway date: 2025-02-05T11:00:00Z --- -AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries). This helps allowing you to manage AI interactions effectively and ensure your applications remain responsive and reliable. +AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries), making it easier to keep your applications responsive and reliable. -These can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/). +Timeouts and retries can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/). **Request timeouts** - A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond. From cbf988e692e85a7582fd76bd163e7b5d098457d1 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Thu, 6 Feb 2025 10:22:23 -0800 Subject: [PATCH 3/4] Update 2025-02-05-aig-request-handling.mdx --- .../changelogs-next/2025-02-05-aig-request-handling.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx index b6add1f39e8dfff..e920b54ebcb1377 100644 --- a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx +++ b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx @@ -3,7 +3,7 @@ title: Request timeouts and retries with AI Gateway description: AI Gateway has added additional request handing, with request timeouts and request retries products: - ai-gateway -date: 2025-02-05T11:00:00Z +date: 2025-02-06T11:00:00Z --- AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gateway/configuration/request-handling/#request-timeouts) and [Request Retries](/ai-gateway/configuration/request-handling/#request-retries), making it easier to keep your applications responsive and reliable. @@ -11,7 +11,7 @@ AI Gateway adds additional ways to handle requests - [Request Timeouts](/ai-gate Timeouts and retries can be used on both the [Universal Endpoint](/ai-gateway/providers/universal) or directly to a [supported provider](/ai-gateway/providers/). **Request timeouts** -- A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond. +A [request timeout](/ai-gateway/configuration/request-handling/#request-timeouts) allows you to trigger [fallbacks](/ai-gateway/configuration/fallbacks/) or a retry if a provider takes too long to respond. To set a request timeout directly to a provider, add a `cf-aig-request-timeout` header. @@ -23,7 +23,7 @@ curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/workers-ai/@ --data '{"prompt": "What is Cloudflare?"}' ``` **Request retries** -- A [request retry](/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without manual intervention. +A [request retry](/ai-gateway/configuration/request-handling/#request-retries) automatically retries failed requests, so you can recover from temporary issues without intervening. To set up request retries directly to a provider, add the following headers: - cf-aig-max-attempts (number) From 8dff0ddf2b8f0ac553a9c0c5026247ae4593f3b7 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Thu, 6 Feb 2025 13:33:09 -0800 Subject: [PATCH 4/4] Update 2025-02-05-aig-request-handling.mdx typo --- src/content/changelogs-next/2025-02-05-aig-request-handling.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx index e920b54ebcb1377..06f64192fe4cf65 100644 --- a/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx +++ b/src/content/changelogs-next/2025-02-05-aig-request-handling.mdx @@ -1,6 +1,6 @@ --- title: Request timeouts and retries with AI Gateway -description: AI Gateway has added additional request handing, with request timeouts and request retries +description: AI Gateway has added additional request handling, with request timeouts and request retries products: - ai-gateway date: 2025-02-06T11:00:00Z