You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/configuration/request-handling.mdx
+6-11Lines changed: 6 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@ A timeout is set in milliseconds. Additionaly, the timeout is based on when the
30
30
31
31
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request timeout specifies the timeout duration for requests and triggers a fallback.
32
32
33
-
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property either as a universal attribute or within the provider-specific `config` object.
33
+
For a Universal Endpoint, configure the timeout value by setting a `requestTimeout` property within the provider-specific `config` object. Each provider can have a different `requestTimeout` value for granular customization.
To further customize request handling, you can include unique `RequestTimeout` values for each provider and a default across your Universal Endpoint.
90
-
91
-
In this case, the most specific value takes precedence. A timeout value on a specific provider would take precedence over the one on the endpoint itself.
92
-
93
88
#### Direct provider
94
89
95
90
If set on a [provider](/ai-gateway/providers/) request, request timeout specifies the timeout duration for a request and - if exceeded - returns an error.
@@ -130,7 +125,7 @@ On the final retry attempt, your gateway will wait until the request completes,
130
125
131
126
If set on a [Universal Endpoint](/ai-gateway/providers/universal/), a request retry will automatically retry failed requests up to five times before triggering any configured fallbacks.
132
127
133
-
For a Universal Endpoint, configure the timeout value by setting the following properties in the overall or provider-specific `config`:
128
+
For a Universal Endpoint, configure the retry settings with the following properties in the provider-specific `config`:
134
129
135
130
```json
136
131
config:{
@@ -140,11 +135,11 @@ config:{
140
135
}
141
136
```
142
137
143
-
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), the values can interact with each other to provide more customized logic.
138
+
As with the [request timeout](/ai-gateway/configuration/request-handling/#universal-endpoint), each provider can have a different retry settings for granular customization.
144
139
145
140
#### Direct provider
146
141
147
-
If set on a [provider](/ai-gateway/providers/) request, request timeout specifies the timeout duration for a request and - if exceeded - returns an error.
142
+
If set on a [provider](/ai-gateway/providers/) request, a request retry will automatically retry failed requests up to five times. On the final retry attempt, your gateway will wait until the request completes, regardless of how long it takes.
148
143
149
144
For a provider-specific endpoint, configure the timeout value by adding different header values:
0 commit comments