You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/providers/universal.mdx
+72Lines changed: 72 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,3 +70,75 @@ ws.on("message", function incoming(message) {
70
70
console.log(message.toString());
71
71
});
72
72
```
73
+
74
+
## Header configuration hierarchy
75
+
76
+
The Universal Endpoint allows you to set fallback models or providers and customize headers for each provider or request. You can configure headers at three levels:
77
+
78
+
1.**Provider level**: Headers specific to a particular provider.
79
+
2.**Request level**: Headers included in individual requests.
80
+
3.**Gateway settings**: Default headers configured in your gateway dashboard.
81
+
82
+
Since the same settings can be configured in multiple locations, AI Gateway applies a hierarchy to determine which configuration takes precedence:
83
+
84
+
-**Provider-level headers** override all other configurations.
85
+
-**Request-level headers** are used if no provider-level headers are set.
86
+
-**Gateway-level settings** are used only if no headers are configured at the provider or request levels.
87
+
88
+
This hierarchy ensures consistent behavior, prioritizing the most specific configurations. Use provider-level and request-level headers for fine-tuned control, and gateway settings for general defaults.
89
+
90
+
## Example
91
+
92
+
This example demonstrates how headers set at different levels impact caching behavior:
93
+
94
+
-**Request-level header**: The `cf-aig-cache-ttl` is set to `3600` seconds, applying this caching duration to the request by default.
95
+
-**Provider-level header**: For the fallback provider (OpenAI), `cf-aig-cache-ttl` is explicitly set to `0` seconds, overriding the request-level header and disabling caching for responses when OpenAI is used as the provider.
96
+
97
+
This shows how provider-level headers take precedence over request-level headers, allowing for granular control of caching behavior.
[cf-aig-event-id](/ai-gateway/evaluations/add-human-feedback-api/#3-retrieve-the-cf-aig-log-id)Unique identifier for an event, used to trace specific events through the system.
7
+
8
+
- term: cf-aig-step
9
+
general_definition: |-
10
+
Identifies the processing step in the AI Gateway flow for better tracking and debugging.
11
+
12
+
- term: cf-aig-log-id
13
+
general_definition: |-
14
+
Identifier for logging, enabling detailed tracking of requests.
15
+
16
+
- term: cf-aig-collect-log
17
+
general_definition: |-
18
+
Specifies if logging data should be collected for a particular request.
19
+
20
+
- term: cf-aig-custom-cost
21
+
general_definition: |-
22
+
Allows the customization of request cost to reflect user-defined parameters.
23
+
24
+
- term: cf-aig-cache-key
25
+
general_definition: |-
26
+
Custom key to determine cache storage and retrieval.
27
+
28
+
- term: cf-aig-cache-status
29
+
general_definition: |-
30
+
Status indicator for caching, showing if a request was served from cache.
31
+
32
+
- term: cf-aig-metadata
33
+
general_definition: |-
34
+
Additional metadata associated with a request.
35
+
36
+
- term: cf-aig-cache-ttl
37
+
general_definition: |-
38
+
Specifies the cache time-to-live for responses.
39
+
40
+
- term: cf-aig-skip-cache
41
+
general_definition: |-
42
+
Header to bypass caching for a specific request.
43
+
44
+
# Deprecated headers
45
+
- term: cf-cache-ttl
46
+
general_definition: |-
47
+
Deprecated: This header is replaced by `cf-aig-cache-ttl`. It specifies cache time-to-live.
48
+
49
+
- term: cf-skip-cache
50
+
general_definition: |-
51
+
Deprecated: This header is replaced by `cf-aig-skip-cache`. It bypasses caching for a specific request.
0 commit comments