Skip to content

Commit 01c34b5

Browse files
added header hierachy docs
1 parent f741940 commit 01c34b5

File tree

2 files changed

+123
-0
lines changed

2 files changed

+123
-0
lines changed

src/content/docs/ai-gateway/providers/universal.mdx

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,3 +70,75 @@ ws.on("message", function incoming(message) {
7070
console.log(message.toString());
7171
});
7272
```
73+
74+
## Header configuration hierarchy
75+
76+
The Universal Endpoint allows you to set fallback models or providers and customize headers for each provider or request. You can configure headers at three levels:
77+
78+
1. **Provider level**: Headers specific to a particular provider.
79+
2. **Request level**: Headers included in individual requests.
80+
3. **Gateway settings**: Default headers configured in your gateway dashboard.
81+
82+
Since the same settings can be configured in multiple locations, AI Gateway applies a hierarchy to determine which configuration takes precedence:
83+
84+
- **Provider-level headers** override all other configurations.
85+
- **Request-level headers** are used if no provider-level headers are set.
86+
- **Gateway-level settings** are used only if no headers are configured at the provider or request levels.
87+
88+
This hierarchy ensures consistent behavior, prioritizing the most specific configurations. Use provider-level and request-level headers for fine-tuned control, and gateway settings for general defaults.
89+
90+
## Example
91+
92+
This example demonstrates how headers set at different levels impact caching behavior:
93+
94+
- **Request-level header**: The `cf-aig-cache-ttl` is set to `3600` seconds, applying this caching duration to the request by default.
95+
- **Provider-level header**: For the fallback provider (OpenAI), `cf-aig-cache-ttl` is explicitly set to `0` seconds, overriding the request-level header and disabling caching for responses when OpenAI is used as the provider.
96+
97+
This shows how provider-level headers take precedence over request-level headers, allowing for granular control of caching behavior.
98+
99+
```bash
100+
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id} \
101+
--header 'Content-Type: application/json' \
102+
--header 'cf-aig-cache-ttl: 3600' \
103+
--data '[
104+
{
105+
"provider": "workers-ai",
106+
"endpoint": "@cf/meta/llama-3.1-8b-instruct",
107+
"headers": {
108+
"Authorization": "Bearer {cloudflare_token}",
109+
"Content-Type": "application/json"
110+
},
111+
"query": {
112+
"messages": [
113+
{
114+
"role": "system",
115+
"content": "You are a friendly assistant"
116+
},
117+
{
118+
"role": "user",
119+
"content": "What is Cloudflare?"
120+
}
121+
]
122+
}
123+
},
124+
{
125+
"provider": "openai",
126+
"endpoint": "chat/completions",
127+
"headers": {
128+
"Authorization": "Bearer {open_ai_token}",
129+
"Content-Type": "application/json",
130+
"cf-aig-cache-ttl": "0"
131+
},
132+
"query": {
133+
"model": "gpt-4o-mini",
134+
"stream": true,
135+
"messages": [
136+
{
137+
"role": "user",
138+
"content": "What is Cloudflare?"
139+
}
140+
]
141+
}
142+
}
143+
]'
144+
```
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
productName: AI Gateway
3+
entries:
4+
- term: cf-aig-event-id
5+
general_definition: |-
6+
[cf-aig-event-id](/ai-gateway/evaluations/add-human-feedback-api/#3-retrieve-the-cf-aig-log-id)Unique identifier for an event, used to trace specific events through the system.
7+
8+
- term: cf-aig-step
9+
general_definition: |-
10+
Identifies the processing step in the AI Gateway flow for better tracking and debugging.
11+
12+
- term: cf-aig-log-id
13+
general_definition: |-
14+
Identifier for logging, enabling detailed tracking of requests.
15+
16+
- term: cf-aig-collect-log
17+
general_definition: |-
18+
Specifies if logging data should be collected for a particular request.
19+
20+
- term: cf-aig-custom-cost
21+
general_definition: |-
22+
Allows the customization of request cost to reflect user-defined parameters.
23+
24+
- term: cf-aig-cache-key
25+
general_definition: |-
26+
Custom key to determine cache storage and retrieval.
27+
28+
- term: cf-aig-cache-status
29+
general_definition: |-
30+
Status indicator for caching, showing if a request was served from cache.
31+
32+
- term: cf-aig-metadata
33+
general_definition: |-
34+
Additional metadata associated with a request.
35+
36+
- term: cf-aig-cache-ttl
37+
general_definition: |-
38+
Specifies the cache time-to-live for responses.
39+
40+
- term: cf-aig-skip-cache
41+
general_definition: |-
42+
Header to bypass caching for a specific request.
43+
44+
# Deprecated headers
45+
- term: cf-cache-ttl
46+
general_definition: |-
47+
Deprecated: This header is replaced by `cf-aig-cache-ttl`. It specifies cache time-to-live.
48+
49+
- term: cf-skip-cache
50+
general_definition: |-
51+
Deprecated: This header is replaced by `cf-aig-skip-cache`. It bypasses caching for a specific request.

0 commit comments

Comments
 (0)