You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/ai-gateway/configuration/caching.mdx
+2-8Lines changed: 2 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,15 +9,14 @@ description: Override caching settings on a per-request basis.
9
9
10
10
import { TabItem, Tabs } from"~/components";
11
11
12
-
AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's edge network for identical requests. This can significantly improve response times and reduce costs.
12
+
AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's cache for identical requests.
13
13
14
14
## Benefits of Using Caching
15
15
16
16
-**Reduced Latency:** Serve responses faster to your users by avoiding a round trip to the origin AI provider for repeated requests.
17
17
-**Cost Savings:** Minimize the number of paid requests made to your AI provider, especially for frequently accessed or non-dynamic content.
18
18
-**Increased Throughput:** Offload repetitive requests from your AI provider, allowing it to handle unique requests more efficiently.
19
19
20
-
21
20
:::note
22
21
23
22
Currently caching is supported only for text and image responses, and it applies only to identical requests.
@@ -58,12 +57,7 @@ To check whether a response comes from cache or not, **cf-aig-cache-status** wil
58
57
59
58
## Per-request caching
60
59
61
-
While your gateway's default cache settings provide a good baseline, you might encounter scenarios where:
62
-
63
-
-**Freshness is critical:** Some API calls must always fetch the absolute latest data from the origin provider, irrespective of global caching rules.
64
-
-**Content has varying lifespans:** A global Time To Live (TTL) might be too long for frequently updated information or too short for highly static content, leading to either stale data or reduced cache effectiveness.
65
-
-**Responses are dynamic or personalized:** Caching user-specific or highly dynamic responses with a generic cache key could lead to incorrect data being served.
66
-
-**Specific caching strategies are needed:** You might want to define exactly how a particular piece of content is cached, separate from other requests.
60
+
While your gateway's default cache settings provide a good baseline, you might encounter scenarios requiring more granular control. For example, instances where when data freshness is needed, content has varying lifespans, or responses are dynamic or personalized.
67
61
68
62
To address these needs, AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers. This gives you the precision to optimize caching for individual API calls, ensuring the right balance of performance, cost-efficiency, and data accuracy.
0 commit comments