Skip to content

Commit 878efba

Browse files
authored
Update caching.mdx
updated based on suggestions
1 parent ad50272 commit 878efba

File tree

1 file changed

+2
-8
lines changed

1 file changed

+2
-8
lines changed

src/content/docs/ai-gateway/configuration/caching.mdx

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,14 @@ description: Override caching settings on a per-request basis.
99

1010
import { TabItem, Tabs } from "~/components";
1111

12-
AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's edge network for identical requests. This can significantly improve response times and reduce costs.
12+
AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's cache for identical requests.
1313

1414
## Benefits of Using Caching
1515

1616
- **Reduced Latency:** Serve responses faster to your users by avoiding a round trip to the origin AI provider for repeated requests.
1717
- **Cost Savings:** Minimize the number of paid requests made to your AI provider, especially for frequently accessed or non-dynamic content.
1818
- **Increased Throughput:** Offload repetitive requests from your AI provider, allowing it to handle unique requests more efficiently.
1919

20-
2120
:::note
2221

2322
Currently caching is supported only for text and image responses, and it applies only to identical requests.
@@ -58,12 +57,7 @@ To check whether a response comes from cache or not, **cf-aig-cache-status** wil
5857

5958
## Per-request caching
6059

61-
While your gateway's default cache settings provide a good baseline, you might encounter scenarios where:
62-
63-
- **Freshness is critical:** Some API calls must always fetch the absolute latest data from the origin provider, irrespective of global caching rules.
64-
- **Content has varying lifespans:** A global Time To Live (TTL) might be too long for frequently updated information or too short for highly static content, leading to either stale data or reduced cache effectiveness.
65-
- **Responses are dynamic or personalized:** Caching user-specific or highly dynamic responses with a generic cache key could lead to incorrect data being served.
66-
- **Specific caching strategies are needed:** You might want to define exactly how a particular piece of content is cached, separate from other requests.
60+
While your gateway's default cache settings provide a good baseline, you might encounter scenarios requiring more granular control. For example, instances where when data freshness is needed, content has varying lifespans, or responses are dynamic or personalized.
6761

6862
To address these needs, AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers. This gives you the precision to optimize caching for individual API calls, ensuring the right balance of performance, cost-efficiency, and data accuracy.
6963

0 commit comments

Comments
 (0)