Skip to content

Commit f001400

Browse files
kathaylkodster28
andauthored
Update caching.mdx (#22253)
* Update caching.mdx update caching documentation. trial using new gemini access * Update src/content/docs/ai-gateway/configuration/caching.mdx Co-authored-by: Kody Jackson <[email protected]> * Update caching.mdx updated based on suggestions * Update src/content/docs/ai-gateway/configuration/caching.mdx Co-authored-by: Kody Jackson <[email protected]> * Update src/content/docs/ai-gateway/configuration/caching.mdx Co-authored-by: Kody Jackson <[email protected]> --------- Co-authored-by: Kody Jackson <[email protected]>
1 parent 65dea83 commit f001400

File tree

1 file changed

+12
-2
lines changed

1 file changed

+12
-2
lines changed

src/content/docs/ai-gateway/configuration/caching.mdx

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,13 @@ description: Override caching settings on a per-request basis.
99

1010
import { TabItem, Tabs } from "~/components";
1111

12-
Enable and customize your gateway cache to serve requests directly from Cloudflare's cache, instead of the original model provider, for faster requests and cost savings.
12+
AI Gateway can cache responses from your AI model providers, serving them directly from Cloudflare's cache for identical requests.
13+
14+
## Benefits of Using Caching
15+
16+
- **Reduced Latency:** Serve responses faster to your users by avoiding a round trip to the origin AI provider for repeated requests.
17+
- **Cost Savings:** Minimize the number of paid requests made to your AI provider, especially for frequently accessed or non-dynamic content.
18+
- **Increased Throughput:** Offload repetitive requests from your AI provider, allowing it to handle unique requests more efficiently.
1319

1420
:::note
1521

@@ -51,7 +57,11 @@ To check whether a response comes from cache or not, **cf-aig-cache-status** wil
5157

5258
## Per-request caching
5359

54-
In order to override the default cache behavior defined on the settings tab, you can, on a per-request basis, set headers for the following options:
60+
While your gateway's default cache settings provide a good baseline, you might need more granular control. These situations could be data freshness, content with varying lifespans, or dynamic or personalized responses.
61+
62+
To address these needs, AI Gateway allows you to override default cache behaviors on a per-request basis using specific HTTP headers. This gives you the precision to optimize caching for individual API calls.
63+
64+
The following headers allow you to define this per-request cache behavior:
5565

5666
:::note
5767

0 commit comments

Comments
 (0)