From b11b8dbfc245528379388f4036d49be1995aa277 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Wed, 20 Nov 2024 15:23:51 -0800 Subject: [PATCH 1/2] Update caching.mdx trying to make sure it's clear cache is only for identical requests --- src/content/docs/ai-gateway/configuration/caching.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/configuration/caching.mdx b/src/content/docs/ai-gateway/configuration/caching.mdx index df41af27e4b723..0aa5fa3d7a8d90 100644 --- a/src/content/docs/ai-gateway/configuration/caching.mdx +++ b/src/content/docs/ai-gateway/configuration/caching.mdx @@ -13,7 +13,8 @@ Enable and customize your gateway cache to serve requests directly from Cloudfla :::note -Currently caching is supported only for text and image responses, and it applies only to identical requests. This is helpful for use cases when there are limited prompt options - for example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration. +Currently caching is supported only for text and image responses, and it applies only to identical requests. +This is helpful for use cases when there are limited prompt options - for example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration. We plan on adding semantic search for caching in the future to improve cache hit rates. ::: From 7542315f4adb3b4f44cb704bed6b474230937802 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Fri, 22 Nov 2024 14:14:04 -0800 Subject: [PATCH 2/2] Update caching.mdx --- src/content/docs/ai-gateway/configuration/caching.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/src/content/docs/ai-gateway/configuration/caching.mdx b/src/content/docs/ai-gateway/configuration/caching.mdx index 0aa5fa3d7a8d90..6d7a6851228400 100644 --- a/src/content/docs/ai-gateway/configuration/caching.mdx +++ b/src/content/docs/ai-gateway/configuration/caching.mdx @@ -14,6 +14,7 @@ Enable and customize your gateway cache to serve requests directly from Cloudfla :::note Currently caching is supported only for text and image responses, and it applies only to identical requests. + This is helpful for use cases when there are limited prompt options - for example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration. We plan on adding semantic search for caching in the future to improve cache hit rates. :::