rebase

gitName · gitName · commit 029703a90a58 · 2025-04-29T11:22:50.000-07:00
diff --git a/articles/api-management/azure-openai-semantic-cache-lookup-policy.md b/articles/api-management/azure-openai-semantic-cache-lookup-policy.md
@@ -9,7 +9,7 @@ ms.collection: ce-skilling-ai-copilot
 ms.custom:
   - build-2024
 ms.topic: reference
-ms.date: 12/13/2024
+ms.date: 04/29/2025
 ms.author: danlep
 ---
 
@@ -22,7 +22,7 @@ Use the `azure-openai-semantic-cache-lookup` policy to perform cache lookup of r
 > [!NOTE]
 > * This policy must have a corresponding [Cache responses to Azure OpenAI API requests](azure-openai-semantic-cache-store-policy.md) policy. 
 > * For prerequisites and steps to enable semantic caching, see [Enable semantic caching for Azure OpenAI APIs in Azure API Management](azure-openai-enable-semantic-caching.md).
-> * Currently, this policy is in preview.
+
 
 [!INCLUDE [api-management-policy-generic-alert](../../includes/api-management-policy-generic-alert.md)]
 
@@ -45,10 +45,10 @@ Use the `azure-openai-semantic-cache-lookup` policy to perform cache lookup of r
 
 | Attribute         | Description                                            | Required | Default |
 | ----------------- | ------------------------------------------------------ | -------- | ------- |
-| score-threshold	| Similarity score threshold used to determine whether to return a cached response to a prompt. Value is a decimal between 0.0 and 1.0. [Learn more](../redis/tutorial-semantic-cache.md#change-the-similarity-threshold). | Yes |	N/A |
+| score-threshold	| Similarity score threshold used to determine whether to return a cached response to a prompt. Value is a decimal between 0.0 and 1.0. Smaller values represent greater semantic similarity. [Learn more](../redis/tutorial-semantic-cache.md#change-the-similarity-threshold). | Yes |	N/A |
 | embeddings-backend-id | [Backend](backends.md) ID for OpenAI embeddings API call. |	Yes |	N/A |
 | embeddings-backend-auth | Authentication used for Azure OpenAI embeddings API backend. | Yes. Must be set to `system-assigned`. | N/A |
-| ignore-system-messages | Boolean. If set to `true`, removes system messages from a GPT chat completion prompt before assessing cache similarity. | No | false |
+| ignore-system-messages | Boolean. When set to `true` (recommended), removes system messages from a GPT chat completion prompt before assessing cache similarity. | No | false |
 | max-message-count | If specified, number of remaining dialog messages after which caching is skipped. | No | N/A |
                                              
 ## Elements
@@ -67,7 +67,8 @@ Use the `azure-openai-semantic-cache-lookup` policy to perform cache lookup of r
 ### Usage notes
 
 - This policy can only be used once in a policy section.
-
+- Fine-tune the value of `score-threshold` based on your application to ensure that the right sensitivity is used when determining which queries to cache. Start with a low value such as 0.05 and adjust to optimize the ratio of cache hits to misses.
+- The embeddings model should have enough capacity and sufficient context size to accommodate the prompt volume and prompts.
 
 ## Examples
 
diff --git a/articles/api-management/llm-emit-token-metric-policy.md b/articles/api-management/llm-emit-token-metric-policy.md
@@ -18,9 +18,6 @@ ms.custom:
 
 The `llm-emit-token-metric` policy sends custom metrics to Application Insights about consumption of large language model (LLM) tokens through LLM APIs. Token count metrics include: Total Tokens, Prompt Tokens, and Completion Tokens. 
 
-> [!NOTE]
-> Currently, this policy is in preview.
-
 [!INCLUDE [api-management-policy-generic-alert](../../includes/api-management-policy-generic-alert.md)]
 
 [!INCLUDE [api-management-llm-models](../../includes/api-management-llm-models.md)]
diff --git a/articles/api-management/llm-semantic-cache-lookup-policy.md b/articles/api-management/llm-semantic-cache-lookup-policy.md
@@ -9,7 +9,7 @@ ms.collection: ce-skilling-ai-copilot
 ms.custom:
   - build-2024
 ms.topic: reference
-ms.date: 12/13/2024
+ms.date: 04/29/2025
 ms.author: danlep
 ---
 
@@ -22,7 +22,6 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
 > [!NOTE]
 > * This policy must have a corresponding [Cache responses to large language model API requests](llm-semantic-cache-store-policy.md) policy. 
 > * For prerequisites and steps to enable semantic caching, see [Enable semantic caching for Azure OpenAI APIs in Azure API Management](azure-openai-enable-semantic-caching.md).
-> * Currently, this policy is in preview.
 
 [!INCLUDE [api-management-policy-generic-alert](../../includes/api-management-policy-generic-alert.md)]
 
@@ -45,10 +44,10 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
 
 | Attribute         | Description                                            | Required | Default |
 | ----------------- | ------------------------------------------------------ | -------- | ------- |
-| score-threshold	| Similarity score threshold used to determine whether to return a cached response to a prompt. Value is a decimal between 0.0 and 1.0. [Learn more](../redis/tutorial-semantic-cache.md#change-the-similarity-threshold). | Yes |	N/A |
+| score-threshold	| Similarity score threshold used to determine whether to return a cached response to a prompt. Value is a decimal between 0.0 and 1.0. Smaller values represent greater semantic similarity. [Learn more](../redis/tutorial-semantic-cache.md#change-the-similarity-threshold). | Yes |	N/A |
 | embeddings-backend-id | [Backend](backends.md) ID for OpenAI embeddings API call. |	Yes |	N/A |
 | embeddings-backend-auth | Authentication used for Azure OpenAI embeddings API backend. | Yes. Must be set to `system-assigned`. | N/A |
-| ignore-system-messages | Boolean. If set to `true`, removes system messages from a GPT chat completion prompt before assessing cache similarity. | No | false |
+| ignore-system-messages | Boolean. When set to `true` (recommended), removes system messages from a GPT chat completion prompt before assessing cache similarity. | No | false |
 | max-message-count | If specified, number of remaining dialog messages after which caching is skipped. | No | N/A |
                                              
 ## Elements
@@ -67,6 +66,8 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
 ### Usage notes
 
 - This policy can only be used once in a policy section.
+- Fine-tune the value of `score-threshold` based on your application to ensure that the right sensitivity is used when determining which queries to cache. Start with a low value such as 0.05 and adjust to optimize the ratio of cache hits to misses.
+- The embeddings model should have enough capacity and sufficient context size to accommodate the prompt volume and prompts.
 
 
 ## Examples
diff --git a/articles/api-management/llm-semantic-cache-store-policy.md b/articles/api-management/llm-semantic-cache-store-policy.md
@@ -21,7 +21,6 @@ The `llm-semantic-cache-store` policy caches responses to chat completion API re
 > [!NOTE]
 > * This policy must have a corresponding [Get cached responses to large language model API requests](llm-semantic-cache-lookup-policy.md) policy. 
 > * For prerequisites and steps to enable semantic caching, see [Enable semantic caching for Azure OpenAI APIs in Azure API Management](azure-openai-enable-semantic-caching.md). 
-> * Currently, this policy is in preview.
 
 [!INCLUDE [api-management-policy-generic-alert](../../includes/api-management-policy-generic-alert.md)]
 
diff --git a/articles/api-management/llm-token-limit-policy.md b/articles/api-management/llm-token-limit-policy.md
@@ -20,9 +20,6 @@ The `llm-token-limit` policy prevents large language model (LLM) API usage spike
 
 By relying on token usage metrics returned from the LLM endpoint, the policy can accurately monitor and enforce limits in real time. The policy also enables precalculation of prompt tokens by API Management, minimizing unnecessary requests to the LLM backend if the limit is already exceeded.
 
-> [!NOTE]
-> Currently, this policy is in preview.
-
 [!INCLUDE [api-management-policy-generic-alert](../../includes/api-management-policy-generic-alert.md)]
 
 [!INCLUDE [api-management-llm-models](../../includes/api-management-llm-models.md)]
diff --git a/includes/api-management-llm-models.md b/includes/api-management-llm-models.md
@@ -4,10 +4,10 @@ ms.service: azure-api-management
 ms.custom:
   - build-2024
 ms.topic: include
-ms.date: 07/09/2024
+ms.date: 04/29/2025
 ms.author: danlep
 ---
 
 ## Supported models
 
-Use the policy with LLM APIs added to Azure API Management that are available through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api).
+Use the policy with LLM APIs added to Azure API Management that are available through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api) or with OpenAI-compatible models served through third-party inference providers.
