You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/api-management/llm-semantic-cache-lookup-policy.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -68,6 +68,9 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
68
68
- This policy can only be used once in a policy section.
69
69
- Fine-tune the value of `score-threshold` based on your application to ensure that the right sensitivity is used when determining which queries to cache. Start with a low value such as 0.05 and adjust to optimize the ratio of cache hits to misses.
70
70
- The embeddings model should have enough capacity and sufficient context size to accommodate the prompt volume and prompts.
71
+
- Score threshold above 0.2 may lead to cache mismatch. Consider using lower value for sensitive use cases.
72
+
- Control cross-user access to cache entries by specifying `vary-by`with specific user or user-group identifiers.
73
+
- Consider adding [llm-content-safety](./llm-content-safety-policy.md) policy with prompt shield to protect from prompt attacks.
0 commit comments