Skip to content

Commit 20d9def

Browse files
authored
Update llm-semantic-cache-lookup-policy.md
added usages notes for score threshold values and vary-by for caching sensitive data.
1 parent 94af00b commit 20d9def

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

articles/api-management/llm-semantic-cache-lookup-policy.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,9 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
6868
- This policy can only be used once in a policy section.
6969
- Fine-tune the value of `score-threshold` based on your application to ensure that the right sensitivity is used when determining which queries to cache. Start with a low value such as 0.05 and adjust to optimize the ratio of cache hits to misses.
7070
- The embeddings model should have enough capacity and sufficient context size to accommodate the prompt volume and prompts.
71+
- Score threshold above 0.2 may lead to cache mismatch. Consider using lower value for sensitive use cases.
72+
- Control cross-user access to cache entries by specifying `vary-by`with specific user or user-group identifiers.
73+
- Consider adding [llm-content-safety](./llm-content-safety-policy.md) policy with prompt shield to protect from prompt attacks.
7174

7275

7376
## Examples

0 commit comments

Comments
 (0)