Skip to content

Commit 4802332

Browse files
Merge pull request #301057 from akamenev/patch-2
Update llm-semantic-cache-lookup-policy.md
2 parents 31b982a + 20d9def commit 4802332

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

articles/api-management/llm-semantic-cache-lookup-policy.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,9 @@ Use the `llm-semantic-cache-lookup` policy to perform cache lookup of responses
6868
- This policy can only be used once in a policy section.
6969
- Fine-tune the value of `score-threshold` based on your application to ensure that the right sensitivity is used when determining which queries to cache. Start with a low value such as 0.05 and adjust to optimize the ratio of cache hits to misses.
7070
- The embeddings model should have enough capacity and sufficient context size to accommodate the prompt volume and prompts.
71+
- Score threshold above 0.2 may lead to cache mismatch. Consider using lower value for sensitive use cases.
72+
- Control cross-user access to cache entries by specifying `vary-by`with specific user or user-group identifiers.
73+
- Consider adding [llm-content-safety](./llm-content-safety-policy.md) policy with prompt shield to protect from prompt attacks.
7174

7275

7376
## Examples

0 commit comments

Comments
 (0)