You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|[Enforce content safety checks on LLM requests](llm-content-safety-policy.md)| Enforces content safety checks on LLM requests (prompts) by transmitting them to the [Azure AI Content Safety](/azure/ai-services/content-safety/overview) service before sending to the backend LLM. | Yes | Yes | Yes | Yes | Yes |
62
63
|[Validate content](validate-content-policy.md)| Validates the size or content of a request or response body against one or more API schemas. The supported schema formats are JSON and XML. | Yes | Yes | Yes | Yes | Yes |
63
64
|[Validate GraphQL request](validate-graphql-request-policy.md)| Validates and authorizes a request to a GraphQL API. | Yes | Yes | Yes | Yes | No |
64
65
|[Validate OData request](validate-odata-request-policy.md)| Validates a request to an OData API to ensure conformance with the OData specification. | Yes | Yes | Yes | Yes | Yes |
The `llm-content-safety` policy enforces content safety checks by leveraging Azure Content Safety. When enabled and the policy detects malicious content, API management returns a `403` error code. Use the policy in scenarios such as the following:
19
+
The `llm-content-safety` policy enforces content safety checks on LLM requests (prompts) by transmitting them to the [Azure AI Content Safety](/azure/ai-services/content-safety/overview) service before sending to the backend LLM. When enabled and AI Content Safety detects malicious content, API Management blocks the request and returns a `403` error code.
20
20
21
-
* Block requests that contain harmful content or hate speech
22
-
* Apply a blocklist to prevent specific content from being sent
21
+
Use the policy in scenarios such as the following:
22
+
23
+
* Block requests that contain predefined categories of harmful content or hate speech
24
+
* Apply custom blocklists to prevent specific content from being sent
23
25
* Shield against prompts that match attack patterns
24
26
25
27
> [!NOTE]
@@ -30,9 +32,7 @@ The `llm-content-safety` policy enforces content safety checks by leveraging Azu
30
32
## Prerequisites
31
33
32
34
* An [Azure AI Content Safety](/azure/ai-services/content-safety/) resource.
33
-
* API Management [backend](backends.md) configured to route content-safety API calls to the Azure Content Safety service.
34
-
35
-
35
+
* An API Management [backend](backends.md) configured to route content safety API calls and authenticate to the Azure AI Content Safety service.
36
36
37
37
38
38
## Policy statement
@@ -56,24 +56,24 @@ The `llm-content-safety` policy enforces content safety checks by leveraging Azu
| backend-id | Identifier (name) of the Azure Content Safety backend to route content-safety API calls to. Policy expressions are allowed. | Yes | N/A |
59
+
| backend-id | Identifier (name) of the Azure AI Content Safety backend to route content-safety API calls to. Policy expressions are allowed. | Yes | N/A |
60
60
| shield-prompt | If set to `true`, content is checked for user attacks. Otherwise, skip this check. Policy expressions are allowed. | No |`false`|
61
61
62
62
63
63
## Elements
64
64
65
65
| Element | Description | Required |
66
66
| -------------- | -----| -------- |
67
-
| categories | A list of `category` elements that specify settings for blocking messages when the category is detected. | No |
68
-
| blocklists | A list of blocklist `id` elements for which detection will cause the message to be blocked. Policy expressions are allowed. | No |
67
+
| categories | A list of `category` elements that specify settings for blocking requests when the category is detected. | No |
68
+
| blocklists | A list of [blocklist](/azure/ai-services/content-safety/how-to/use-blocklist)`id` elements from the Azure AI Content Safety instance for which detection causes the request to be blocked. Policy expressions are allowed. | No |
| name | Specifies the name of this category. The attribute must have one of the following values: `Hate`, `SelfHarm`, `Sexual`, `Violence`. Policy expressions are allowed. | Yes | N/A |
85
-
| threshold | Specifies the threshold value for this category at which messages are blocked. Messages with content severities less than the threshold are not blocked. The value must be between 0 and 7. Policy expressions are allowed. | Yes | N/A |
85
+
| threshold | Specifies the threshold value for this category at which request are blocked. Requests with content severities less than the threshold are not blocked. The value must be between 0 and 7. Policy expressions are allowed. | Yes | N/A |
86
86
87
87
88
88
## Usage
@@ -93,15 +93,32 @@ Policy expressions are allowed. | No | `FourSeverityLevels` |
93
93
94
94
### Usage notes
95
95
96
+
* The policy runs on a concatenation of all text content in the request.
97
+
* If the request exceeds the character limit of Azure AI Content Safety, or an internal error occurs, a `500` error is returned.
96
98
* This policy can be used multiple times per policy definition.
97
99
98
-
## Examples
100
+
## Example
101
+
102
+
The following example enforces content safety checks on LLM requests using the Azure AI Content Safety service. The policy blocks requests that contain hate speech or violence with a severity level of 4 or higher. The `shield-prompt` attribute is set to `true` to check for adversarial attacks.
0 commit comments