You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/concepts/content-filtering.md
+51-22Lines changed: 51 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ manager: nitinme
6
6
ms.service: azure-ai-studio
7
7
ms.custom:
8
8
- ignite-2023
9
-
ms.topic: conceptual
9
+
ms.topic: how-to
10
10
ms.date: 5/21/2024
11
11
ms.reviewer: eur
12
12
ms.author: pafarley
@@ -29,17 +29,60 @@ The content filtering models have been trained and tested on the following langu
29
29
You can create a content filter or use the default content filter for Azure OpenAI model deployment, and can also use a default content filter for other text models curated by Azure AI in the [model catalog](../how-to/model-catalog-overview.md). The custom content filters for those models aren't yet available. Models available through Models as a Service have content filtering enabled by default and can't be configured.
30
30
31
31
## How to create a content filter?
32
-
For any model deployment in [Azure AI Studio](https://ai.azure.com), you could directly use the default content filter, but when you want to have more customized setting on content filter, for example set a stricter or looser filter, or enable more advanced capabilities, like jailbreak risk detection and protected material detection. To create a content filter, you could go to **Build**, choose one of your projects, then select **Content filters** in the left navigation bar, and create a content filter.
32
+
For any model deployment in [Azure AI Studio](https://ai.azure.com), you could directly use the default content filter, but when you want to have more customized setting on content filter, for example set a stricter or looser filter, or enable more advanced capabilities, like jailbreak risk detection and protected material detection.
33
33
34
-
:::image type="content" source="../media/content-safety/content-filter/create-content-filter.png" alt-text="Screenshot of create content filter." lightbox="../media/content-safety/content-filter/create-content-filter.png":::
34
+
Follow these steps to create a content filter:
35
35
36
-
### Content filtering categories and configurability
36
+
1. Go to [AI Studio](https://ai.azure.com) and select a project.
37
+
1. Select **Content filters** from the left pane and then select **+ New content filter**.
37
38
38
-
The content filtering system integrated in Azure AI Studio contains neural multi-class classification models aimed at detecting and filtering harmful content; the models cover four categories (hate, sexual, violence, and self-harm) across four severity levels (safe, low, medium, and high). Content detected at the 'safe' severity level is labeled in annotations but isn't subject to filtering and isn't configurable.
39
+
:::image type="content" source="../media/content-safety/content-filter/create-content-filter.png" alt-text="Screenshot of the button to create a new content filter." lightbox="../media/content-safety/content-filter/create-content-filter.png":::
40
+
41
+
1. On the **Basic information** page, enter a name for your content filter. Select a connection to associate with the content filter. Then select **Next**.
42
+
43
+
:::image type="content" source="../media/content-safety/content-filter/create-content-filter-basic.png" alt-text="Screenshot of the option to select or enter basic information such as the filter name when creating a content filter." lightbox="../media/content-safety/content-filter/create-content-filter-basic.png":::
44
+
45
+
1. On the **Input filters** page, you can set the filter for the input prompt. For example, you can enable prompt shields for jailbreak attacks. Then select **Next**.
46
+
47
+
:::image type="content" source="../media/content-safety/content-filter/configure-threshold.png" alt-text="Screenshot of the option to select input filters when creating a content filter." lightbox="../media/content-safety/content-filter/configure-threshold.png":::
48
+
49
+
Content will be annotated by category and blocked according to the threshold you set. For the violence, hate, sexual, and self-harm categories, adjust the slider to block content of high, medium, or low severity.
50
+
51
+
1. On the **Output filters** page, you can set the filter for the output completion. For example, you can enable filters for protected material detection. Then select **Next**.
52
+
53
+
Content will be annotated by each categories and blocked according to the threshold. For violent content, hate content, sexual content, and self-harm content category, adjust the threshold to block harmful content with equal or higher severity levels.
54
+
55
+
1. Optionally, on the **Deployment** page, you can associate the content filter with a deployment. You can also associate the content filter with a deployment later. Then select **Create**.
56
+
57
+
:::image type="content" source="../media/content-safety/content-filter/create-content-filter-deployment.png" alt-text="Screenshot of the option to select a deployment when creating a content filter." lightbox="../media/content-safety/content-filter/create-content-filter-deployment.png":::
58
+
59
+
Content filtering configurations are created at the hub level in AI Studio. Learn more about configurability in the [Azure OpenAI docs](/azure/ai-services/openai/how-to/content-filters).
60
+
61
+
1. On the **Review** page, review the settings and then select **Create filter**.
62
+
63
+
64
+
## How to apply a content filter?
65
+
66
+
A default content filter is set when you create a deployment. You can also apply your custom content filter to your deployment.
67
+
68
+
Follow these steps to apply a content filter to a deployment:
39
69
40
-
:::image type="content" source="../media/content-safety/content-filter/configure-threshold.png" alt-text="Screenshot of configuring the threshold." lightbox="../media/content-safety/content-filter/configure-threshold.png":::
70
+
1. Go to [AI Studio](https://ai.azure.com) and select a project.
71
+
1. Select **Deployments** and choose one of your deployments, then select **Edit**.
41
72
42
-
#### Categories
73
+
:::image type="content" source="../media/content-safety/content-filter/deployment-edit.png" alt-text="Screenshot of the button to edit a deployment." lightbox="../media/content-safety/content-filter/deployment-edit.png":::
74
+
75
+
1. In the **Update deployment** window, select the content filter you want to apply to the deployment.
76
+
77
+
:::image type="content" source="../media/content-safety/content-filter/apply-content-filter.png" alt-text="Screenshot of apply content filter." lightbox="../media/content-safety/content-filter/apply-content-filter.png":::
78
+
79
+
Now, you can go to the playground to test whether the content filter works as expected!
80
+
81
+
## Content filtering categories and configurability
82
+
83
+
The content filtering system integrated in Azure AI Studio contains neural multi-class classification models aimed at detecting and filtering harmful content; the models cover four categories (hate, sexual, violence, and self-harm) across four severity levels (safe, low, medium, and high). Content detected at the 'safe' severity level is labeled in annotations but isn't subject to filtering and isn't configurable.
84
+
85
+
### Categories
43
86
44
87
|Category|Description|
45
88
|--------|-----------|
@@ -48,7 +91,7 @@ The content filtering system integrated in Azure AI Studio contains neural multi
48
91
| Violence | The violence category describes language related to physical actions intended to hurt, injure, damage, or kill someone or something; describes weapons, etc. |
49
92
| Self-Harm | The self-harm category describes language related to physical actions intended to purposely hurt, injure, or damage one's body, or kill oneself.|
50
93
51
-
####Severity levels
94
+
### Severity levels
52
95
53
96
|Category|Description|
54
97
|--------|-----------|
@@ -70,22 +113,8 @@ The default content filtering configuration for the GPT model series is set to f
70
113
71
114
<sup>1</sup> For Azure OpenAI models, only customers who have been approved for modified content filtering have full content filtering control, including configuring content filters at severity level high only or turning off content filters. Apply for modified content filters via this form: [Azure OpenAI Limited Access Review: Modified Content Filters and Abuse Monitoring (microsoft.com)](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xURE01NDY1OUhBRzQ3MkQxMUhZSE1ZUlJKTiQlQCN0PWcu)
72
115
73
-
Content filtering configurations are created within a Resource in Azure AI Studio and can be associated with Deployments. Learn more about configurability in the [Azure OpenAI docs](/azure/ai-services/openai/how-to/content-filters).
74
-
75
116
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
76
117
77
-
### More filters for generative AI scenarios
78
-
You could also enable filters for generative AI scenarios: jailbreak risk detection and protected material detection.
79
-
80
-
:::image type="content" source="../media/content-safety/content-filter/additional-models.png" alt-text="Screenshot of additional models." lightbox="../media/content-safety/content-filter/additional-models.png":::
81
-
82
-
## How to apply a content filter?
83
-
84
-
A default content filter is set when you create a deployment. You can also apply your custom content filter to your deployment. Select **Deployments** and choose one of your deployments, then select **Edit**, a window of updating deployment will open up. Then you can update the deployment by selecting one of your created content filters.
85
-
86
-
:::image type="content" source="../media/content-safety/content-filter/apply-content-filter.png" alt-text="Screenshot of apply content filter." lightbox="../media/content-safety/content-filter/apply-content-filter.png":::
87
-
88
-
Now, you can go to the playground to test whether the content filter works as expected!
0 commit comments