Skip to content

Commit 00fd4dc

Browse files
authored
Merge pull request #1861 from MicrosoftDocs/main
Publish 12/6 11:00 AM IST
2 parents d62332b + b875585 commit 00fd4dc

File tree

9 files changed

+91
-110
lines changed

9 files changed

+91
-110
lines changed

articles/ai-services/content-safety/whats-new.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Learn what's new in the service. These items might be release notes, videos, blo
2121
### Upcoming deprecations
2222

2323
To align with Content Safety versioning and lifecycle management policies, the following versions are scheduled for deprecation:
24-
* **Effective October 28, 2024**: All versions except `2024-09-01`, `2024-09-15-preview`, and `2024-09-30-preview` will be deprecated and no longer supported. We encourage users to transition to the latest available versions to continue receiving full support and updates. If you have any questions about this process or need assistance with the transition, please reach out to our support team.
24+
* **Effective March 1st, 2025**: All versions except `2024-09-01`, `2024-09-15-preview`, and `2024-09-30-preview` will be deprecated and no longer supported. We encourage users to transition to the latest available versions to continue receiving full support and updates. If you have any questions about this process or need assistance with the transition, please reach out to our support team.
2525

2626
## September 2024
2727

articles/ai-services/openai/how-to/content-filters.md

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,26 @@
11
---
2-
title: 'Use content filters (preview) with Azure OpenAI Service'
2+
title: 'Use content filters (preview) with Azure AI Foundry'
33
titleSuffix: Azure OpenAI
4-
description: Learn how to use and configure the content filters that come with Azure OpenAI Service, including getting approval for gated modifications.
4+
description: Learn how to use and configure the content filters that come with Azure AI Foundry, including getting approval for gated modifications.
55
#services: cognitive-services
66
manager: nitinme
77
ms.service: azure-ai-openai
88
ms.topic: how-to
9-
ms.date: 10/04/2024
9+
ms.date: 12/05/2024
1010
author: mrbullwinkle
1111
ms.author: mbullwin
1212
recommendations: false
1313
ms.custom: FY25Q1-Linter
14-
# customer intent: As a developer, I want to learn how to configure content filters with Azure OpenAI Service so that I can ensure that my applications comply with our Code of Conduct.
14+
# customer intent: As a developer, I want to learn how to configure content filters with Azure AI Foundry so that I can ensure that my applications comply with our Code of Conduct.
1515
---
1616

17-
# How to configure content filters with Azure OpenAI Service
17+
# How to configure content filters with Azure AI Foundry
1818

19-
The content filtering system integrated into Azure OpenAI Service runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories. The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md). Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
19+
The content filtering system integrated into Azure AI Foundry runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories.
20+
21+
The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md).
22+
23+
Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
2024

2125
> [!NOTE]
2226
> All customers have the ability to modify the content filters and configure the severity thresholds (low, medium, high). Approval is required for turning the content filters partially or fully off. Managed customers only may apply for full content filtering control via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR). At this time, it is not possible to become a managed customer.
@@ -37,7 +41,7 @@ You can configure the following filter categories in addition to the default har
3741

3842
|Filter category |Status |Default setting |Applied to prompt or completion? |Description |
3943
|---------|---------|---------|---------|---|
40-
|Prompt Shields for direct attacks (jailbreak) |GA| On | User prompt | Filters / annotates user prompts that might present a Jailbreak Risk. For more information about annotations, visit [Azure OpenAI Service content filtering](/azure/ai-services/openai/concepts/content-filter?tabs=python#annotations-preview). |
44+
|Prompt Shields for direct attacks (jailbreak) |GA| On | User prompt | Filters / annotates user prompts that might present a Jailbreak Risk. For more information about annotations, visit [Azure AI Foundry content filtering](/azure/ai-services/openai/concepts/content-filter?tabs=python#annotations-preview). |
4145
|Prompt Shields for indirect attacks | GA| Off | User prompt | Filter / annotate Indirect Attacks, also referred to as Indirect Prompt Attacks or Cross-Domain Prompt Injection Attacks, a potential vulnerability where third parties place malicious instructions inside of documents that the generative AI system can access and process. Requires: [Document embedding and formatting](/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#embedding-documents-in-your-prompt). |
4246
| Protected material - code |GA| On | Completion | Filters protected code or gets the example citation and license information in annotations for code snippets that match any public code sources, powered by GitHub Copilot. For more information about consuming annotations, see the [content filtering concepts guide](/azure/ai-services/openai/concepts/content-filter#annotations-preview) |
4347
| Protected material - text | GA| On | Completion | Identifies and blocks known text content from being displayed in the model output (for example, song lyrics, recipes, and selected web content). |
@@ -60,5 +64,5 @@ We recommend informing your content filtering configuration decisions through an
6064
## Related content
6165

6266
- Learn more about Responsible AI practices for Azure OpenAI: [Overview of Responsible AI practices for Azure OpenAI models](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
63-
- Read more about [content filtering categories and severity levels](../concepts/content-filter.md) with Azure OpenAI Service.
67+
- Read more about [content filtering categories and severity levels](../concepts/content-filter.md) with Azure AI Foundry.
6468
- Learn more about red teaming from our: [Introduction to red teaming large language models (LLMs) article](../concepts/red-teaming.md).

articles/ai-services/openai/how-to/dall-e.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,13 +24,13 @@ OpenAI's DALL-E models generate images based on user-provided text prompts. This
2424
#### [DALL-E 3](#tab/dalle3)
2525

2626
- An Azure subscription. You can [create one for free](https://azure.microsoft.com/pricing/purchase-options/azure-account?icid=ai-services).
27-
- An Azure OpenAI resource created in the *Sweden Central* region. For more information, see [Create and deploy an Azure OpenAI Service resource](../how-to/create-resource.md).
28-
- Deploy a *dall-e-3* model with your Azure OpenAI resource.
27+
- An Azure OpenAI resource created in a supported region. See [Region availability](/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability).
28+
- - Deploy a *dall-e-3* model with your Azure OpenAI resource.
2929

3030
#### [DALL-E 2 (preview)](#tab/dalle2)
3131

3232
- An Azure subscription. You can [create one for free](https://azure.microsoft.com/pricing/purchase-options/azure-account?icid=ai-services).
33-
- An Azure OpenAI resource created in the *East US* region. For more information, see [Create and deploy an Azure OpenAI Service resource](../how-to/create-resource.md).
33+
- An Azure OpenAI resource created in a supported region. See [Region availability](/azure/ai-services/openai/concepts/models#model-summary-table-and-region-availability). For more information, see [Create and deploy an Azure OpenAI Service resource](../how-to/create-resource.md).
3434

3535
---
3636

articles/ai-services/openai/how-to/risks-safety-monitor.md

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,28 @@
11
---
2-
title: How to use Risks & Safety monitoring in Azure OpenAI Studio
2+
title: How to use Risks & Safety monitoring in Azure AI Foundry
33
titleSuffix: Azure OpenAI Service
44
description: Learn how to check statistics and insights from your Azure OpenAI content filtering activity.
55
author: PatrickFarley
66
ms.author: pafarley
77
ms.service: azure-ai-openai
88
ms.topic: how-to
9-
ms.date: 10/03/2024
9+
ms.date: 12/05/2024
1010
manager: nitinme
1111
---
1212

13-
# Use Risks & Safety monitoring in Azure OpenAI Studio (preview)
13+
# Use Risks & Safety monitoring in Azure AI Foundry (preview)
1414

15-
When you use an Azure OpenAI model deployment with a content filter, you may want to check the results of the filtering activity. You can use that information to further adjust your filter configuration to serve your specific business needs and Responsible AI principles.
15+
When you use an Azure OpenAI model deployment with a content filter, you may want to check the results of the filtering activity. You can use that information to further adjust your [filter configuration](/azure/ai-services/openai/how-to/content-filters) to serve your specific business needs and Responsible AI principles.
1616

17-
[Azure OpenAI Studio](https://oai.azure.com/) provides a Risks & Safety monitoring dashboard for each of your deployments that uses a content filter configuration.
17+
[Azure AI Foundry](https://oai.azure.com/) provides a Risks & Safety monitoring dashboard for each of your deployments that uses a content filter configuration.
1818

1919
## Access Risks & Safety monitoring
2020

2121
To access Risks & Safety monitoring, you need an Azure OpenAI resource in one of the supported Azure regions: East US, Switzerland North, France Central, Sweden Central, Canada East. You also need a model deployment that uses a content filter configuration.
2222

23-
Go to [Azure OpenAI Studio](https://oai.azure.com/) and sign in with the credentials associated with your Azure OpenAI resource. Select the **Deployments** tab on the left and then select your model deployment from the list. On the deployment's page, select the **Risks & Safety** tab at the top.
23+
Go to [Azure AI Foundry](https://oai.azure.com/) and sign in with the credentials associated with your Azure OpenAI resource. Select a project. Then select the **Models + endpoints** tab on the left and then select your model deployment from the list. On the deployment's page, select the **Metrics** tab at the top. Then select **Open in Azure Monitor** to view the full report in the Azure portal.
2424

25-
## Content detection
26-
27-
The **Content detection** pane shows information about content filter activity. Your content filter configuration is applied as described in the [Content filtering documentation](/azure/ai-services/openai/how-to/content-filters).
25+
## Configure metrics
2826

2927
### Report description
3028

@@ -35,7 +33,9 @@ Content filtering data is shown in the following ways:
3533
- **Severity distribution by category**: This view shows the severity levels detected for each harm category, across the whole selected time range. This is not limited to _blocked_ content but rather includes all content that was flagged by the content filters.
3634
- **Severity rate distribution over time by category**: This view shows the rates of detected severity levels over time, for each harm category. Select the tabs to switch between supported categories.
3735

36+
<!--
3837
:::image type="content" source="../media/how-to/content-detection.png" alt-text="Screenshot of the content detection pane in the Risks & Safety monitoring page." lightbox="../media/how-to/content-detection.png":::
38+
-->
3939

4040
### Recommended actions
4141

@@ -56,7 +56,7 @@ To use Potentially abusive user detection, you need:
5656
### Set up your Azure Data Explorer database
5757

5858
In order to protect the data privacy of user information and manage the permission of the data, we support the option for our customers to bring their own storage to get the detailed potentially abusive user detection insights (including user GUID and statistics on harmful request by category) stored in a compliant way and with full control. Follow these steps to enable it:
59-
1. In Azure OpenAI Studio, navigate to the model deployment that you'd like to set up user abuse analysis with, and select **Add a data store**.
59+
1. In Azure AI Foundry, navigate to the model deployment that you'd like to set up user abuse analysis with, and select **Add a data store**.
6060
1. Fill in the required information and select **Save**. We recommend you create a new database to store the analysis results.
6161
1. After you connect the data store, take the following steps to grant permission to write analysis results to the connected database:
6262
1. Go to your Azure OpenAI resource's page in the Azure portal, and choose the **Identity** tab.
@@ -81,14 +81,16 @@ The potentially abusive user detection relies on the user information that custo
8181
- **Total abuse request ratio/count**
8282
- **Abuse ratio/count by category**
8383

84+
<!--
8485
:::image type="content" source="../media/how-to/potentially-abusive-user.png" alt-text="Screenshot of the Potentially abusive user detection pane in the Risks & Safety monitoring page." lightbox="../media/how-to/potentially-abusive-user.png":::
86+
-->
8587

8688
### Recommended actions
8789

8890
Combine this data with enriched signals to validate whether the detected users are truly abusive or not. If they are, then take responsive action such as throttling or suspending the user to ensure the responsible use of your application.
8991

9092
## Next steps
9193

92-
Next, create or edit a content filter configuration in Azure OpenAI Studio.
94+
Next, create or edit a content filter configuration in Azure AI Foundry.
9395

9496
- [Configure content filters with Azure OpenAI Service](/azure/ai-services/openai/how-to/content-filters)

0 commit comments

Comments
 (0)