Skip to content

Commit 84c1410

Browse files
committed
freshness
1 parent 3676ab8 commit 84c1410

File tree

6 files changed

+78
-55
lines changed

6 files changed

+78
-55
lines changed

articles/ai-services/content-safety/concepts/jailbreak-detection.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,27 @@
11
---
22
title: "Prompt Shields in Azure AI Content Safety"
33
titleSuffix: Azure AI services
4-
description: Learn about User Prompt injection attacks and the Prompt Shields feature that helps prevent them.
4+
description: Learn about User Prompt injection attacks and document attacks and how to prevent them with the Prompt Shields feature.
55
#services: cognitive-services
66
author: PatrickFarley
77
manager: nitinme
88
ms.service: azure-ai-content-safety
99
ms.custom: build-2023
1010
ms.topic: conceptual
11-
ms.date: 03/15/2024
11+
ms.date: 09/25/2024
1212
ms.author: pafarley
1313
---
1414

1515
# Prompt Shields
1616

17-
Generative AI models can pose risks of exploitation by malicious actors. To mitigate these risks, we integrate safety mechanisms to restrict the behavior of large language models (LLMs) within a safe operational scope. However, despite these safeguards, LLMs can still be vulnerable to adversarial inputs that bypass the integrated safety protocols.
17+
Generative AI models can pose risks of being exploited by malicious actors. To mitigate these risks, we integrate safety mechanisms to restrict the behavior of large language models (LLMs) within a safe operational scope. However, despite these safeguards, LLMs can still be vulnerable to adversarial inputs that bypass the integrated safety protocols.
1818

19-
Prompt Shields is a unified API that analyzes LLM inputs and detects User Prompt attacks and Document attacks, which are two common types of adversarial inputs.
19+
Prompt Shields is a unified API that analyzes LLM inputs and detects adversarial user input attacks.
2020

2121

2222
## Types of input attacks
2323

24-
The two types of input attacks that Prompt Shields detects are described in this table.
24+
The types of input attacks that Prompt Shields detects are described in this table.
2525

2626
| Type | Attacker | Entry point | Method | Objective/impact | Resulting behavior |
2727
|-------|----------|---------|---------|---------|---------|
@@ -92,11 +92,11 @@ Prompt Shields have been specifically trained and tested on the following langua
9292

9393
See [Input requirements](/azure/ai-services/content-safety/overview#input-requirements) for maximum text length limitations.
9494

95-
### Regions
95+
### Region availability
9696

9797
To use this API, you must create your Azure AI Content Safety resource in the supported regions. See [Region availability](/azure/ai-services/content-safety/overview#region-availability).
9898

99-
### TPS limitations
99+
### Rate limitations
100100

101101
See [Query rates](/azure/ai-services/content-safety/overview#query-rates).
102102

articles/ai-services/content-safety/index.yml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ metadata:
1111
ms.topic: landing-page # Required
1212
author: PatrickFarley #Required; your GitHub user alias, with correct capitalization.
1313
ms.author: pafarley #Required; microsoft alias of author; optional team alias.
14-
ms.date: 02/14/2024 #Required; mm/dd/yyyy format.
14+
ms.date: 09/24/2024 #Required; mm/dd/yyyy format.
1515
# linkListType: architecture | concept | deploy | download | get-started | how-to-guide | learn | overview | quickstart | reference | tutorial | video | whats-new
1616

1717

@@ -44,7 +44,7 @@ landingContent:
4444
links:
4545
- text: Harm categories
4646
url: concepts/harm-categories.md
47-
- text: Custom categories (standard) (preview)
47+
- text: Custom categories (preview)
4848
url: concepts/custom-categories.md
4949
- linkListType: quickstart
5050
links:
@@ -56,6 +56,8 @@ landingContent:
5656
links:
5757
- text: Use custom categories (standard) (preview)
5858
url: how-to/custom-categories.md
59+
- text: Use custom categories (rapid) (preview)
60+
url: how-to/custom-categories-rapid.md
5961

6062

6163
- title: Text moderation
@@ -64,7 +66,7 @@ landingContent:
6466
links:
6567
- text: Harm categories
6668
url: concepts/harm-categories.md
67-
- text: Custom categories (standard) (preview)
69+
- text: Custom categories (preview)
6870
url: concepts/custom-categories.md
6971
- text: Groundedness detection
7072
url: concepts/groundedness.md
@@ -80,6 +82,8 @@ landingContent:
8082
links:
8183
- text: Use custom categories (standard) (preview)
8284
url: how-to/custom-categories.md
85+
- text: Use custom categories (rapid) (preview)
86+
url: how-to/custom-categories-rapid.md
8387
- text: Use a blocklist
8488
url: how-to/use-blocklist.md
8589

articles/ai-services/openai/concepts/content-filter.md

Lines changed: 1 addition & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -81,31 +81,9 @@ Detecting indirect attacks requires using document delimiters when constructing
8181

8282
## Configurability
8383

84-
Azure OpenAI Service includes default safety settings applied to all models, excluding Azure OpenAI Whisper. These configurations provide you with a responsible experience by default, including content filtering models, blocklists, prompt transformation, [content credentials](../concepts/content-credentials.md), and others. [Read more about it here](/azure/ai-services/openai/concepts/default-safety-policies). All customers can also configure content filters and create custom safety policies that are tailored to their use case requirements. The configurability feature allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels as described in the table below:
84+
[!INCLUDE [content-filter-configurability](../includes/content-filter-configurability.md)]
8585

86-
| Severity filtered | Configurable for prompts | Configurable for completions | Descriptions |
87-
|-------------------|--------------------------|------------------------------|--------------|
88-
| Low, medium, high | Yes | Yes | Strictest filtering configuration. Content detected at severity levels low, medium, and high is filtered.|
89-
| Medium, high | Yes | Yes | Content detected at severity level low isn't filtered, content at medium and high is filtered.|
90-
| High | Yes| Yes | Content detected at severity levels low and medium isn't filtered. Only content at severity level high is filtered. |
91-
| No filters | If approved<sup>1</sup>| If approved<sup>1</sup>| No content is filtered regardless of severity level detected. Requires approval<sup>1</sup>.|
92-
|Annotate only | If approved<sup>1</sup>| If approved<sup>1</sup>| Disables the filter functionality, so content will not be blocked, but annotations are returned via API response. Requires approval<sup>1</sup>.|
9386

94-
<sup>1</sup> For Azure OpenAI models, only customers who have been approved for modified content filtering have full content filtering control and can turn off content filters. Apply for modified content filters via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR) For Azure Government customers, please apply for modified content filters via this form: [Azure Government - Request Modified Content Filtering for Azure OpenAI Service](https://aka.ms/AOAIGovModifyContentFilter).
95-
96-
Configurable content filters for inputs (prompts) and outputs (completions) are available for the following Azure OpenAI models:
97-
98-
* GPT model series
99-
* GPT-4 Turbo Vision GA<sup>*</sup> (turbo-2024-04-09)
100-
* GPT-4o
101-
* GPT-4o mini
102-
* DALL-E 2 and 3
103-
104-
<sup>*</sup>Only available for GPT-4 Turbo Vision GA, does not apply to GPT-4 Turbo Vision preview
105-
106-
Content filtering configurations are created within a Resource in Azure AI Studio, and can be associated with Deployments. [Learn more about configurability here](../how-to/content-filters.md).
107-
108-
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
10987

11088
## Scenario details
11189

articles/ai-services/openai/concepts/gpt-with-vision.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
---
22
title: GPT-4 Turbo with Vision concepts
33
titleSuffix: Azure OpenAI
4-
description: Learn about vision chats enabled by GPT-4 Turbo with Vision.
4+
description: Learn concepts related to using images in your AI model chats, enabled through GPT-4 Turbo with Vision and other models.
55
author: PatrickFarley
66
ms.author: pafarley
77
ms.service: azure-ai-openai
88
ms.topic: conceptual
9-
ms.date: 01/02/2024
9+
ms.date: 09/24/2024
1010
manager: nitinme
1111
---
1212

@@ -36,6 +36,7 @@ See the [Tokens section of the overview](/azure/ai-services/openai/overview#toke
3636

3737

3838
### Example image price calculation
39+
3940
> [!IMPORTANT]
4041
> The following content is an example only, and prices are subject to change in the future.
4142
@@ -68,7 +69,7 @@ For a typical use case, take a 3-minute video with a 100-token prompt input. The
6869

6970
Additionally, there's a one-time indexing cost of $0.15 to generate the Video Retrieval index for this 3-minute video. This index can be reused across any number of Video Retrieval and GPT-4 Turbo with Vision API calls.
7071

71-
## Limitations
72+
## Input limitations
7273

7374
This section describes the limitations of GPT-4 Turbo with Vision.
7475

articles/ai-services/openai/how-to/content-filters.md

Lines changed: 18 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,39 @@
11
---
2-
title: 'How to use content filters (preview) with Azure OpenAI Service'
2+
title: 'Use content filters (preview) with Azure OpenAI Service'
33
titleSuffix: Azure OpenAI
4-
description: Learn how to use content filters (preview) with Azure OpenAI Service.
4+
description: Learn how to use and configure the content filters that come with Azure OpenAI Service, including getting approval for gated modifications.
55
#services: cognitive-services
66
manager: nitinme
77
ms.service: azure-ai-openai
88
ms.topic: how-to
9-
ms.date: 04/16/2024
9+
ms.date: 09/25/2024
1010
author: mrbullwinkle
1111
ms.author: mbullwin
1212
recommendations: false
13+
ms.custom: FY25Q1-Linter
14+
# customer intent: As a developer, I want to learn how to configure content filters with Azure OpenAI Service so that I can ensure that my applications comply with our Code of Conduct.
1315
---
1416

1517
# How to configure content filters with Azure OpenAI Service
1618

19+
The content filtering system integrated into Azure OpenAI Service runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories. The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md). Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
20+
1721
> [!NOTE]
1822
> All customers have the ability to modify the content filters and configure the severity thresholds (low, medium, high). Approval is required for turning the content filters partially or fully off. Managed customers only may apply for full content filtering control via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR). At this time, it is not possible to become a managed customer.
1923
20-
The content filtering system integrated into Azure OpenAI Service runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories. The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md). Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
21-
22-
Content filters can be configured at resource level. Once a new configuration is created, it can be associated with one or more deployments. For more information about model deployment, see the [resource deployment guide](create-resource.md).
24+
Content filters can be configured at the resource level. Once a new configuration is created, it can be associated with one or more deployments. For more information about model deployment, see the [resource deployment guide](create-resource.md).
2325

24-
The configurability feature allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels as described in the table below. Content detected at the 'safe' severity level is labeled in annotations but is not subject to filtering and isn't configurable.
26+
## Prerequisites
2527

26-
| Severity filtered | Configurable for prompts | Configurable for completions | Descriptions |
27-
|-------------------|--------------------------|------------------------------|--------------|
28-
| Low, medium, high | Yes | Yes | Strictest filtering configuration. Content detected at severity levels low, medium, and high is filtered. |
29-
| Medium, high | Yes | Yes | Content detected at severity level low isn't filtered, content at medium and high is filtered. |
30-
| High | Yes| Yes | Content detected at severity levels low and medium isn't filtered. Only content at severity level high is filtered. |
31-
| No filters | If approved<sup>\*</sup>| If approved<sup>\*</sup>| No content is filtered regardless of severity level detected. Requires approval<sup>\*</sup>.|
32-
|Annotate only | If approved<sup>\*</sup>| If approved<sup>\*</sup>| Disables the filter functionality, so content will not be blocked, but annotations are returned via API response. Requires approval<sup>\*</sup>|
28+
* You must have an Azure OpenAI resource and a large language model (LLM) deployment to configure content filters. Follow a [quickstart](/azure/ai-services/openai/chatgpt-quickstart?) to get started.
3329

34-
<sup>\*</sup> Only approved customers have full content filtering control and can turn the content filters partially or fully off. Managed customers only can apply for full content filtering control via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR). At this time, it is not possible to become a managed customer.
30+
## Understand content filter configurability
3531

36-
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
32+
[!INCLUDE [content-filter-configurability](../includes/content-filter-configurability.md)]
33+
34+
## Understand other filters
3735

36+
You can configure the following filter categories in addition to the default harm category filters.
3837

3938
|Filter category |Status |Default setting |Applied to prompt or completion? |Description |
4039
|---------|---------|---------|---------|
@@ -44,7 +43,7 @@ Customers are responsible for ensuring that applications integrating Azure OpenA
4443
| Protected material - text | GA| On | Completion | Identifies and blocks known text content from being displayed in the model output (for example, song lyrics, recipes, and selected web content). |
4544

4645

47-
## Configuring content filters via Azure OpenAI Studio
46+
## Configure content filters via Azure OpenAI Studio
4847

4948
The following steps show how to set up a customized content filtering configuration for your resource.
5049

@@ -101,11 +100,11 @@ The following steps show how to set up a customized content filtering configurat
101100
> [!NOTE]
102101
> Before deleting a content filtering configuration, you will need to unassign it from any deployment in the Deployments tab.
103102
104-
## Best practices
103+
## Follow best practices
105104

106105
We recommend informing your content filtering configuration decisions through an iterative identification (for example, red team testing, stress-testing, and analysis) and measurement process to address the potential harms that are relevant for a specific model, application, and deployment scenario. After you implement mitigations such as content filtering, repeat measurement to test effectiveness. Recommendations and best practices for Responsible AI for Azure OpenAI, grounded in the [Microsoft Responsible AI Standard](https://aka.ms/RAI) can be found in the [Responsible AI Overview for Azure OpenAI](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
107106

108-
## Next steps
107+
## Related content
109108

110109
- Learn more about Responsible AI practices for Azure OpenAI: [Overview of Responsible AI practices for Azure OpenAI models](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
111110
- Read more about [content filtering categories and severity levels](../concepts/content-filter.md) with Azure OpenAI Service.

0 commit comments

Comments
 (0)