You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/jailbreak-detection.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,27 +1,27 @@
1
1
---
2
2
title: "Prompt Shields in Azure AI Content Safety"
3
3
titleSuffix: Azure AI services
4
-
description: Learn about User Prompt injection attacks and the Prompt Shields feature that helps prevent them.
4
+
description: Learn about User Prompt injection attacks and document attacks and how to prevent them with the Prompt Shields feature.
5
5
#services: cognitive-services
6
6
author: PatrickFarley
7
7
manager: nitinme
8
8
ms.service: azure-ai-content-safety
9
9
ms.custom: build-2023
10
10
ms.topic: conceptual
11
-
ms.date: 03/15/2024
11
+
ms.date: 09/25/2024
12
12
ms.author: pafarley
13
13
---
14
14
15
15
# Prompt Shields
16
16
17
-
Generative AI models can pose risks of exploitation by malicious actors. To mitigate these risks, we integrate safety mechanisms to restrict the behavior of large language models (LLMs) within a safe operational scope. However, despite these safeguards, LLMs can still be vulnerable to adversarial inputs that bypass the integrated safety protocols.
17
+
Generative AI models can pose risks of being exploited by malicious actors. To mitigate these risks, we integrate safety mechanisms to restrict the behavior of large language models (LLMs) within a safe operational scope. However, despite these safeguards, LLMs can still be vulnerable to adversarial inputs that bypass the integrated safety protocols.
18
18
19
-
Prompt Shields is a unified API that analyzes LLM inputs and detects User Prompt attacks and Document attacks, which are two common types of adversarial inputs.
19
+
Prompt Shields is a unified API that analyzes LLM inputs and detects adversarial user input attacks.
20
20
21
21
22
22
## Types of input attacks
23
23
24
-
The two types of input attacks that Prompt Shields detects are described in this table.
24
+
The types of input attacks that Prompt Shields detects are described in this table.
25
25
26
26
| Type | Attacker | Entry point | Method | Objective/impact | Resulting behavior |
@@ -92,11 +92,11 @@ Prompt Shields have been specifically trained and tested on the following langua
92
92
93
93
See [Input requirements](/azure/ai-services/content-safety/overview#input-requirements) for maximum text length limitations.
94
94
95
-
### Regions
95
+
### Region availability
96
96
97
97
To use this API, you must create your Azure AI Content Safety resource in the supported regions. See [Region availability](/azure/ai-services/content-safety/overview#region-availability).
98
98
99
-
### TPS limitations
99
+
### Rate limitations
100
100
101
101
See [Query rates](/azure/ai-services/content-safety/overview#query-rates).
Copy file name to clipboardExpand all lines: articles/ai-services/openai/concepts/content-filter.md
+1-23Lines changed: 1 addition & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -81,31 +81,9 @@ Detecting indirect attacks requires using document delimiters when constructing
81
81
82
82
## Configurability
83
83
84
-
Azure OpenAI Service includes default safety settings applied to all models, excluding Azure OpenAI Whisper. These configurations provide you with a responsible experience by default, including content filtering models, blocklists, prompt transformation, [content credentials](../concepts/content-credentials.md), and others. [Read more about it here](/azure/ai-services/openai/concepts/default-safety-policies). All customers can also configure content filters and create custom safety policies that are tailored to their use case requirements. The configurability feature allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels as described in the table below:
| Low, medium, high | Yes | Yes | Strictest filtering configuration. Content detected at severity levels low, medium, and high is filtered.|
89
-
| Medium, high | Yes | Yes | Content detected at severity level low isn't filtered, content at medium and high is filtered.|
90
-
| High | Yes| Yes | Content detected at severity levels low and medium isn't filtered. Only content at severity level high is filtered. |
91
-
| No filters | If approved<sup>1</sup>| If approved<sup>1</sup>| No content is filtered regardless of severity level detected. Requires approval<sup>1</sup>.|
92
-
|Annotate only | If approved<sup>1</sup>| If approved<sup>1</sup>| Disables the filter functionality, so content will not be blocked, but annotations are returned via API response. Requires approval<sup>1</sup>.|
93
86
94
-
<sup>1</sup> For Azure OpenAI models, only customers who have been approved for modified content filtering have full content filtering control and can turn off content filters. Apply for modified content filters via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR) For Azure Government customers, please apply for modified content filters via this form: [Azure Government - Request Modified Content Filtering for Azure OpenAI Service](https://aka.ms/AOAIGovModifyContentFilter).
95
-
96
-
Configurable content filters for inputs (prompts) and outputs (completions) are available for the following Azure OpenAI models:
<sup>*</sup>Only available for GPT-4 Turbo Vision GA, does not apply to GPT-4 Turbo Vision preview
105
-
106
-
Content filtering configurations are created within a Resource in Azure AI Studio, and can be associated with Deployments. [Learn more about configurability here](../how-to/content-filters.md).
107
-
108
-
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
Copy file name to clipboardExpand all lines: articles/ai-services/openai/concepts/gpt-with-vision.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,12 @@
1
1
---
2
2
title: GPT-4 Turbo with Vision concepts
3
3
titleSuffix: Azure OpenAI
4
-
description: Learn about vision chats enabled by GPT-4 Turbo with Vision.
4
+
description: Learn concepts related to using images in your AI model chats, enabled through GPT-4 Turbo with Vision and other models.
5
5
author: PatrickFarley
6
6
ms.author: pafarley
7
7
ms.service: azure-ai-openai
8
8
ms.topic: conceptual
9
-
ms.date: 01/02/2024
9
+
ms.date: 09/24/2024
10
10
manager: nitinme
11
11
---
12
12
@@ -36,6 +36,7 @@ See the [Tokens section of the overview](/azure/ai-services/openai/overview#toke
36
36
37
37
38
38
### Example image price calculation
39
+
39
40
> [!IMPORTANT]
40
41
> The following content is an example only, and prices are subject to change in the future.
41
42
@@ -68,7 +69,7 @@ For a typical use case, take a 3-minute video with a 100-token prompt input. The
68
69
69
70
Additionally, there's a one-time indexing cost of $0.15 to generate the Video Retrieval index for this 3-minute video. This index can be reused across any number of Video Retrieval and GPT-4 Turbo with Vision API calls.
70
71
71
-
## Limitations
72
+
## Input limitations
72
73
73
74
This section describes the limitations of GPT-4 Turbo with Vision.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/how-to/content-filters.md
+18-19Lines changed: 18 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,40 +1,39 @@
1
1
---
2
-
title: 'How to use content filters (preview) with Azure OpenAI Service'
2
+
title: 'Use content filters (preview) with Azure OpenAI Service'
3
3
titleSuffix: Azure OpenAI
4
-
description: Learn how to use content filters (preview) with Azure OpenAI Service.
4
+
description: Learn how to use and configure the content filters that come with Azure OpenAI Service, including getting approval for gated modifications.
5
5
#services: cognitive-services
6
6
manager: nitinme
7
7
ms.service: azure-ai-openai
8
8
ms.topic: how-to
9
-
ms.date: 04/16/2024
9
+
ms.date: 09/25/2024
10
10
author: mrbullwinkle
11
11
ms.author: mbullwin
12
12
recommendations: false
13
+
ms.custom: FY25Q1-Linter
14
+
# customer intent: As a developer, I want to learn how to configure content filters with Azure OpenAI Service so that I can ensure that my applications comply with our Code of Conduct.
13
15
---
14
16
15
17
# How to configure content filters with Azure OpenAI Service
16
18
19
+
The content filtering system integrated into Azure OpenAI Service runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories. The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md). Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
20
+
17
21
> [!NOTE]
18
22
> All customers have the ability to modify the content filters and configure the severity thresholds (low, medium, high). Approval is required for turning the content filters partially or fully off. Managed customers only may apply for full content filtering control via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR). At this time, it is not possible to become a managed customer.
19
23
20
-
The content filtering system integrated into Azure OpenAI Service runs alongside the core models, including DALL-E image generation models. It uses an ensemble of multi-class classification models to detect four categories of harmful content (violence, hate, sexual, and self-harm) at four severity levels respectively (safe, low, medium, and high), and optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories. The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. Learn more about content categories, severity levels, and the behavior of the content filtering system [here](../concepts/content-filter.md). Jailbreak risk detection and protected text and code models are optional and off by default. For jailbreak and protected material text and code models, the configurability feature allows all customers to turn the models on and off. The models are by default off and can be turned on per your scenario. Some models are required to be on for certain scenarios to retain coverage under the [Customer Copyright Commitment](/legal/cognitive-services/openai/customer-copyright-commitment?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
21
-
22
-
Content filters can be configured at resource level. Once a new configuration is created, it can be associated with one or more deployments. For more information about model deployment, see the [resource deployment guide](create-resource.md).
24
+
Content filters can be configured at the resource level. Once a new configuration is created, it can be associated with one or more deployments. For more information about model deployment, see the [resource deployment guide](create-resource.md).
23
25
24
-
The configurability feature allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels as described in the table below. Content detected at the 'safe' severity level is labeled in annotations but is not subject to filtering and isn't configurable.
26
+
## Prerequisites
25
27
26
-
| Severity filtered | Configurable for prompts | Configurable for completions | Descriptions |
| Low, medium, high | Yes | Yes | Strictest filtering configuration. Content detected at severity levels low, medium, and high is filtered. |
29
-
| Medium, high | Yes | Yes | Content detected at severity level low isn't filtered, content at medium and high is filtered. |
30
-
| High | Yes| Yes | Content detected at severity levels low and medium isn't filtered. Only content at severity level high is filtered. |
31
-
| No filters | If approved<sup>\*</sup>| If approved<sup>\*</sup>| No content is filtered regardless of severity level detected. Requires approval<sup>\*</sup>.|
32
-
|Annotate only | If approved<sup>\*</sup>| If approved<sup>\*</sup>| Disables the filter functionality, so content will not be blocked, but annotations are returned via API response. Requires approval<sup>\*</sup>|
28
+
* You must have an Azure OpenAI resource and a large language model (LLM) deployment to configure content filters. Follow a [quickstart](/azure/ai-services/openai/chatgpt-quickstart?) to get started.
33
29
34
-
<sup>\*</sup> Only approved customers have full content filtering control and can turn the content filters partially or fully off. Managed customers only can apply for full content filtering control via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR). At this time, it is not possible to become a managed customer.
30
+
## Understand content filter configurability
35
31
36
-
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
You can configure the following filter categories in addition to the default harm category filters.
38
37
39
38
|Filter category |Status |Default setting |Applied to prompt or completion? |Description |
40
39
|---------|---------|---------|---------|
@@ -44,7 +43,7 @@ Customers are responsible for ensuring that applications integrating Azure OpenA
44
43
| Protected material - text | GA| On | Completion | Identifies and blocks known text content from being displayed in the model output (for example, song lyrics, recipes, and selected web content). |
45
44
46
45
47
-
## Configuring content filters via Azure OpenAI Studio
46
+
## Configure content filters via Azure OpenAI Studio
48
47
49
48
The following steps show how to set up a customized content filtering configuration for your resource.
50
49
@@ -101,11 +100,11 @@ The following steps show how to set up a customized content filtering configurat
101
100
> [!NOTE]
102
101
> Before deleting a content filtering configuration, you will need to unassign it from any deployment in the Deployments tab.
103
102
104
-
## Best practices
103
+
## Follow best practices
105
104
106
105
We recommend informing your content filtering configuration decisions through an iterative identification (for example, red team testing, stress-testing, and analysis) and measurement process to address the potential harms that are relevant for a specific model, application, and deployment scenario. After you implement mitigations such as content filtering, repeat measurement to test effectiveness. Recommendations and best practices for Responsible AI for Azure OpenAI, grounded in the [Microsoft Responsible AI Standard](https://aka.ms/RAI) can be found in the [Responsible AI Overview for Azure OpenAI](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
107
106
108
-
## Next steps
107
+
## Related content
109
108
110
109
- Learn more about Responsible AI practices for Azure OpenAI: [Overview of Responsible AI practices for Azure OpenAI models](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
111
110
- Read more about [content filtering categories and severity levels](../concepts/content-filter.md) with Azure OpenAI Service.
0 commit comments