You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/quickstart-jailbreak.md
+40-4Lines changed: 40 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: "Quickstart: Prompt Shields (preview)"
2
+
title: "Quickstart: Prompt Shields "
3
3
titleSuffix: Azure AI services
4
4
description: Learn how to detect large language model input attack risks and mitigate risk with Azure AI Content Safety.
5
5
services: ai-services
@@ -13,10 +13,46 @@ ms.author: pafarley
13
13
14
14
# Quickstart: Prompt Shields (preview)
15
15
16
-
Follow this guide to use Azure AI Content Safety Prompt Shields to check your large language model (LLM) inputs for both User Prompt and Document attacks.
16
+
"Prompt Shields" in Azure AI Content Safety are specifically designed to safeguard generative AI systems from generating harmful or inappropriate content. These shields detect and mitigate risks associated with both User Prompt Attacks (malicious or harmful user-generated inputs) and Document Attacks (inputs containing harmful content embedded within documents). The use of "Prompt Shields" is crucial in environments where GenAI is employed, ensuring that AI outputs remain safe, compliant, and trustworthy.
17
+
18
+
The primary objectives of the "Prompt Shields" feature for GenAI applications are:
19
+
20
+
- To detect and block harmful or policy-violating user prompts that could lead to unsafe AI outputs.
21
+
- To identify and mitigate document attacks where harmful content is embedded within user-provided documents.
22
+
- To maintain the integrity, safety, and compliance of AI-generated content, thereby preventing misuse of GenAI systems.
17
23
18
24
For more information on Prompt Shields, see the [Prompt Shields concept page](./concepts/jailbreak-detection.md). For API input limits, see the [Input requirements](./overview.md#input-requirements) section of the Overview.
19
25
26
+
27
+
28
+
29
+
## User scenarios
30
+
### 1. AI content creation platforms: Detecting harmful prompts
31
+
- Scenario: An AI content creation platform uses generative AI models to produce marketing copy, social media posts, and articles based on user-provided prompts. To prevent the generation of harmful or inappropriate content, the platform integrates "Prompt Shields."
32
+
- User: Content creators, platform administrators, and compliance officers.
33
+
- Action: The platform uses Azure AI Content Safety's "Prompt Shields" to analyze user prompts before generating content. If a prompt is detected as potentially harmful or likely to lead to policy-violating outputs (e.g., prompts asking for defamatory content or hate speech), the shield blocks the prompt and alerts the user to modify their input.
34
+
- Outcome: The platform ensures all AI-generated content is safe, ethical, and compliant with community guidelines, enhancing user trust and protecting the platform's reputation.
35
+
### 2. AI-powered chatbots: Mitigating risk from user prompt attacks
36
+
- Scenario: A customer service provider uses AI-powered chatbots for automated support. To safeguard against user prompts that could lead the AI to generate inappropriate or unsafe responses, the provider uses "Prompt Shields."
37
+
- User: Customer service agents, chatbot developers, and compliance teams.
38
+
- Action: The chatbot system integrates "Prompt Shields" to monitor and evaluate user inputs in real-time. If a user prompt is identified as potentially harmful or designed to exploit the AI (e.g., attempting to provoke inappropriate responses or extract sensitive information), the shield intervenes by blocking the response or redirecting the query to a human agent.
39
+
- Outcome: The customer service provider maintains high standards of interaction safety and compliance, preventing the chatbot from generating responses that could harm users or breach policies.
- Scenario: An e-learning platform employs GenAI to generate personalized educational content based on student inputs and reference documents. To avoid generating inappropriate or misleading educational content, the platform utilizes "Prompt Shields."
42
+
- User: Educators, content developers, and compliance officers.
43
+
- Action: The platform uses "Prompt Shields" to analyze both user prompts and uploaded documents for content that could lead to unsafe or policy-violating AI outputs. If a prompt or document is detected as likely to generate inappropriate educational content, the shield blocks it and suggests alternative, safe inputs.
44
+
- Outcome: The platform ensures that all AI-generated educational materials are appropriate and compliant with academic standards, fostering a safe and effective learning environment.
45
+
### 4. Healthcare AI assistants: Blocking unsafe prompts and document inputs
46
+
- Scenario: A healthcare provider uses AI assistants to offer preliminary medical advice based on user inputs and uploaded medical documents. To ensure the AI does not generate unsafe or misleading medical advice, the provider implements "Prompt Shields."
47
+
- User: Healthcare providers, AI developers, and compliance teams.
48
+
- Action: The AI assistant employs "Prompt Shields" to analyze patient prompts and uploaded medical documents for harmful or misleading content. If a prompt or document is identified as potentially leading to unsafe medical advice, the shield prevents the AI from generating a response and redirects the patient to a human healthcare professional.
49
+
- Outcome: The healthcare provider ensures that AI-generated medical advice remains safe and accurate, protecting patient safety and maintaining compliance with healthcare regulations.
50
+
### 5. Generative AI for creative writing: Protecting against prompt manipulation
51
+
- Scenario: A creative writing platform uses GenAI to assist writers in generating stories, poetry, and scripts based on user inputs. To prevent the generation of inappropriate or offensive content, the platform incorporates "Prompt Shields."
52
+
- User: Writers, platform moderators, and content reviewers.
53
+
- Action: The platform integrates "Prompt Shields" to evaluate user prompts for creative writing. If a prompt is detected as likely to produce offensive, defamatory, or otherwise inappropriate content, the shield blocks the AI from generating such content and suggests revisions to the user.
54
+
55
+
20
56
## Prerequisites
21
57
22
58
* An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/)
@@ -33,7 +69,7 @@ This section walks through a sample request with cURL. Paste the command below i
33
69
1. Optionally, replace the `"userPrompt"` or `"documents"` fields in the body with your own text you'd like to analyze.
34
70
35
71
```shell
36
-
curl --location --request POST '<endpoint>/contentsafety/text:shieldPrompt?api-version=2024-02-15-preview' \
72
+
curl --location --request POST '<endpoint>/contentsafety/text:shieldPrompt?api-version=2024-09-01' \
@@ -48,7 +84,7 @@ The following fields must be included in the URL:
48
84
49
85
| Name | Required? | Description | Type |
50
86
| :-- | :-- | :----- | :----- |
51
-
|**API Version**| Required | This is the API version to be used. The current version is: api-version=2024-02-15-preview. Example: `<endpoint>/contentsafety/text:shieldPrompt?api-version=2024-02-15-preview`| String |
87
+
|**API Version**| Required | This is the API version to be used. The current version is: api-version=2024-09-01. Example: `<endpoint>/contentsafety/text:shieldPrompt?api-version=2024-09-01`| String |
52
88
53
89
The parameters in the request body are defined in this table:
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/quickstart-protected-material.md
+39-7Lines changed: 39 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
-
title: "Quickstart: Detect protected material (preview)"
2
+
title: "Quickstart: Protected Material for Text"
3
3
titleSuffix: Azure AI services
4
-
description: Learn how to detect protected material generated by large language models and mitigate risk with Azure AI Content Safety.
4
+
description: Learn how to Protected Material for Text generated by large language models and mitigate risk with Azure AI Content Safety.
5
5
services: ai-services
6
6
author: PatrickFarley
7
7
manager: nitinme
@@ -11,13 +11,46 @@ ms.date: 10/30/2023
11
11
ms.author: pafarley
12
12
---
13
13
14
-
# Quickstart: Detect protected material (preview)
14
+
# Quickstart: Protected Material Detection for Text
15
15
16
-
Protected material text describes language that matches known text content (for example, song lyrics, articles, recipes, selected web content). This feature can be used to identify and block known text content from being displayed in language model output (English content only). For more information, see [Protected material concepts](./concepts/protected-material.md).
16
+
The "Protected Material for Text" feature provides a robust solution for managing risks associated with GenAI-generated content (English content only). By detecting and preventing the display of protected material, organizations can ensure compliance with intellectual property laws, maintain content originality, and protect their reputations. Protected material refers to content that matches known text from copyrighted sources, such as song lyrics, articles, recipes, or other selected web content.
17
+
The key objectives of the "Protected Material Detection for Text" feature for GenAI-generated content are:
18
+
19
+
- To detect and prevent the display of protected content generated by AI models.
20
+
- To enable organizations to manage risks associated with AI-generated content.
21
+
- To ensure that AI-generated content complies with legal, ethical, and content policy guidelines.
17
22
18
23
For more information on protected material detection, see the [Protected material detection concept page](./concepts/protected-material.md). For API input limits, see the [Input requirements](./overview.md#input-requirements) section of the Overview.
19
24
20
25
26
+
27
+
## User scenarios
28
+
### 1. Content generation platforms for creative writing
29
+
- Scenario: A content generation platform that uses GenAI for creative writing (e.g., blog posts, stories, marketing copy) integrates the "Protected Material for Text" feature to prevent the generation of content that closely matches known copyrighted material.
30
+
- User: Platform administrators and content creators.
31
+
- Action: The platform uses Azure AI Content Safety to scan AI-generated content before it is provided to users. If the generated text matches protected material, the content is flagged and either blocked or revised.
32
+
- Outcome: The platform avoids potential copyright infringements and ensures that all generated content is original and compliant with intellectual property laws.
33
+
### 2. Automated social media content creation
34
+
- Scenario: A digital marketing agency uses GenAI to automate social media content creation. The agency integrates the "Protected Material for Text" feature to avoid publishing AI-generated content that includes copyrighted text, such as song lyrics or excerpts from books.
35
+
- User: Digital marketers and social media managers.
36
+
- Action: The agency employs Azure AI Content Safety to check all AI-generated social media content for matches against a database of protected material. Content that matches is flagged for revision or blocked from posting.
37
+
- Outcome: The agency maintains compliance with copyright laws and avoids reputational risks associated with posting unauthorized content.
38
+
### 3. AI-assisted news writing
39
+
- Scenario: A news outlet uses GenAI to assist journalists in drafting articles and reports. To ensure the content does not unintentionally replicate protected news articles or other copyrighted material, the outlet uses the "Protected Material for Text" feature.
40
+
- User: Journalists, editors, and compliance officers.
41
+
- Action: The news outlet integrates Azure AI Content Safety into its content creation workflow. AI-generated drafts are automatically scanned for protected content before submission for editorial review.
42
+
- Outcome: The news outlet prevents accidental copyright violations and maintains the integrity and originality of its reporting.
43
+
### 4. E-learning platforms using AI for content generation
44
+
- Scenario: An e-learning platform employs GenAI to generate educational content, such as summaries, quizzes, and explanatory text. The platform uses the "Protected Material for Text" feature to ensure the generated content does not include protected material from textbooks, articles, or academic papers.
45
+
- User: Educational content creators and compliance officers.
46
+
- Action: The platform integrates the feature to scan AI-generated educational materials. If any content matches known protected academic material, it is flagged for revision or automatically removed.
47
+
- Outcome: The platform maintains educational content quality and complies with copyright laws, avoiding the use of protected material in AI-generated learning resources.
48
+
### 5. AI-powered recipe generators
49
+
- Scenario: A food and recipe website uses GenAI to generate new recipes based on user preferences. To avoid generating content that matches protected recipes from famous cookbooks or websites, the website integrates the "Protected Material for Text" feature.
50
+
- User: Content managers and platform administrators.
51
+
- Action: The website uses Azure AI Content Safety to check AI-generated recipes against a database of known protected content. If a generated recipe matches a protected one, it is flagged and revised or blocked.
52
+
- Outcome: The website ensures that all AI-generated recipes are original, reducing the risk of copyright infringement.
53
+
21
54
## Prerequisites
22
55
23
56
* An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/)
@@ -36,7 +69,7 @@ The following section walks through a sample request with cURL. Paste the comman
36
69
> See [Input requirements](./overview.md#input-requirements) for maximum text length limitations. Protected material detection is meant to be run on LLM completions, not user prompts.
37
70
38
71
```shell
39
-
curl --location --request POST '<endpoint>/contentsafety/text:detectProtectedMaterial?api-version=2023-10-15-preview' \
72
+
curl --location --request POST '<endpoint>/contentsafety/text:detectProtectedMaterial?api-version=2024-09-01' \
@@ -47,7 +80,7 @@ The below fields must be included in the url:
47
80
48
81
| Name |Required | Description | Type |
49
82
| :------- |-------- |:--------------- | ------ |
50
-
|**API Version**|Required |This is the API version to be checked. The current version is: api-version=2023-10-15-preview. Example: `<endpoint>/contentsafety/text:detectProtectedMaterial?api-version=2023-10-15-preview`|String |
83
+
|**API Version**|Required |This is the API version to be checked. The current version is: api-version=2024-09-01. Example: `<endpoint>/contentsafety/text:detectProtectedMaterial?api-version=2024-09-01`|String |
51
84
52
85
The parameters in the request body are defined in this table:
53
86
@@ -95,4 +128,3 @@ If you want to clean up and remove an Azure AI services subscription, you can de
95
128
96
129
*[Protected material detection concepts](./concepts/protected-material.md)
97
130
* Configure filters for each category and test on datasets using [Content Safety Studio](studio-quickstart.md), export the code and deploy.
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-deploy-online-endpoints.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -854,7 +854,7 @@ Next, deploy your online endpoint to Azure.
854
854
1. Create the deployment named `blue` under the endpoint.
855
855
856
856
```azurecli
857
-
az ml online-deployment create -name blue --endpoint $ENDPOINT_NAME -f endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml --all-traffic
857
+
az ml online-deployment create --name blue --endpoint $ENDPOINT_NAME -f endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml --all-traffic
858
858
```
859
859
860
860
The deployment creation can take up to 15 minutes, depending on whether the underlying environment or image is being built for the first time. Subsequent deployments that use the same environment are processed faster.
Copy file name to clipboardExpand all lines: articles/search/search-capacity-planning.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -144,7 +144,7 @@ Basic search services have lower search unit counts.
144
144
145
145
+ On search services created before April 3, 2024, a basic search service can have exactly one partition and up to three replicas, for a maximum limit of three SUs. The only adjustable resource is replicas.
146
146
147
-
+ On search services created after April 3, 2024 in [supported regions](search-limits-quotas-capacity.md#supported-regions-with-higher-storage-limits), basic services can have up to three partitions and three replicas. The maximum SU limit is nine to support a full complement of partitions and replicas.
147
+
+ On search services created after April 3, 2024 in [supported regions](search-limits-quotas-capacity.md#service-limits), basic services can have up to three partitions and three replicas. The maximum SU limit is nine to support a full complement of partitions and replicas.
148
148
149
149
For search services on any billable tier, regardless of creation date, you need a minimum of two replicas for high availability on queries.
Copy file name to clipboardExpand all lines: articles/search/search-limits-quotas-capacity.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -91,7 +91,7 @@ Vector limits vary by service creation date and tier. To check the age of your s
91
91
92
92
### Vector limits on services created after May 17, 2024
93
93
94
-
The highest vector limits are available on search services created after May 17, 2024 in a [supported region](#supported-regions-with-higher-storage-limits).
94
+
The highest vector limits are available on search services created after May 17, 2024 in a [supported region](#service-limits).
@@ -104,7 +104,7 @@ The highest vector limits are available on search services created after May 17,
104
104
105
105
### Vector limits on services created between April 3, 2024 and May 17, 2024
106
106
107
-
The following vector limits are available on search services created after April 3, 2024 in a [supported region](#supported-regions-with-higher-storage-limits).
107
+
The following vector limits are available on search services created after April 3, 2024 in a [supported region](#service-limits).
Copy file name to clipboardExpand all lines: articles/search/search-performance-tips.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,7 +103,7 @@ The tier of your search service and the number of replicas/partitions also have
103
103
104
104
### Tip: Create a new high capacity search service
105
105
106
-
Basic and standard services created [in supported regions]([supported regions](search-limits-quotas-capacity.md#supported-regions-with-higher-storage-limits) after April 3, 2024 have more storage per partition than older services. Before upgrading to a higher tier and a higher billable rate, revisit the [tier service limits](search-limits-quotas-capacity.md#service-limits) to see if the same tier on a newer service gives you the necessary storage.
106
+
Basic and standard services created [in supported regions]([supported regions](search-limits-quotas-capacity.md#service-limits) after April 3, 2024 have more storage per partition than older services. Before upgrading to a higher tier and a higher billable rate, revisit the [tier service limits](search-limits-quotas-capacity.md#service-limits) to see if the same tier on a newer service gives you the necessary storage.
Copy file name to clipboardExpand all lines: articles/search/search-sku-manage-costs.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,7 +72,7 @@ Cost management is built into the Azure infrastructure. Review [Billing and cost
72
72
73
73
Follow these guidelines to minimize costs of an Azure AI Search solution.
74
74
75
-
1. If possible, create a search service [in a region that has more storage per partition]([supported regions](search-limits-quotas-capacity.md#supported-regions-with-higher-storage-limits). If you're using multiple Azure resources in your solution, create them in the same region, or in as few regions as possible, to minimize or eliminate bandwidth charges.
75
+
1. If possible, create a search service [in a region that has more storage per partition]([supported regions](search-limits-quotas-capacity.md#service-limits). If you're using multiple Azure resources in your solution, create them in the same region, or in as few regions as possible, to minimize or eliminate bandwidth charges.
76
76
77
77
1.[Scale up](search-capacity-planning.md) for resource-intensive operations like indexing, and then readjust downwards for regular query workloads. If there are predictable patterns to your workloads, you might be able to synchronize scale up to coincide with the expected volume (you would need to write code to automate this).
0 commit comments