You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/computer-vision/concept-face-recognition.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,7 @@
1
1
---
2
2
title: "Face recognition - Face"
3
3
titleSuffix: Azure AI services
4
-
description: Learn the concept of Face recognition, its related operations, and the underlying data structures.
5
-
#services: cognitive-services
4
+
description: Learn the concept of Face recognition, its operations, and data structures, including PersonGroup creation, identification, and verification.
6
5
author: PatrickFarley
7
6
manager: nitinme
8
7
@@ -11,7 +10,7 @@ ms.subservice: azure-ai-face
11
10
ms.custom:
12
11
- ignite-2023
13
12
ms.topic: conceptual
14
-
ms.date: 02/14/2024
13
+
ms.date: 10/16/2024
15
14
ms.author: pafarley
16
15
---
17
16
@@ -50,7 +49,7 @@ Use the following tips to ensure that your input images give the most accurate r
* You can utilize the `qualityForRecognition` attribute in the [face detection](./how-to/identity-detect-faces.md) operation when using applicable detection models as a general guideline of whether the image is likely of sufficient quality to attempt face recognition on. Only `"high"` quality images are recommended for person enrollment and quality at or above `"medium"` is recommended for identification scenarios.
52
+
* You can use the `qualityForRecognition` attribute in the [face detection](./how-to/identity-detect-faces.md) operation when using applicable detection models as a general guideline of whether the image is likely of sufficient quality to attempt face recognition on. Only `"high"` quality images are recommended for person enrollment and quality at or above `"medium"` is recommended for identification scenarios.
#Customer intent: As a developer of content management software, I want to find out whether Azure Content Moderator is the right solution for my moderation needs.
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/groundedness.md
+10-12Lines changed: 10 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,15 +7,14 @@ author: PatrickFarley
7
7
manager: nitinme
8
8
ms.service: azure-ai-content-safety
9
9
ms.topic: conceptual
10
-
ms.date: 03/15/2024
10
+
ms.date: 10/16/2024
11
11
ms.author: pafarley
12
12
---
13
13
14
14
# Groundedness detection
15
15
16
16
The Groundedness detection API detects whether the text responses of large language models (LLMs) are grounded in the source materials provided by the users. Ungroundedness refers to instances where the LLMs produce information that is non-factual or inaccurate from what was present in the source materials.
17
17
18
-
19
18
## Key terms
20
19
21
20
-**Retrieval Augmented Generation (RAG)**: RAG is a technique for augmenting LLM knowledge with other data. LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data that was available at the time they were trained. If you want to build AI applications that can reason about private data or data introduced after a model’s cutoff date, you need to provide the model with that specific information. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG). For more information, see [Retrieval-augmented generation (RAG)](https://python.langchain.com/docs/tutorials/rag/).
@@ -49,11 +48,11 @@ Groundedness detection supports text-based Summarization and QnA tasks to ensure
49
48
50
49
The groundedness detection API includes a correction feature that automatically corrects any detected ungroundedness in the text based on the provided grounding sources. When the correction feature is enabled, the response includes a `corrected Text` field that presents the corrected text aligned with the grounding sources.
51
50
52
-
Below, see several common scenarios that illustrate how and when to apply these features to achieve the best outcomes.
51
+
### Use cases
53
52
53
+
Below, see several common scenarios that illustrate how and when to apply these features to achieve the best outcomes.
54
54
55
-
### Summarization in medical contexts
56
-
**Use case:**
55
+
#### Summarization in medical contexts
57
56
58
57
You're summarizing medical documents, and it’s critical that the names of patients in the summaries are accurate and consistent with the provided grounding sources.
59
58
@@ -74,8 +73,7 @@ Example API Request:
74
73
75
74
The correction feature detects that `Kevin` is ungrounded because it conflicts with the grounding source `Jane`. The API returns the corrected text: `"The patient name is Jane."`
76
75
77
-
### Question and answer (QnA) task with customer support data
78
-
**Use case:**
76
+
#### Question and answer (QnA) task with customer support data
79
77
80
78
You're implementing a QnA system for a customer support chatbot. It’s essential that the answers provided by the AI align with the most recent and accurate information available.
81
79
@@ -99,8 +97,8 @@ Example API Request:
99
97
The API detects that `5%` is ungrounded because it does not match the provided grounding source `4.5%`. The response includes the correction text: `"The interest rate is 4.5%."`
100
98
101
99
102
-
### Content creation with historical data
103
-
**Use case**:
100
+
####Content creation with historical data
101
+
104
102
You're creating content that involves historical data or events, where accuracy is critical to maintaining credibility and avoiding misinformation.
105
103
106
104
Example API Request:
@@ -116,11 +114,11 @@ Example API Request:
116
114
}
117
115
```
118
116
**Expected outcome:**
117
+
119
118
The API detects the ungrounded date `1065` and corrects it to `1066` based on the grounding source. The response includes the corrected text: `"The Battle of Hastings occurred in 1066."`
120
119
121
120
122
-
### Internal documentation summarization
123
-
**Use case:**
121
+
#### Internal documentation summarization
124
122
125
123
You're summarizing internal documents where product names, version numbers, or other specific data points must remain consistent.
126
124
@@ -159,7 +157,7 @@ Currently, the Groundedness detection API supports English language content. Whi
159
157
160
158
See [Input requirements](../overview.md#input-requirements) for maximum text length limitations.
161
159
162
-
### Regions
160
+
### Region availability
163
161
164
162
To use this API, you must create your Azure AI Content Safety resource in the supported regions. See [Region availability](/azure/ai-services/content-safety/overview#region-availability).
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/harm-categories.md
+5-6Lines changed: 5 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,13 +2,12 @@
2
2
title: "Harm categories in Azure AI Content Safety"
3
3
titleSuffix: Azure AI services
4
4
description: Learn about the different content moderation flags and severity levels that the Azure AI Content Safety service returns.
5
-
#services: cognitive-services
6
5
author: PatrickFarley
7
6
manager: nitinme
8
7
ms.service: azure-ai-content-safety
9
8
ms.custom: build-2023
10
9
ms.topic: conceptual
11
-
ms.date: 01/20/2024
10
+
ms.date: 10/16/2024
12
11
ms.author: pafarley
13
12
---
14
13
@@ -23,9 +22,9 @@ Content Safety recognizes four distinct categories of objectionable content.
23
22
24
23
| Category | Description |API term |
25
24
| --------- | ------------------- | -- |
26
-
| Hate and Fairness | Hate and fairness-related harms refer to any content that attacks or uses discriminatory language with reference to a person or Identity group based on certain differentiating attributes of these groups. <br><br>This includes, but is not limited to:<ul><li>Race, ethnicity, nationality</li><li>Gender identity groups and expression</li><li>Sexual orientation</li><li>Religion</li><li>Personal appearance and body size</li><li>Disability status</li><li>Harassment and bullying</li></ul> |`Hate`|
25
+
| Hate and Fairness | Hate and fairness harms refer to any content that attacks or uses discriminatory language with reference to a person or identity group based on certain differentiating attributes of these groups. <br><br>This includes, but is not limited to:<ul><li>Race, ethnicity, nationality</li><li>Gender identity groups and expression</li><li>Sexual orientation</li><li>Religion</li><li>Personal appearance and body size</li><li>Disability status</li><li>Harassment and bullying</li></ul> |`Hate`|
27
26
| Sexual | Sexual describes language related to anatomical organs and genitals, romantic relationships and sexual acts, acts portrayed in erotic or affectionate terms, including those portrayed as an assault or a forced sexual violent act against one’s will. <br><br> This includes but is not limited to:<ul><li>Vulgar content</li><li>Prostitution</li><li>Nudity and Pornography</li><li>Abuse</li><li>Child exploitation, child abuse, child grooming</li></ul> |`Sexual`|
28
-
| Violence | Violence describes language related to physical actions intended to hurt, injure, damage, or kill someone or something; describes weapons, guns and related entities. <br><br>This includes, but isn't limited to: <ul><li>Weapons</li><li>Bullying and intimidation</li><li>Terrorist and violent extremism</li><li>Stalking</li></ul> |`Violence`|
27
+
| Violence | Violence describes language related to physical actions intended to hurt, injure, damage, or kill someone or something; describes weapons, guns, and related entities. <br><br>This includes, but isn't limited to: <ul><li>Weapons</li><li>Bullying and intimidation</li><li>Terrorist and violent extremism</li><li>Stalking</li></ul> |`Violence`|
29
28
| Self-Harm | Self-harm describes language related to physical actions intended to purposely hurt, injure, damage one’s body or kill oneself. <br><br> This includes, but isn't limited to: <ul><li>Eating Disorders</li><li>Bullying and intimidation</li></ul> |`SelfHarm`|
30
29
31
30
Classification can be multi-labeled. For example, when a text sample goes through the text moderation model, it could be classified as both Sexual content and Violence.
@@ -34,7 +33,7 @@ Classification can be multi-labeled. For example, when a text sample goes throug
34
33
35
34
Every harm category the service applies also comes with a severity level rating. The severity level is meant to indicate the severity of the consequences of showing the flagged content.
36
35
37
-
**Text**: The current version of the text model supports the full 0-7 severity scale. The classifier detects amongst all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
36
+
**Text**: The current version of the text model supports the full 0-7 severity scale. The classifier detects among all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
38
37
-`[0,1]` -> `0`
39
38
-`[2,3]` -> `2`
40
39
-`[4,5]` -> `4`
@@ -46,7 +45,7 @@ Every harm category the service applies also comes with a severity level rating.
46
45
-`4`
47
46
-`6`
48
47
49
-
**Image with text**: The current version of the multimodal model supports the full 0-7 severity scale. The classifier detects amongst all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
48
+
**Image with text**: The current version of the multimodal model supports the full 0-7 severity scale. The classifier detects among all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/jailbreak-detection.md
+28-1Lines changed: 28 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ manager: nitinme
8
8
ms.service: azure-ai-content-safety
9
9
ms.custom: build-2023
10
10
ms.topic: conceptual
11
-
ms.date: 09/25/2024
11
+
ms.date: 10/16/2024
12
12
ms.author: pafarley
13
13
---
14
14
@@ -19,6 +19,33 @@ Generative AI models can pose risks of being exploited by malicious actors. To m
19
19
Prompt Shields is a unified API that analyzes LLM inputs and detects adversarial user input attacks.
20
20
21
21
22
+
## User scenarios
23
+
### AI content creation platforms: Detecting harmful prompts
24
+
- Scenario: An AI content creation platform uses generative AI models to produce marketing copy, social media posts, and articles based on user-provided prompts. To prevent the generation of harmful or inappropriate content, the platform integrates "Prompt Shields."
25
+
- User: Content creators, platform administrators, and compliance officers.
26
+
- Action: The platform uses Azure AI Content Safety's "Prompt Shields" to analyze user prompts before generating content. If a prompt is detected as potentially harmful or likely to lead to policy-violating outputs (e.g., prompts asking for defamatory content or hate speech), the shield blocks the prompt and alerts the user to modify their input.
27
+
- Outcome: The platform ensures all AI-generated content is safe, ethical, and compliant with community guidelines, enhancing user trust and protecting the platform's reputation.
28
+
### AI-powered chatbots: Mitigating risk from user prompt attacks
29
+
- Scenario: A customer service provider uses AI-powered chatbots for automated support. To safeguard against user prompts that could lead the AI to generate inappropriate or unsafe responses, the provider uses "Prompt Shields."
30
+
- User: Customer service agents, chatbot developers, and compliance teams.
31
+
- Action: The chatbot system integrates "Prompt Shields" to monitor and evaluate user inputs in real-time. If a user prompt is identified as potentially harmful or designed to exploit the AI (e.g., attempting to provoke inappropriate responses or extract sensitive information), the shield intervenes by blocking the response or redirecting the query to a human agent.
32
+
- Outcome: The customer service provider maintains high standards of interaction safety and compliance, preventing the chatbot from generating responses that could harm users or breach policies.
- Scenario: An e-learning platform employs GenAI to generate personalized educational content based on student inputs and reference documents. To avoid generating inappropriate or misleading educational content, the platform utilizes "Prompt Shields."
35
+
- User: Educators, content developers, and compliance officers.
36
+
- Action: The platform uses "Prompt Shields" to analyze both user prompts and uploaded documents for content that could lead to unsafe or policy-violating AI outputs. If a prompt or document is detected as likely to generate inappropriate educational content, the shield blocks it and suggests alternative, safe inputs.
37
+
- Outcome: The platform ensures that all AI-generated educational materials are appropriate and compliant with academic standards, fostering a safe and effective learning environment.
38
+
### Healthcare AI assistants: Blocking unsafe prompts and document inputs
39
+
- Scenario: A healthcare provider uses AI assistants to offer preliminary medical advice based on user inputs and uploaded medical documents. To ensure the AI does not generate unsafe or misleading medical advice, the provider implements "Prompt Shields."
40
+
- User: Healthcare providers, AI developers, and compliance teams.
41
+
- Action: The AI assistant employs "Prompt Shields" to analyze patient prompts and uploaded medical documents for harmful or misleading content. If a prompt or document is identified as potentially leading to unsafe medical advice, the shield prevents the AI from generating a response and redirects the patient to a human healthcare professional.
42
+
- Outcome: The healthcare provider ensures that AI-generated medical advice remains safe and accurate, protecting patient safety and maintaining compliance with healthcare regulations.
43
+
### Generative AI for creative writing: Protecting against prompt manipulation
44
+
- Scenario: A creative writing platform uses GenAI to assist writers in generating stories, poetry, and scripts based on user inputs. To prevent the generation of inappropriate or offensive content, the platform incorporates "Prompt Shields."
45
+
- User: Writers, platform moderators, and content reviewers.
46
+
- Action: The platform integrates "Prompt Shields" to evaluate user prompts for creative writing. If a prompt is detected as likely to produce offensive, defamatory, or otherwise inappropriate content, the shield blocks the AI from generating such content and suggests revisions to the user.
47
+
48
+
22
49
## Types of input attacks
23
50
24
51
The types of input attacks that Prompt Shields detects are described in this table.
0 commit comments