You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Azure AI Content Safety custom category feature uses a multi-step process for creating, training, and using custom content classification models. Here's a look at the workflow:
54
+
The Azure AI Content Safety custom categories feature uses a multi-step process for creating, training, and using custom content classification models. Here's a look at the workflow:
55
55
56
56
### Step 1: Definition and setup
57
57
@@ -73,7 +73,7 @@ You use the **analyzeCustomCategory** API to analyze text content and determine
73
73
74
74
#### [Custom categories (rapid) API](#tab/rapid)
75
75
76
-
To use the custom category (rapid) API, you first create an **incident** object with a text description. Then, you upload any number of image or text samples to the incident. The LLM on the backend will then use these to evaluate future input content. No training step is needed.
76
+
To use the custom categories (rapid) API, you first create an **incident** object with a text description. Then, you upload any number of image or text samples to the incident. The LLM on the backend will then use these to evaluate future input content. No training step is needed.
77
77
78
78
You can include your defined incident in a regular text analysis or image analysis request. The service will indicate whether the submitted content is an instance of your incident. The service can still do other content moderation tasks in the same API call.
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/groundedness.md
+111-5Lines changed: 111 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,13 +18,14 @@ The Groundedness detection API detects whether the text responses of large langu
18
18
19
19
## Key terms
20
20
21
-
-**Retrieval Augmented Generation (RAG)**: RAG is a technique for augmenting LLM knowledge with other data. LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data that was available at the time they were trained. If you want to build AI applications that can reason about private data or data introduced after a model’s cutoff date, you need to provide the model with that specific information. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG). For more information, see [Retrieval-augmented generation (RAG)](https://python.langchain.com/docs/use_cases/question_answering/).
21
+
-**Retrieval Augmented Generation (RAG)**: RAG is a technique for augmenting LLM knowledge with other data. LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data that was available at the time they were trained. If you want to build AI applications that can reason about private data or data introduced after a model’s cutoff date, you need to provide the model with that specific information. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG). For more information, see [Retrieval-augmented generation (RAG)](https://python.langchain.com/docs/tutorials/rag/).
22
+
-**Groundedness and Ungroundedness in LLMs**: This refers to the extent to which the model's outputs are based on provided information or reflect reliable sources accurately. A grounded response adheres closely to the given information, avoiding speculation or fabrication. In groundedness measurements, source information is crucial and serves as the grounding source.
22
23
23
-
-**Groundedness and Ungroundedness in LLMs**: This refers to the extent to which the model’s outputs are based on provided information or reflect reliable sources accurately. A grounded response adheres closely to the given information, avoiding speculation or fabrication. In groundedness measurements, source information is crucial and serves as the grounding source.
24
+
## Groundedness detection options
24
25
25
-
## Groundedness detection features
26
+
The following options are available for gGroundedness detection in Azure AI Content Safety:
26
27
27
-
-**Domain Selection**: Users can choose an established domain to ensure more tailored detection that aligns with the specific needs of their field. Currently the available domains are `MEDICAL` and `GENERIC`.
28
+
-**Domain Selection**: Users can choose an established domain to ensure more tailored detection that aligns with the specific needs of their field. The current available domains are `MEDICAL` and `GENERIC`.
28
29
-**Task Specification**: This feature lets you select the task you're doing, such as QnA (question & answering) and Summarization, with adjustable settings according to the task type.
29
30
-**Speed vs Interpretability**: There are two modes that trade off speed with result interpretability.
30
31
- Non-Reasoning mode: Offers fast detection capability; easy to embed into online applications.
@@ -43,6 +44,111 @@ Groundedness detection supports text-based Summarization and QnA tasks to ensure
43
44
- Medical QnA: For medical QnA, the function helps verify the accuracy of medical answers and advice provided by AI systems to healthcare professionals and patients, reducing the risk of medical errors.
44
45
- Educational QnA: In educational settings, the function can be applied to QnA tasks to confirm that answers to academic questions or test prep queries are factually accurate, supporting the learning process.
45
46
47
+
48
+
## Groundedness correction
49
+
50
+
The groundedness detection API includes a correction feature that automatically corrects any detected ungroundedness in the text based on the provided grounding sources. When the correction feature is enabled, the response includes a `corrected Text` field that presents the corrected text aligned with the grounding sources.
51
+
52
+
Below, see several common scenarios that illustrate how and when to apply these features to achieve the best outcomes.
53
+
54
+
55
+
### Summarization in medical contexts
56
+
**Use case:**
57
+
58
+
You're summarizing medical documents, and it’s critical that the names of patients in the summaries are accurate and consistent with the provided grounding sources.
59
+
60
+
Example API Request:
61
+
62
+
```json
63
+
{
64
+
"domain": "Medical",
65
+
"task": "Summarization",
66
+
"text": "The patient name is Kevin.",
67
+
"groundingSources": [
68
+
"The patient name is Jane."
69
+
],
70
+
}
71
+
```
72
+
73
+
**Expected outcome:**
74
+
75
+
The correction feature detects that `Kevin` is ungrounded because it conflicts with the grounding source `Jane`. The API returns the corrected text: `"The patient name is Jane."`
76
+
77
+
### Question and answer (QnA) task with customer support data
78
+
**Use case:**
79
+
80
+
You're implementing a QnA system for a customer support chatbot. It’s essential that the answers provided by the AI align with the most recent and accurate information available.
81
+
82
+
Example API Request:
83
+
84
+
```json
85
+
{
86
+
"domain": "Generic",
87
+
"task": "QnA",
88
+
"qna": {
89
+
"query": "What is the current interest rate?"
90
+
},
91
+
"text": "The interest rate is 5%.",
92
+
"groundingSources": [
93
+
"As of July 2024, the interest rate is 4.5%."
94
+
],
95
+
}
96
+
```
97
+
**Expected outcome:**
98
+
99
+
The API detects that `5%` is ungrounded because it does not match the provided grounding source `4.5%`. The response includes the correction text: `"The interest rate is 4.5%."`
100
+
101
+
102
+
### Content creation with historical data
103
+
**Use case**:
104
+
You're creating content that involves historical data or events, where accuracy is critical to maintaining credibility and avoiding misinformation.
105
+
106
+
Example API Request:
107
+
108
+
```json
109
+
{
110
+
"domain": "Generic",
111
+
"task": "Summarization",
112
+
"text": "The Battle of Hastings occurred in 1065.",
113
+
"groundingSources": [
114
+
"The Battle of Hastings occurred in 1066."
115
+
],
116
+
}
117
+
```
118
+
**Expected outcome:**
119
+
The API detects the ungrounded date `1065` and correct it to `1066` based on the grounding source. The response includes the corrected text: `"The Battle of Hastings occurred in 1066."`
120
+
121
+
122
+
### Internal documentation summarization
123
+
**Use case:**
124
+
125
+
You're summarizing internal documents where product names, version numbers, or other specific data points must remain consistent.
126
+
127
+
Example API Request:
128
+
129
+
```json
130
+
{
131
+
"domain": "Generic",
132
+
"task": "Summarization",
133
+
"text": "Our latest product is SuperWidget v2.1.",
134
+
"groundingSources": [
135
+
"Our latest product is SuperWidget v2.2."
136
+
],
137
+
}
138
+
```
139
+
140
+
**Expected outcome:**
141
+
142
+
The correction feature identifies `SuperWidget v2.1` as ungrounded and update it to `SuperWidget v2.2` in the response. The response returns the corrected text: `"Our latest product is SuperWidget v2.2."`
143
+
144
+
## Best practices
145
+
146
+
Adhere to the following best practices when setting up RAG systems to get the best performance out of the groundedness detection API:
147
+
- When dealing with product names or version numbers, use grounding sources directly from internal release notes or official product documentation to ensure accuracy.
148
+
- For historical content, cross-reference your grounding sources with trusted academic or historical databases to ensure the highest level of accuracy.
149
+
- In a dynamic environment like finance, always use the most recent and reliable grounding sources to ensure your AI system provides accurate and timely information.
150
+
- Always ensure that your grounding sources are accurate and up-to-date, particularly in sensitive fields like healthcare. This minimizes the risk of errors in the summarization process.
151
+
46
152
## Limitations
47
153
48
154
### Language availability
@@ -57,7 +163,7 @@ See [Input requirements](../overview.md#input-requirements) for maximum text len
57
163
58
164
To use this API, you must create your Azure AI Content Safety resource in the supported regions. See [Region availability](/azure/ai-services/content-safety/overview#region-availability).
59
165
60
-
### TPS limitations
166
+
### Rate limitations
61
167
62
168
See [Query rates](/azure/ai-services/content-safety/overview#query-rates).
Copy file name to clipboardExpand all lines: articles/ai-services/content-safety/concepts/harm-categories.md
+18-10Lines changed: 18 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,21 +35,29 @@ Classification can be multi-labeled. For example, when a text sample goes throug
35
35
Every harm category the service applies also comes with a severity level rating. The severity level is meant to indicate the severity of the consequences of showing the flagged content.
36
36
37
37
**Text**: The current version of the text model supports the full 0-7 severity scale. The classifier detects amongst all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
38
-
-[0,1] -> 0
39
-
-[2,3] -> 2
40
-
-[4,5] -> 4
41
-
-[6,7] -> 6
42
-
43
-
**Image**: The current version of the image model supports the trimmed version of the full 0-7 severity scale. The classifier only returns severities 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
44
-
-[0,1] -> 0
45
-
-[2,3] -> 2
46
-
-[4,5] -> 4
47
-
-[6,7] -> 6
38
+
-`[0,1]` -> `0`
39
+
-`[2,3]` -> `2`
40
+
-`[4,5]` -> `4`
41
+
-`[6,7]` -> `6`
42
+
43
+
**Image**: The current version of the image model supports the trimmed version of the full 0-7 severity scale. The classifier only returns severities 0, 2, 4, and 6.
44
+
-`0`
45
+
-`2`
46
+
-`4`
47
+
-`6`
48
+
49
+
**Image with text**: The current version of the multimodal model supports the full 0-7 severity scale. The classifier detects amongst all severities along this scale. If the user specifies, it can return severities in the trimmed scale of 0, 2, 4, and 6; each two adjacent levels are mapped to a single level.
0 commit comments