You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/Content-Moderator/text-moderation-api.md
+33-60Lines changed: 33 additions & 60 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ ms.author: pafarley
16
16
17
17
# Learn text moderation concepts
18
18
19
-
Use Content Moderator’s machine-assisted text moderation and [human review](Review-Tool-User-Guide/human-in-the-loop.md) capabilities to moderate text content.
19
+
Use Content Moderator's machine-assisted text moderation and [human review](Review-Tool-User-Guide/human-in-the-loop.md) capabilities to moderate text content.
20
20
21
21
You either block, approve or review the content based on your policies and thresholds. Use it to augment human moderation of environments where partners, employees and consumers generate text content. These include chat rooms, discussion boards, chatbots, e-commerce catalogs, and documents.
22
22
@@ -33,13 +33,13 @@ The service response includes the following information:
33
33
34
34
If the API detects any profane terms in any of the [supported languages](Text-Moderation-API-Languages.md), those terms are included in the response. The response also contains their location (`Index`) in the original text. The `ListId` in the following sample JSON refers to terms found in [custom term lists](try-terms-list-api.md) if available.
35
35
36
-
"Terms": [
37
-
{
38
-
"Index": 118,
39
-
"OriginalIndex": 118,
40
-
"ListId": 0,
41
-
"Term": "crap"
42
-
}
36
+
"Terms": [
37
+
{
38
+
"Index": 118,
39
+
"OriginalIndex": 118,
40
+
"ListId": 0,
41
+
"Term": "crap"
42
+
}
43
43
44
44
> [!NOTE]
45
45
> For the **language** parameter, assign `eng` or leave it empty to see the machine-assisted **classification** response (preview feature). **This feature supports English only**.
@@ -48,22 +48,22 @@ If the API detects any profane terms in any of the [supported languages](Text-Mo
48
48
49
49
## Classification
50
50
51
-
Content Moderator’s machine-assisted **text classification feature** supports **English only**, and helps detect potentially undesired content. The flagged content may be assessed as inappropriate depending on context. It conveys the likelihood of each category and may recommend a human review. The feature uses a trained model to identify possible abusive, derogatory or discriminatory language. This includes slang, abbreviated words, offensive, and intentionally misspelled words for review.
51
+
Content Moderator's machine-assisted **text classification feature** supports **English only**, and helps detect potentially undesired content. The flagged content may be assessed as inappropriate depending on context. It conveys the likelihood of each category and may recommend a human review. The feature uses a trained model to identify possible abusive, derogatory or discriminatory language. This includes slang, abbreviated words, offensive, and intentionally misspelled words for review.
52
52
53
53
The following extract in the JSON extract shows an example output:
54
54
55
-
"Classification": {
56
-
"ReviewRecommended": true,
57
-
"Category1": {
58
-
"Score": 1.5113095059859916E-06
59
-
},
60
-
"Category2": {
61
-
"Score": 0.12747249007225037
62
-
},
63
-
"Category3": {
64
-
"Score": 0.98799997568130493
65
-
}
66
-
}
55
+
"Classification": {
56
+
"ReviewRecommended": true,
57
+
"Category1": {
58
+
"Score": 1.5113095059859916E-06
59
+
},
60
+
"Category2": {
61
+
"Score": 0.12747249007225037
62
+
},
63
+
"Category3": {
64
+
"Score": 0.98799997568130493
65
+
}
66
+
}
67
67
68
68
### Explanation
69
69
@@ -78,11 +78,9 @@ The following extract in the JSON extract shows an example output:
78
78
The personal data feature detects the potential presence of this information:
79
79
80
80
- Email address
81
-
- US Mailing address
81
+
- US mailing address
82
82
- IP address
83
-
- US Phone number
84
-
- UK Phone number
85
-
- Social Security Number (SSN)
83
+
- US phone number
86
84
87
85
The following example shows a sample response:
88
86
@@ -113,51 +111,26 @@ The following example shows a sample response:
113
111
"CountryCode":"US",
114
112
"Text":"425 555 0111",
115
113
"Index":212
116
-
},
117
-
{
118
-
"CountryCode":"UK",
119
-
"Text":"+123 456 7890",
120
-
"Index":208
121
-
},
122
-
{
123
-
"CountryCode":"UK",
124
-
"Text":"0234 567 8901",
125
-
"Index":228
126
-
},
127
-
{
128
-
"CountryCode":"UK",
129
-
"Text":"0456 789 0123",
130
-
"Index":245
131
114
}
132
115
],
133
116
"Address":[
134
117
{
135
118
"Text":"1234 Main Boulevard, Panapolis WA 96555",
136
119
"Index":89
137
120
}
138
-
],
139
-
"SSN":[
140
-
{
141
-
"Text":"999999999",
142
-
"Index":56
143
-
},
144
-
{
145
-
"Text":"999-99-9999",
146
-
"Index":267
147
-
}
148
121
]
149
122
}
150
123
```
151
124
152
125
## Auto-correction
153
126
154
-
Suppose the input text is (the ‘lzay’ and 'f0x' are intentional):
127
+
Suppose the input text is (the 'lzay' and 'f0x' are intentional):
155
128
156
-
The qu!ck brown f0x jumps over the lzay dog.
129
+
The qu!ck brown f0x jumps over the lzay dog.
157
130
158
131
If you ask for auto-correction, the response contains the corrected version of the text:
159
132
160
-
The quick brown fox jumps over the lazy dog.
133
+
The quick brown fox jumps over the lazy dog.
161
134
162
135
## Creating and managing your custom lists of terms
163
136
@@ -169,13 +142,13 @@ While the default, global list of terms works great for most cases, you may want
169
142
170
143
The following example shows the matching List ID:
171
144
172
-
"Terms": [
173
-
{
174
-
"Index": 118,
175
-
"OriginalIndex": 118,
176
-
"ListId": 231.
177
-
"Term": "crap"
178
-
}
145
+
"Terms": [
146
+
{
147
+
"Index": 118,
148
+
"OriginalIndex": 118,
149
+
"ListId": 231.
150
+
"Term": "crap"
151
+
}
179
152
180
153
The Content Moderator provides a [Term List API](https://westus.dev.cognitive.microsoft.com/docs/services/57cf755e3f9b070c105bd2c2/operations/57cf755e3f9b070868a1f67f) with operations for managing custom term lists. Start with the [Term Lists API Console](try-terms-list-api.md) and use the REST API code samples. Also check out the [Term Lists .NET quickstart](term-lists-quickstart-dotnet.md) if you are familiar with Visual Studio and C#.
Copy file name to clipboardExpand all lines: articles/cognitive-services/Content-Moderator/try-text-api.md
+3-28Lines changed: 3 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -59,7 +59,7 @@ For **Content-Type**, select the type of content you want to screen. For this ex
59
59
In the **Request body** box, enter some text. The following example shows an intentional typo in the text.
60
60
61
61
```
62
-
Is this a grabage or crap email [email protected], phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555. These are all UK phone numbers, the last two being Microsoft UK support numbers: +44 870 608 4000 or 0344 800 2400 or 0800 820 3300. Also, 999-99-9999 looks like a social security number (SSN).
62
+
Is this a grabage or crap email [email protected], phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555.
63
63
```
64
64
65
65
## Analyze the response
@@ -70,8 +70,8 @@ The following response shows the various insights from the API. It contains pote
70
70
> The machine-assisted 'Classification' feature is in preview and supports English only.
71
71
72
72
```json
73
-
{"OriginalText":"Is this a grabage or crap email [email protected], phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555.\r\nThese are all UK phone numbers: +44 123 456 7890 or 0234 567 8901 or 0456 789 0123.\r\nAlso, 999-99-9999 looks like a social security number (SSN).",
74
-
"NormalizedText":"Is this a grabage or crap email abcdef@ abcd. com, phone: 4255550111, IP: 255. 255. 255. 255, 1234 Main Boulevard, Panapolis WA 96555.\r\nThese are all UK phone numbers: +44 123 456 7890 or 0234 567 8901 or 0456 789 0123. \r\nAlso, 999- 99- 9999 looks like a social security number ( SSN) .",
73
+
{"OriginalText":"Is this a grabage or crap email [email protected], phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555.",
74
+
"NormalizedText":"Is this a grabage or crap email abcdef@ abcd. com, phone: 4255550111, IP: 255. 255. 255. 255, 1234 Main Boulevard, Panapolis WA 96555.",
75
75
"Misrepresentation":null,
76
76
"PII":{
77
77
"Email":[
@@ -99,38 +99,13 @@ The following response shows the various insights from the API. It contains pote
0 commit comments