You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/concepts/use-asynchronously.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ manager: nitinme
8
8
ms.service: cognitive-services
9
9
ms.subservice: language-service
10
10
ms.topic: conceptual
11
-
ms.date: 06/28/2022
11
+
ms.date: 08/02/2022
12
12
ms.author: aahi
13
13
---
14
14
@@ -26,6 +26,7 @@ Currently, the following features are available to be used asynchronously:
26
26
* Customer content detection
27
27
* Sentiment analysis and opinion mining
28
28
* Text Analytics for health
29
+
* Personal Identifiable information (PII)
29
30
30
31
When you send asynchronous requests, you will incur charges based on number of text records you include in your request, for each feature use. For example, if you send a text record for sentiment analysis and NER, it will be counted as sending two text records, and you will be charged for both according to your [pricing tier](https://azure.microsoft.com/pricing/details/cognitive-services/language-service/).
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/personally-identifiable-information/concepts/entity-categories.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,9 +23,8 @@ The PII feature includes the ability to detect personal (`PII`) and health (`PHI
23
23
24
24
> [!NOTE]
25
25
> To detect protected health information (PHI), use the `domain=phi` parameter and model version `2020-04-01` or later.
26
-
>
27
-
> For example: `https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.1/entities/recognition/pii?domain=phi&model-version=2021-01-15`
28
-
26
+
27
+
29
28
The following entity categories are returned when you're sending API requests PII feature.
30
29
31
30
## Category: Person
@@ -440,7 +439,7 @@ The entity in this category can have the following subcategories.
440
439
441
440
### Azure information
442
441
443
-
These entity categories includes identifiable Azure information, including authentication information and connection strings. Not returned as PHI.
442
+
These entity categories include identifiable Azure information like authentication information and connection strings. Not returned as PHI.
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/personally-identifiable-information/how-to-call-for-conversations.md
+7-11Lines changed: 7 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ You can submit the input to the API as list of conversation items. Analysis is p
39
39
40
40
When using the async feature, the API results are available for 24 hours from the time the request was ingested, and is indicated in the response. After this time period, the results are purged and are no longer available for retrieval.
41
41
42
-
When you submit data to conversational PII, we can send one conversation (chat or spoken) per request.
42
+
When you submit data to conversational PII, you can send one conversation (chat or spoken) per request.
43
43
44
44
The API will attempt to detect all the [defined entity categories](concepts/conversations-entity-categories.md) for a given conversation input. If you want to specify which entities will be detected and returned, use the optional `piiCategories` parameter with the appropriate entity categories.
45
45
@@ -62,22 +62,18 @@ When you get results from PII detection, you can stream the results to an applic
# How to detect and redact Personally Identifying Information (PII)
18
18
19
-
The PII feature can evaluate unstructured text, extract sensitive information (PII) and health information (PHI) in text across several pre-defined categories.
19
+
The PII feature can evaluate unstructured text, extract and redact sensitive information (PII) and health information (PHI) in text across several [pre-defined categories](concepts/entity-categories.md).
20
20
21
21
## Determine how to process the data (optional)
22
22
@@ -26,20 +26,82 @@ By default, this feature will use the latest available AI model on your text. Yo
26
26
27
27
### Input languages
28
28
29
-
When you submit documents to be processed, you can specify which of [the supported languages](language-support.md) they're written in. if you don't specify a language, key phrase extraction will default to English. The API may return offsets in the response to support different [multilingual and emoji encodings](../concepts/multilingual-emoji-support.md).
29
+
When you submit documents to be processed, you can specify which of [the supported languages](language-support.md) they're written in. if you don't specify a language, extraction will default to English. The API may return offsets in the response to support different [multilingual and emoji encodings](../concepts/multilingual-emoji-support.md).
30
30
31
31
## Submitting data
32
32
33
33
Analysis is performed upon receipt of the request. Using the PII detection feature synchronously is stateless. No data is stored in your account, and results are returned immediately in the response.
The API will attempt to detect the [defined entity categories](concepts/entity-categories.md) for a given document language. If you want to specify which entities will be detected and returned, use the optional `piiCategories` parameter with the appropriate entity categories. This parameter can also let you detect entities that aren't enabled by default for your document language. The following URL example would detect a French driver's license number that might occur in English text, along with the default English entities.
37
+
## Select which entities to be returned
38
+
39
+
The API will attempt to detect the [defined entity categories](concepts/entity-categories.md) for a given document language. If you want to specify which entities will be detected and returned, use the optional `piiCategories` parameter with the appropriate entity categories. This parameter can also let you detect entities that aren't enabled by default for your document language. The following example would detect only `Person`. You can specify one or more [entity types](concepts/entity-categories.md) to be returned.
38
40
39
41
> [!TIP]
40
42
> If you don't include `default` when specifying entity categories, The API will only return the entity categories you specify.
"text": "We went to Contoso foodplace located at downtown Seattle last week for a dinner party, and we adore the spot! They provide marvelous food and they have a great menu. The chief cook happens to be the owner (I think his name is John Doe) and he is super nice, coming out of the kitchen and greeted us all. We enjoyed very much dining in the place! The pasta I ordered was tender and juicy, and the place was impeccably clean. You can even pre-order from their online menu at www.contosofoodplace.com, call 112-555-0176 or send email to [email protected]! The only complaint I have is the food didn't come fast enough. Overall I highly recommend it!"
70
+
}
71
+
]
72
+
}
73
+
}
74
+
75
+
```
76
+
77
+
**Output:**
78
+
79
+
```bash
80
+
81
+
{
82
+
"kind": "PiiEntityRecognitionResults",
83
+
"results": {
84
+
"documents": [
85
+
{
86
+
"redactedText": "We went to Contoso foodplace located at downtown Seattle last week for a dinner party, and we adore the spot! They provide marvelous food and they have a great menu. The chief cook happens to be the owner (I think his name is ********) and he is super nice, coming out of the kitchen and greeted us all. We enjoyed very much dining in the place! The pasta I ordered was tender and juicy, and the place was impeccably clean. You can even pre-order from their online menu at www.contosofoodplace.com, call 112-555-0176 or send email to [email protected]! The only complaint I have is the food didn't come fast enough. Overall I highly recommend it!",
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/personally-identifiable-information/language-support.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,14 +8,14 @@ manager: nitinme
8
8
ms.service: cognitive-services
9
9
ms.subservice: language-service
10
10
ms.topic: conceptual
11
-
ms.date: 11/02/2021
11
+
ms.date: 08/02/2022
12
12
ms.author: aahi
13
13
ms.custom: language-service-pii, ignite-fall-2021
14
14
---
15
15
16
16
# Personally Identifiable Information (PII) detection language support
17
17
18
-
Use this article to learn which natural languages are supported by the PII feature of Azure Cognitive Service for Language.
18
+
Use this article to learn which natural languages are supported by the PII and conversation PII (preview) features of Azure Cognitive Service for Language.
19
19
20
20
> [!NOTE]
21
21
> * Languages are added as new [model versions](how-to-call.md#specify-the-pii-detection-model) are released.
@@ -24,7 +24,7 @@ Use this article to learn which natural languages are supported by the PII featu
24
24
25
25
## PII language support
26
26
27
-
| Language | Language code | Starting with v3 model version:| Notes |
27
+
| Language | Language code | Starting with model version | Notes |
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/personally-identifiable-information/overview.md
+21-4Lines changed: 21 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,32 +8,49 @@ manager: nitinme
8
8
ms.service: cognitive-services
9
9
ms.subservice: language-service
10
10
ms.topic: overview
11
-
ms.date: 06/15/2022
11
+
ms.date: 08/02/2022
12
12
ms.author: aahi
13
13
ms.custom: language-service-pii, ignite-fall-2021
14
14
---
15
15
16
16
# What is Personally Identifiable Information (PII) detection in Azure Cognitive Service for Language?
17
17
18
-
PII detection is one of the features offered by [Azure Cognitive Service for Language](../overview.md), a collection of machine learning and AI algorithms in the cloud for developing intelligent applications that involve written language. The PII detection feature can identify, categorize, and redact sensitive information in unstructured text. For example: phone numbers, email addresses, and forms of identification. The method for utilizing PII in conversations is different than other use cases, and articles for this use have been separated.
18
+
PII detection is one of the features offered by [Azure Cognitive Service for Language](../overview.md), a collection of machine learning and AI algorithms in the cloud for developing intelligent applications that involve written language. The PII detection feature can **identify, categorize, and redact** sensitive information in unstructured text. For example: phone numbers, email addresses, and forms of identification. The method for utilizing PII in conversations is different than other use cases, and articles for this use have been separated.
19
19
20
20
*[**Quickstarts**](quickstart.md) are getting-started instructions to guide you through making requests to the service.
21
21
*[**How-to guides**](how-to-call.md) contain instructions for using the service in more specific or customized ways.
22
22
* The [**conceptual articles**](concepts/entity-categories.md) provide in-depth explanations of the service's functionality and features.
23
23
24
+
PII comes into two shapes:
25
+
*[PII](how-to-call.md) - works on unstructured text.
26
+
*[Conversation PII (preview)](how-to-call-for-conversations.md) - tailored model to work on conversation transcription.
27
+
24
28
25
29
[!INCLUDE [Typical workflow for pre-configured language features](../includes/overview-typical-workflow.md)]
An AI system includes not only the technology, but also the people who will use it, the people who will be affected by it, and the environment in which it is deployed. Read the [transparency note for PII](/legal/cognitive-services/language-service/transparency-note-personally-identifiable-information?context=/azure/cognitive-services/language-service/context/context) to learn about responsible AI use and deployment in your systems. You can also see the following articles for more information:
37
+
An AI system includes not only the technology, but also the people who will use it, the people who will be affected by it, and the environment in which it's deployed. Read the [transparency note for PII](/legal/cognitive-services/language-service/transparency-note-personally-identifiable-information?context=/azure/cognitive-services/language-service/context/context) to learn about responsible AI use and deployment in your systems. You can also see the following articles for more information:
32
38
33
39
[!INCLUDE [Responsible AI links](../includes/overview-responsible-ai-links.md)]
34
40
41
+
## Example scenarios
42
+
43
+
***Apply sensitivity labels** - For example, based on the results from the PII service, a public sensitivity label might be applied to documents where no PII entities are detected. For documents where US addresses and phone numbers are recognized, a confidential label might be applied. A highly confidential label might be used for documents where bank routing numbers are recognized.
44
+
***Redact some categories of personal information from documents that get wider circulation** - For example, if customer contact records are accessible to first line support representatives, the company may want to redact the customer's personal information besides their name from the version of the customer history to preserve the customer's privacy.
45
+
***Redact personal information in order to reduce unconscious bias** - For example, during a company's resume review process, they may want to block name, address and phone number to help reduce unconscious gender or other biases.
46
+
***Replace personal information in source data for machine learning to reduce unfairness** – For example, if you want to remove names that might reveal gender when training a machine learning model, you could use the service to identify them and you could replace them with generic placeholders for model training.
47
+
***Remove personal information from call center transcription** – For example, if you want to remove names or other PII data that happen between the agent and the customer in a call center scenario. You could use the service to identify and remove them.
48
+
***Data cleaning for data science** - PII can be used to make the data ready for data scientists and engineers to be able to use these data to train their machine learning models. Redacting the data to make sure that customer data isn't exposed.
49
+
50
+
51
+
35
52
## Next steps
36
53
37
54
There are two ways to get started using the entity linking feature:
38
-
*[Language Studio](../language-studio.md), which is a web-based platform that enables you to try several Azure Cognitive Service for Language features without needing to write code.
55
+
*[Language Studio](../language-studio.md), which is a web-based platform that enables you to try several Language service features without needing to write code.
39
56
* The [quickstart article](quickstart.md) for instructions on making requests to the service using the REST API and client library SDK.
Copy file name to clipboardExpand all lines: articles/dms/migration-using-azure-data-studio.md
-1Lines changed: 0 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -129,7 +129,6 @@ When you migrate database(s) using the Azure SQL migration extension for Azure D
129
129
- When migrating to SQL Server on Azure Virtual Machines, SQL Server 2008 and below as target versions are not supported currently.
130
130
- If you are using SQL Server 2012 or SQL Server 2014 you need to store your source database backup files on an Azure Storage Blob Container instead of using the network share option. Store the backup files as page blobs since block blobs are only supported in SQL 2016 and after.
131
131
- Migrating to Azure SQL Database isn't supported.
132
-
- Azure storage accounts secured by specific firewall rules or configured with a private endpoint are not supported for migrations.
133
132
- You can't use an existing self-hosted integration runtime created from Azure Data Factory for database migrations with DMS. Initially, the self-hosted integration runtime should be created using the Azure SQL migration extension in Azure Data Studio and can be reused for further database migrations.
0 commit comments