Skip to content

Commit 6f1fc86

Browse files
Merge pull request #693 from jboback/pii-containers
Added PII container documentation
2 parents 31d56f9 + ecc0512 commit 6f1fc86

File tree

5 files changed

+165
-0
lines changed

5 files changed

+165
-0
lines changed

articles/ai-services/cognitive-services-container-support.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ Azure AI containers provide the following set of Docker containers, each of whic
5353
| [Language service][ta-containers-sentiment] | **Sentiment Analysis** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/sentiment/about)) | Analyzes raw text for clues about positive or negative sentiment. This version of sentiment analysis returns sentiment labels (for example *positive* or *negative*) for each document and sentence within it. | Generally available. <br> This container can also [run in disconnected environments](containers/disconnected-containers.md). |
5454
| [Language service][ta-containers-health] | **Text Analytics for health** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/healthcare/about))| Extract and label medical information from unstructured clinical text. | Generally available |
5555
| [Language service][ta-containers-ner] | **Named Entity Recognition** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/ner/about))| Extract named entities from text. | Generally available. <br>This container can also [run in disconnected environments](containers/disconnected-containers.md). |
56+
| [Language service][ta-containers-pii] | **Personally Identifiable Information (PII) detection** ([image](https://mcr.microsoft.com/en-us/product/azure-cognitive-services/textanalytics/pii/tags))| Detect and redact personally identifiable information entities from text. | Generally available. <br>This container can also [run in disconnected environments](containers/disconnected-containers.md). |
5657
| [Language service][ta-containers-cner] | **Custom Named Entity Recognition** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/customner/about))| Extract named entities from text, using a custom model you create using your data. | Generally available |
5758
| [Language service][ta-containers-summarization] | **Summarization** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/summarization/about))| Summarize text from various sources. | Public preview. <br>This container can also [run in disconnected environments](containers/disconnected-containers.md). |
5859
| [Translator][tr-containers] | **Translator** ([image](https://mcr.microsoft.com/product/azure-cognitive-services/translator/text-translation/about))| Translate text in several languages and dialects. | Generally available. Gated - [request access](https://aka.ms/csgate-translator). <br>This container can also [run in disconnected environments](containers/disconnected-containers.md). |
@@ -131,5 +132,6 @@ Install and explore the functionality provided by containers in Azure AI service
131132
[ta-containers-cner]: language-service/custom-named-entity-recognition/how-to/use-containers.md
132133
[ta-containers-summarization]: language-service/summarization/how-to/use-containers.md
133134
[ta-containers-ner]: language-service/named-entity-recognition/how-to/use-containers.md
135+
[ta-containers-pii]: language-service/personally-identifiable-information//how-to/use-containers.md
134136
[tr-containers]: translator/containers/translator-how-to-install-container.md
135137
[request-access]: https://aka.ms/csgate

articles/ai-services/containers/disconnected-containers.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ Containers enable you to run Azure AI services APIs in your own environment, and
2727
* [Language Detection](../language-service/language-detection/how-to/use-containers.md)
2828
* [Summarization](../language-service/summarization/how-to/use-containers.md)
2929
* [Named Entity Recognition](../language-service/named-entity-recognition/how-to/use-containers.md)
30+
* [Personally Identifiable Information (PII) detection](../language-service/personally-identifiable-information/how-to/use-containers.md)
3031
* [Azure AI Vision - Read](../computer-vision/computer-vision-how-to-install-containers.md)
3132
* [Document Intelligence](../../ai-services/document-intelligence/containers/disconnected.md)
3233

@@ -87,6 +88,8 @@ See the following documentation for steps on downloading and configuring the con
8788
* [Sentiment Analysis](../language-service/sentiment-opinion-mining/how-to/use-containers.md#run-the-container-disconnected-from-the-internet)
8889
* [Key Phrase Extraction](../language-service/key-phrase-extraction/how-to/use-containers.md#run-the-container-disconnected-from-the-internet)
8990
* [Language Detection](../language-service/language-detection/how-to/use-containers.md#run-the-container-disconnected-from-the-internet)
91+
* [Named Entity Recognition](../language-service/named-entity-recognition/how-to/use-containers.md#run-the-container-disconnected-from-the-internet)
92+
* [Personally Identifiable Information (PII) detection](../language-service/personally-identifiable-information/how-to/use-containers.md#run-the-container-disconnected-from-the-internet)
9093

9194
## Environment variable names in Kubernetes deployments
9295
Some Azure AI Containers, for example Translator, require users to pass environmental variable names that include colons (`:`) when running the container. This will work fine when using Docker, but Kubernetes does not accept colons in environmental variable names.

articles/ai-services/language-service/concepts/configure-containers.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Language service provides each container with a common configuration framework,
2323
* Text Analytics for Health
2424
* Summarization
2525
* Named Entity Recognition (NER)
26+
* Personally Identifiable (PII) detection
2627

2728
## Configuration settings
2829

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
---
2+
title: Use personally identifiable information Docker containers on-premises
3+
titleSuffix: Azure AI services
4+
description: Use Docker containers for the Personally Identifiable Information API to determine the language of written text, on-premises.
5+
#services: cognitive-services
6+
author: jboback
7+
manager: nitinme
8+
ms.service: azure-ai-language
9+
ms.custom:
10+
ms.topic: how-to
11+
ms.date: 10/07/2024
12+
ms.author: jboback
13+
keywords: on-premises, Docker, container
14+
---
15+
16+
# Install and run Personally Identifiable Information (PII) Detection containers
17+
18+
> [!NOTE]
19+
> The data limits for the PII container are 5120 characters and 10 documents max.
20+
21+
Containers enable you to host the PII detection API on your own infrastructure. If you have security or data governance requirements that can't be fulfilled by calling PII detection remotely, then containers might be a good option.
22+
23+
If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
24+
25+
## Prerequisites
26+
27+
You must meet the following prerequisites before using PII detection containers.
28+
29+
* If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/).
30+
* [Docker](https://docs.docker.com/) installed on a host computer. Docker must be configured to allow the containers to connect with and send billing data to Azure.
31+
* On Windows, Docker must also be configured to support Linux containers.
32+
* You should have a basic understanding of [Docker concepts](https://docs.docker.com/get-started/overview/).
33+
* A <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesTextAnalytics" title="Create a Language resource" target="_blank">Language resource </a>
34+
35+
[!INCLUDE [Gathering required parameters](../../../containers/includes/container-gathering-required-parameters.md)]
36+
37+
## Host computer requirements and recommendations
38+
39+
[!INCLUDE [Host Computer requirements](../../../includes/cognitive-services-containers-host-computer.md)]
40+
41+
The following table describes the minimum and recommended specifications for the available container. Each CPU core must be at least 2.6 gigahertz (GHz) or faster.
42+
43+
It is recommended to have a CPU with AVX-512 instruction set, for the best experience (performance and accuracy).
44+
45+
| | Minimum host specs | Recommended host specs |
46+
|---------------------|------------------------|------------------------|
47+
| **PII detection** | 1 core, 2GB memory | 4 cores, 8GB memory |
48+
49+
CPU core and memory correspond to the `--cpus` and `--memory` settings, which are used as part of the `docker run` command.
50+
51+
## Get the container image with `docker pull`
52+
53+
The PII detection container image can be found on the `mcr.microsoft.com` container registry syndicate. It resides within the `azure-cognitive-services/textanalytics/` repository and is named `pii`. The fully qualified container image name is, `mcr.microsoft.com/azure-cognitive-services/textanalytics/pii`
54+
55+
To use the latest version of the container, you can use the `latest` tag, which is for English. You can also find a full list of containers for supported languages using the [tags on the MCR](https://mcr.microsoft.com/product/azure-cognitive-services/textanalytics/pii/tags).
56+
57+
The latest PII detection container is available in several languages. To download the container for the English container, use the command below.
58+
59+
```
60+
docker pull mcr.microsoft.com/azure-cognitive-services/textanalytics/pii:latest
61+
```
62+
63+
[!INCLUDE [Tip for using docker list](../../../includes/cognitive-services-containers-docker-list-tip.md)]
64+
65+
## Run the container with `docker run`
66+
67+
Once the container is on the host computer, use the [docker run](https://docs.docker.com/engine/reference/commandline/run/) command to run the containers. The container will continue to run until you stop it. Replace the placeholders below with your own values:
68+
69+
70+
> [!IMPORTANT]
71+
> * The docker commands in the following sections use the back slash, `\`, as a line continuation character. Replace or remove this based on your host operating system's requirements.
72+
> * The `Eula`, `Billing`, and `ApiKey` options must be specified to run the container; otherwise, the container won't start. For more information, see [Billing](#billing).
73+
74+
To run the PII detection container, execute the following `docker run` command. Replace the placeholders below with your own values:
75+
76+
| Placeholder | Value | Format or example |
77+
|-------------|-------|---|
78+
| **{API_KEY}** | The key for your Language resource. You can find it on your resource's **Key and endpoint** page, on the Azure portal. |`xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`|
79+
| **{ENDPOINT_URI}** | The endpoint for accessing the API. You can find it on your resource's **Key and endpoint** page, on the Azure portal. | `https://<your-custom-subdomain>.cognitiveservices.azure.com` |
80+
| **{IMAGE_TAG}** | The image tag representing the language of the container you want to run. Make sure this matches the `docker pull` command you used. | `latest` |
81+
82+
```bash
83+
docker run --rm -it -p 5000:5000 --memory 8g --cpus 1 \
84+
mcr.microsoft.com/azure-cognitive-services/textanalytics/pii:{IMAGE_TAG} \
85+
Eula=accept \
86+
Billing={ENDPOINT_URI} \
87+
ApiKey={API_KEY}
88+
```
89+
90+
This command:
91+
92+
* Runs a *PII detection* container from the container image
93+
* Allocates one CPU core and 8 gigabytes (GB) of memory
94+
* Exposes TCP port 5000 and allocates a pseudo-TTY for the container
95+
* Automatically removes the container after it exits. The container image is still available on the host computer.
96+
97+
[!INCLUDE [Running multiple containers on the same host](../../../includes/cognitive-services-containers-run-multiple-same-host.md)]
98+
99+
## Query the container's prediction endpoint
100+
101+
The container provides REST-based query prediction endpoint APIs.
102+
103+
Use the host, `http://localhost:5000`, for container APIs.
104+
105+
<!-- ## Validate container is running -->
106+
107+
[!INCLUDE [Container's API documentation](../../../includes/cognitive-services-containers-api-documentation.md)]
108+
109+
For information on how to call PII see [our guide](../how-to-call.md).
110+
111+
## Run the container disconnected from the internet
112+
113+
[!INCLUDE [configure-disconnected-container](../../../containers/includes/configure-disconnected-container.md)]
114+
115+
## Stop the container
116+
117+
[!INCLUDE [How to stop the container](../../../includes/cognitive-services-containers-stop.md)]
118+
119+
## Troubleshooting
120+
121+
If you run the container with an output [mount](../../concepts/configure-containers.md#mount-settings) and logging enabled, the container generates log files that are helpful to troubleshoot issues that happen while starting or running the container.
122+
123+
[!INCLUDE [Azure AI services FAQ note](../../../containers/includes/cognitive-services-faq-note.md)]
124+
125+
## Billing
126+
127+
The PII detection containers send billing information to Azure, using a _Language_ resource on your Azure account.
128+
129+
[!INCLUDE [Container's Billing Settings](../../../includes/cognitive-services-containers-how-to-billing-info.md)]
130+
131+
For more information about these options, see [Configure containers](../../concepts/configure-containers.md).
132+
133+
## Summary
134+
135+
In this article, you learned concepts and workflow for downloading, installing, and running PII detection containers. In summary:
136+
137+
* PII detection provides Linux containers for Docker
138+
* Container images are downloaded from the Microsoft Container Registry (MCR).
139+
* Container images run in Docker.
140+
* You must specify billing information when instantiating a container.
141+
142+
> [!IMPORTANT]
143+
> Azure AI containers are not licensed to run without being connected to Azure for metering. Customers need to enable the containers to communicate billing information with the metering service at all times. Azure AI containers do not send customer data (e.g. text that is being analyzed) to Microsoft.
144+
145+
## Next steps
146+
147+
* See [Configure containers](../../concepts/configure-containers.md) for configuration settings.

articles/ai-services/language-service/toc.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -424,6 +424,18 @@ items:
424424
href: personally-identifiable-information/how-to-call.md
425425
- name: Call PII for conversations
426426
href: personally-identifiable-information/how-to-call-for-conversations.md
427+
- name: Use containers
428+
items:
429+
- name: Use Docker containers
430+
href: personally-identifiable-information/how-to/use-containers.md
431+
- name: Configure containers
432+
href: concepts/configure-containers.md
433+
- name: Use container instances
434+
href: ../containers/azure-container-instance-recipe.md?context=/azure/ai-services/language-service/context/context
435+
- name: Use containers in disconnected environments
436+
href: ../containers/disconnected-containers.md
437+
- name: Azure AI containers overview
438+
href: ../cognitive-services-container-support.md
427439
- name: Concepts
428440
items:
429441
- name: Recognized entity categories

0 commit comments

Comments
 (0)