You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/conversational-language-understanding/how-to/use-containers.md
+20-23Lines changed: 20 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,12 +5,9 @@ description: Use Docker containers for the conversational language understanding
5
5
author: laujan
6
6
manager: nitinme
7
7
ms.service: azure-ai-language
8
-
ms.custom:
9
-
- ignite-2024
10
8
ms.topic: how-to
11
-
ms.date: 04/29/2025
9
+
ms.date: 07/17/2025
12
10
ms.author: lajanuar
13
-
keywords: on-premises, Docker, container
14
11
---
15
12
16
13
# Install and run Conversational Language Understanding (CLU) containers
@@ -26,7 +23,7 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
26
23
27
24
You must meet the following prerequisites before using CLU containers.
28
25
29
-
* If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/).
26
+
*An active Azure subscription. If you don't have one, create a [free account](https://azure.microsoft.com/free/cognitive-services/).
30
27
*[Docker](https://docs.docker.com/) installed on a host computer. Docker must be configured to allow the containers to connect with and send billing data to Azure.
31
28
* On Windows, Docker must also be configured to support Linux containers.
32
29
* You should have a basic understanding of [Docker concepts](https://docs.docker.com/get-started/overview/).
@@ -40,17 +37,17 @@ You must meet the following prerequisites before using CLU containers.
40
37
41
38
The following table describes the minimum and recommended specifications for the available container. Each CPU core must be at least 2.6 gigahertz (GHz) or faster.
42
39
43
-
It's recommended to have a CPU with AVX-512 instruction set, for the best experience (performance and accuracy).
40
+
We recommended that you have a CPU with AVX-512 instruction set, for the best experience (performance and accuracy).
CPU core and memory correspond to the `--cpus` and `--memory` settings, which are used as part of the `docker run` command.
50
47
51
48
## Export your Conversational Language Understanding model
52
49
53
-
Before you proceed with running the docker image, you need to export your own trained model to expose it to your container. Use the following command to extract your model and replace the placeholders below with your own values:
50
+
Before you proceed with running the docker image, you need to export your own trained model to expose it to your container. Use the following command to extract your model and replace the placeholders with your own values:
54
51
55
52
|Placeholder |Value|Format or example|
56
53
|------------|-----|-----------------|
@@ -75,7 +72,7 @@ The CLU container image can be found on the `mcr.microsoft.com` container regist
75
72
76
73
To use the latest version of the container, you can use the `latest` tag, which is for English. You can also find a full list of containers for supported languages using the [tags on the MCR](https://mcr.microsoft.com/product/azure-cognitive-services/language/clu/tags).
77
74
78
-
The latest CLU container is available in several languages. To download the container for the English container, use the command below.
75
+
The latest CLU container is available in several languages. To download the container for the English container, use the following command:
After creating the exported model in the section above, users have to run the container in order to download the deployment package that was created specifically for their exported models.
85
+
After the exported model is created, users have to run the container to download the deployment package that was created specifically for their exported models.
89
86
90
87
| Placeholder| Value | Format or example |
91
88
|---|---|---|
92
89
|**{API_KEY}**| The key for your Language resource. You can find it on your resource's **Key and endpoint** page, on the Azure portal. | See Azure portal |
93
90
|**{ENDPOINT_URI}**| The endpoint for accessing the API. You can find it on your resource's **Key and endpoint** page, on the Azure portal. |`https://<your-custom-subdomain>.cognitiveservices.azure.com`|
94
-
|**{IMAGE_TAG}**| The image tag representing the language of the container you want to run. Make sure this matches the `docker pull` command you used. | latest|
91
+
|**{IMAGE_TAG}**| The image tag representing the language of the container you want to run. Make sure the tag matches the `docker pull` command you used. | latest|
95
92
|**{LOCAL_CLU_PORT}**| Port number assigned for the container in local machine.| 5000 |
96
93
|**{LOCAL_MODEL_DIRECTORY}**| Absolute directory in host machine where exported models are saved in. |`C:\usr\local\myDeploymentPackage`|
97
94
|**{PROJECT_NAME}**| Name of the project that the exported model belongs to | myProject |
DO NOT alter the downloaded files. Even altering the name or folder structure can affect the integrity of the container and might break it.
112
109
113
-
Repeat those steps to download as many models as you'd like to test. They can belong to different projects and have different exported model names.
110
+
Repeat those steps to download as many models as you'd like to test. Your models can belong to different projects and have different (exported) model names.
114
111
115
112
## Run the container with `docker run`
116
113
117
-
Once the container is on the host computer, use the [docker run](https://docs.docker.com/engine/reference/commandline/run/) command to run the containers. The container continues to run until you stop it. Replace the placeholders below with your own values:
114
+
Once the container is on the host computer, use the [docker run](https://docs.docker.com/engine/reference/commandline/run/) command to run the containers. The container continues to run until you stop it. Replace the placeholders with your own values:
118
115
119
116
120
117
> [!IMPORTANT]
121
-
> * The docker commands in the following sections use the back slash, `\`, as a line continuation character. Replace or remove this based on your host operating system's requirements.
122
-
> * The `Eula`, `Billing`, and `ApiKey` options must be specified to run the container; otherwise, the container won't start. For more information, see [Billing](#billing).
118
+
> * The docker commands in the following sections use the back slash, `\`, as a line continuation character. Replace or remove the back slash based on your host operating system's requirements.
119
+
> * The `Eula`, `Billing`, and `ApiKey` options must be specified to run the container; otherwise, the container doesn't start. For more information, see [Billing](#billing).
123
120
124
-
To run the CLU container, execute the following `docker run` command. Replace the placeholders below with your own values:
121
+
To run the CLU container, execute the following `docker run` command. Replace the placeholders with your own values:
125
122
126
123
| Placeholder | Value | Format or example |
127
124
|-------------|-------|---|
128
125
|**{API_KEY}**| The key for your Language resource. You can find it on your resource's **Key and endpoint** page, on the Azure portal. |`xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`|
129
126
|**{ENDPOINT_URI}**| The endpoint for accessing the API. You can find it on your resource's **Key and endpoint** page, on the Azure portal. |`https://<your-custom-subdomain>.cognitiveservices.azure.com`|
130
-
|**{IMAGE_TAG}**| The image tag representing the language of the container you want to run. Make sure this matches the `docker pull` command you used. |`latest`|
127
+
|**{IMAGE_TAG}**| The image tag representing the language of the container you want to run. Make sure the tag matches the `docker pull` command you used. |`latest`|
131
128
|**{LOCAL_CLU_PORT}**|Port number assigned for the container in local machine. |5000 |
132
-
|**{LOCAL_NER_PORT}**|Port number of the NER container. See Run NER Container section below. |5001 (Has to be different that the above port number) |
129
+
|**{LOCAL_NER_PORT}**|Port number of the `NER` container. See Run `NER` Container section. |5001 (Has to be different than the port number) |
133
130
|**{LOCAL_LOGGING_DIRECTORY}**|Absolute directory in host machine where that logs are saved in. |`C:\usr\local\mylogs`|
134
131
|**{LOCAL_MODEL_DIRECTORY}**|Absolute directory in host machine where exported models are saved in. |`C:\usr\local\myDeploymentPackage`|
135
132
@@ -145,13 +142,13 @@ This command:
145
142
146
143
* Runs a *CLU* container from the container image
147
144
* Allocates one CPU core and 8 gigabytes (GB) of memory
148
-
* Exposes TCP port 5000 and allocates a pseudo-TTY for the container
145
+
* Exposes `TCP` port 5000 and allocates a pseudo-TTY for the container
149
146
* Automatically removes the container after it exits. The container image is still available on the host computer.
150
147
151
148
[!INCLUDE [Running multiple containers on the same host](../../../includes/cognitive-services-containers-run-multiple-same-host.md)]
152
149
153
-
## Running NER Container
154
-
CLU relies on NER to handle prebuilt entities. The CLU container works properly without NER if users decide not to integrate it. NER billing is disabled when it's used through CLU, no extra charges are generated unless a call is made directly to NER's container.
150
+
## Running named entity recognition (NER) Container
151
+
CLU relies on NER to handle prebuilt entities. The CLU container works properly without NER if users decide not to integrate it. NER billing is turned off when accessed via CLU, so there are no added charges unless you make a direct call to the NER container.
155
152
156
153
To set up NER in CLU container
157
154
- Follow the [NER container documentation](../../named-entity-recognition/how-to/use-containers.md).
@@ -179,7 +176,7 @@ For information on how to call CLU see [our guide](call-api.md).
179
176
180
177
## Troubleshooting
181
178
182
-
If you run the container with an output [mount](../../concepts/configure-containers.md#mount-settings) and logging enabled, the container generates log files that are helpful to troubleshoot issues that happen while starting or running the container.
179
+
If you run the container with both an output [mount](../../concepts/configure-containers.md#mount-settings) and logging enabled, the container generates log files. The log files can help you troubleshoot any issues that occur during startup or while the container is running.
183
180
184
181
[!INCLUDE [Azure AI services FAQ note](../../../containers/includes/cognitive-services-faq-note.md)]
185
182
@@ -201,7 +198,7 @@ In this article, you learned concepts and workflow for downloading, installing,
201
198
* You must specify billing information when instantiating a container.
202
199
203
200
> [!IMPORTANT]
204
-
> Azure AI containers aren't licensed to run without being connected to Azure for metering. Customers need to enable the containers to communicate billing information with the metering service at all times. Azure AI containers don't send customer data (for example, text that is being analyzed) to Microsoft.
201
+
> Azure AI containers aren't licensed to run without being connected to Azure for metering. Customers must enable containers to always communicate billing information to the metering service. Azure AI containers don't send customer data (for example, text that is being analyzed) to Microsoft.
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/custom-named-entity-recognition/overview.md
+19-17Lines changed: 19 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,17 +6,17 @@ author: laujan
6
6
manager: nitinme
7
7
ms.service: azure-ai-language
8
8
ms.topic: overview
9
-
ms.date: 04/29/2025
9
+
ms.date: 07/17/2025
10
10
ms.author: lajanuar
11
11
ms.custom: language-service-custom-ner
12
12
---
13
13
14
14
# What is custom named entity recognition?
15
15
16
-
Custom NER is one of the custom features offered by [Azure AI Language](../overview.md). It is a cloud-based API service that applies machine-learning intelligence to enable you to build custom models for custom named entity recognition tasks.
16
+
Custom named entity recognition (NER) is a cloud-based API service that uses machine learning to help you build models designed for your unique entity recognition requirements. It's one of the specialized features available through [Azure AI Language](../overview.md). With custom NER, you can create AI models that extract domain-specific entities from unstructured text, such as contracts or financial documents. When you start a Custom NER project, you can repeatedly label data, train and evaluate your model, and improve its performance before deploying it. The quality of your labeled data is essential, as it directly impacts the model's accuracy.
17
+
18
+
To simplify building and customizing your model, the service offers a custom web portal that can be accessed through the [Language studio](https://aka.ms/languageStudio). You can easily get started with the service by following the steps in this [quickstart](quickstart.md).
17
19
18
-
Custom NER enables users to build custom AI models to extract domain-specific entities from unstructured text, such as contracts or financial documents. By creating a Custom NER project, developers can iteratively label data, train, evaluate, and improve model performance before making it available for consumption. The quality of the labeled data greatly impacts model performance. To simplify building and customizing your model, the service offers a custom web portal that can be accessed through the [Language studio](https://aka.ms/languageStudio). You can easily get started with the service by following the steps in this [quickstart](quickstart.md).
19
-
20
20
This documentation contains the following article types:
21
21
22
22
*[Quickstarts](quickstart.md) are getting-started instructions to guide you through making requests to the service.
@@ -25,7 +25,7 @@ This documentation contains the following article types:
25
25
26
26
## Example usage scenarios
27
27
28
-
Custom named entity recognition can be used in multiple scenarios across a variety of industries:
28
+
Custom named entity recognition can be used in multiple scenarios across various industries:
29
29
30
30
### Information extraction
31
31
@@ -37,24 +37,26 @@ Search is foundational to any app that surfaces text content to users. Common sc
37
37
38
38
### Audit and compliance
39
39
40
-
Instead of manually reviewing significantly long text files to audit and apply policies, IT departments in financial or legal enterprises can use custom NER to build automated solutions. These solutions can be helpful to enforce compliance policies, and set up necessary business rules based on knowledge mining pipelines that process structured and unstructured content.
40
+
Instead of manually reviewing long text files to audit and apply policies, IT departments in financial or legal enterprises can use custom NER to build automated solutions. These solutions can be helpful to enforce compliance policies, and set up necessary business rules based on knowledge mining pipelines that process structured and unstructured content.
41
41
42
42
## Project development lifecycle
43
43
44
-
Using custom NER typically involves several different steps.
44
+
Using custom NER typically involves several different steps.
45
45
46
46
:::image type="content" source="media/development-lifecycle.png" alt-text="The development lifecycle" lightbox="media/development-lifecycle.png":::
47
47
48
48
1.**Define your schema**: Know your data and identify the [entities](glossary.md#entity) you want extracted. Avoid ambiguity.
49
49
50
-
2.**Label your data**: Labeling data is a key factor in determining model performance. Label precisely, consistently and completely.
51
-
1.**Label precisely**: Label each entity to its right type always. Only include what you want extracted, avoid unnecessary data in your labels.
52
-
2.**Label consistently**: The same entity should have the same label across all the files.
53
-
3.**Label completely**: Label all the instances of the entity in all your files.
50
+
1.**Label your data**: Labeling data is a key factor in determining model performance. Label precisely, consistently and completely.
51
+
***Label precisely**: Label each entity to its right type always. Only include what you want extracted and avoid unnecessary data in your labels.
52
+
***Label consistently**: The same entity should have the same label across all the files.
53
+
**Label completely**: Label all the instances of the entity in all your files.
54
+
55
+
1.**Train the model**: Your model starts learning from your labeled data.
56
+
57
+
1.**View the model's performance**: After training, review evaluation results and analyze performance for improvement.
54
58
55
-
3.**Train the model**: Your model starts learning from your labeled data.
56
59
57
-
4.**View the model's performance**: After training is completed, view the model's evaluation details, its performance and guidance on how to improve it.
58
60
59
61
6.**Deploy the model**: Deploying a model makes it available for use via the [Analyze API](https://aka.ms/ct-runtime-swagger).
60
62
@@ -73,16 +75,16 @@ As you use custom NER, see the following reference documentation and samples for
An AI system includes not only the technology, but also the people who will use it, the people who will be affected by it, and the environment in which it is deployed. Read the [transparency note](/azure/ai-foundry/responsible-ai/language-service/transparency-note) to learn about responsible AI use and deployment in your systems. You can also see the following articles for more information:
80
+
An AI system includes not only the technology, but also the people who use it, the people affected by it, and the deployment environment. Read the [transparency note](/azure/ai-foundry/responsible-ai/language-service/transparency-note) to learn about responsible AI use and deployment in your systems. For more information, *see* the following articles:
79
81
80
82
[!INCLUDE [Responsible AI links](../includes/overview-responsible-ai-links.md)]
81
83
82
84
## Next steps
83
85
84
-
* Use the [quickstart article](quickstart.md) to start using custom named entity recognition.
86
+
* Use the [quickstart article](quickstart.md) to start using custom named entity recognition.
85
87
86
-
* As you go through the project development lifecycle, review the [glossary](glossary.md) to learn more about the terms used throughout the documentation for this feature.
88
+
* As you go through the project development lifecycle, review the [glossary](glossary.md) to learn more about the terms used throughout the documentation for this feature.
87
89
88
90
* Remember to view the [service limits](service-limits.md) for information such as [regional availability](service-limits.md#regional-availability).
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/personally-identifiable-information/how-to/redact-document-pii.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,7 +57,7 @@ A native document refers to the file format used to create the original document
57
57
58
58
***Let's get started:***
59
59
60
-
* For this project, we use the cURL commandline tool to make REST API calls.
60
+
* For this project, we use the cURL command-line tool to make REST API calls.
61
61
62
62
> [!NOTE]
63
63
> The cURL package is preinstalled on most Windows 10 and Windows 11 and most macOS and Linux distributions. You can check the package version with the following commands:
0 commit comments