You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/conversational-language-understanding/how-to/tag-utterances.md
+54-54Lines changed: 54 additions & 54 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,41 +2,41 @@
2
2
title: How to tag utterances in Conversational Language Understanding
3
3
titleSuffix: Azure AI services
4
4
description: Use this article to tag your utterances in Conversational Language Understanding projects
5
-
author: jboback
5
+
author: laujan
6
6
manager: nitinme
7
7
ms.service: azure-ai-language
8
8
ms.topic: how-to
9
-
ms.date: 11/21/2024
10
-
ms.author: jboback
9
+
ms.date: 05/20/2025
10
+
ms.author: lajanuar
11
11
ms.custom: language-service-clu
12
12
---
13
13
14
14
# Label your utterances in AI Foundry
15
15
16
-
Once you have [built a schema](build-schema.md) for your fine-tuning task, you should add training utterances to your project. The utterances should be similar to what your users use when interacting with the project. When you add an utterance, you have to assign which intent it belongs to. After the utterance is added, label the words within your utterance that you want to extract as entities.
16
+
Once you [build a schema](build-schema.md) for your fine-tuning task, you should add training utterances to your project. The utterances should be similar to what your users use when interacting with the project. When you add an utterance, you have to assign which intent it belongs to. After the utterance is added, label the words within your utterance that you want to extract as entities.
17
17
18
-
Data labeling is a crucial step in the CLU trained development lifecycle; this data are used in the next step when training your model so that your model can learn from the labeled data. If you already have labeled utterances, you can directly [import it into your project](create-project.md#import-project), provided that your data follows the [accepted data format](../concepts/data-formats.md). See [create fine-tuning task](create-project.md#import-project) to learn more about importing labeled data. Labeled data informs the model how to interpret text and is used for training and evaluation.
18
+
Data labeling is a crucial step in the conversational language understanding `CLU` trained development lifecycle; this data are used in the next step when training your model so that your model can learn from the labeled data. If you already labeled utterances, you can directly [import it into your project](create-project.md#import-project), IF your data follows the [accepted data format](../concepts/data-formats.md). See [create fine-tuning task](create-project.md#import-project) to learn more about importing labeled data. Labeled data informs the model how to interpret text and is used for training and evaluation.
19
19
20
20
> [!TIP]
21
-
> Use the **Quick Deploy** option to implement custom CLU intent routing, powered by your own LLM model deployment without adding or labeling any training data.
21
+
> Use the `Quick Deploy` option to implement custom `CLU` intent routing, powered by your own `LLM` model deployment without adding or labeling any training data.
22
22
23
23
## Prerequisites
24
24
25
25
Before you can label your data, you need:
26
26
27
27
* A successfully [created project](create-project.md).
28
28
29
-
See the [CLU development lifecycle](../overview.md#project-development-lifecycle) for more information.
29
+
For more information, see the [conversational language understanding development lifecycle](../overview.md#project-development-lifecycle).
30
30
31
31
## Data labeling guidelines
32
32
33
33
After [building your schema](build-schema.md) and [creating your project](create-project.md), you need to label your data. Labeling your data is important so your model knows which sentences and words are associated with the intents and entities in your project. Spend time labeling your utterances - introducing and refining the data that is used in training your models.
34
34
35
35
As you add utterances and label them, keep in mind:
36
36
37
-
* The machine learning models generalize based on the labeled examples you provide it; the more examples you provide, the more data points the model has to make better generalizations.
37
+
* The machine learning models generalize based on the labeled examples you provide it; the more examples you provide, the more data points the model has to make better generalizations.
38
38
39
-
* The precision, consistency and completeness of your labeled data are key factors to determining model performance.
39
+
* The precision, consistency, and completeness of your labeled data are key factors to determining model performance.
40
40
41
41
***Label precisely**: Label each intent and entity to its right type always. Only include what you want classified and extracted, avoid unnecessary data in your labels.
42
42
***Label consistently**: The same entity should have the same label across all the utterances.
@@ -56,67 +56,67 @@ Use the following steps to label your utterances:
56
56
57
57
1. Go to your project page in [AI Foundry](https://ai.azure.com).
58
58
59
-
2. From the left side menu, select **Manage data**. In this page, you can start adding your utterances and labeling them. You can also upload your utterances directly by clicking on **Upload utterance file** from the top menu, make sure it follows the [accepted format](../concepts/data-formats.md#utterance-file-format).
59
+
1. From the left side menu, select `Manage data`. In this page, you can start adding your utterances and labeling them. You can also upload your utterances directly by clicking on `Upload utterance file` from the top menu. Make sure it follows the [accepted format](../concepts/data-formats.md#utterance-file-format).
60
+
61
+
1. From the top pivots, you can change the view to be `training set` or `testing set`. Learn more about [training and testing sets](train-model.md#data-splitting) and how they're used for model training and evaluation.
60
62
61
-
3. From the top pivots, you can change the view to be **training set** or **testing set**. Learn more about [training and testing sets](train-model.md#data-splitting) and how they're used for model training and evaluation.
62
-
63
63
:::image type="content" source="../media/tag-utterances.png" alt-text="A screenshot of the page for tagging utterances in Language Studio." lightbox="../media/tag-utterances.png":::
64
-
64
+
65
65
> [!TIP]
66
-
> If you are planning on using **Automatically split the testing set from training data** splitting, add all your utterances to the training set.
67
-
68
-
69
-
4. From the **Select intent** dropdown menu, select one of the intents, the language of the utterance (for multilingual projects), and the utterance itself. Press the enter key in the utterance's text box to add the utterance.
66
+
> If you're planning on using `Automatically split the testing set from training data` splitting, add all your utterances to the training set.
67
+
68
+
69
+
1. From the `Select intent` dropdown menu, select one of the intents, the language of the utterance (for multilingual projects), and the utterance itself. Press enter key in the utterance's text box and add the utterance.
70
+
71
+
1. You have two options to label entities in an utterance:
70
72
71
-
5. You have two options to label entities in an utterance:
72
-
73
73
|Option |Description |
74
74
|---------|---------|
75
75
|Label using a brush | Select the brush icon next to an entity in the right pane, then highlight the text in the utterance you want to label. |
76
76
|Label using inline menu | Highlight the word you want to label as an entity, and a menu appears. Select the entity you want to label these words with. |
77
-
78
-
6. In the right side pane, under the **Labels** pivot, you can find all the entity types in your project and the count of labeled instances per each.
79
77
80
-
7. Under the **Distribution** pivot you can view the distribution across training and testing sets. You have two options for viewing:
78
+
1. In the right side pane, under the `Labels` pivot, you can find all the entity types in your project and the count of labeled instances per each.
79
+
80
+
1. Under the `Distribution` pivot, you can view the distribution across training and testing sets. You have two options for viewing:
81
81
**Total instances per labeled entity* where you can view count of all labeled instances of a specific entity.
82
82
**Unique utterances per labeled entity* where each utterance is counted if it contains at least one labeled instance of this entity.
83
83
**Utterances per intent* where you can view count of utterances per intent.
84
84
85
-
:::image type="content" source="../media/label-distribution.png" alt-text="A screenshot showing entity distribution in Language Studio." lightbox="../media/label-distribution.png":::
85
+
:::image type="content" source="../media/label-distribution.png" alt-text="A screenshot showing entity distribution in Language Studio." lightbox="../media/label-distribution.png":::
86
86
87
87
88
88
> [!NOTE]
89
-
> List, regex, and prebuilt components are not shown in the data labeling page, and all labels here only apply to the **learned component**.
89
+
> List, regex, and prebuilt components aren't shown in the data labeling page, and all labels here only apply to the **learned component**.
90
90
91
91
To remove a label:
92
92
1. From within your utterance, select the entity you want to remove a label from.
93
-
3. Scroll through the menu that appears, and select **Remove label**.
93
+
1. Scroll through the menu that appears, and select **Remove label**.
94
94
95
95
To delete an entity:
96
-
1. Select the garbage bin icon next to the entity you want to edit in the right side pane. Then select **Delete** to confirm.
96
+
1. Select the garbage bin icon next to the entity you want to edit in the right side pane. Then select **Delete** to confirm.
97
97
98
98
## Suggest utterances with Azure OpenAI
99
99
100
-
In CLU, use Azure OpenAI to suggest utterances to add to your project using generative language models. It is recommended to use an AI Foundry resource while using CLU, so you do not need to connect multiple resources. In order to use the AI Foundry resource, you'll need to provide your AI Foundry resource with elevated access. To do this, access the Azure Portal. Within your Azure AI resource, provide access as a *Cognitive Services User* to itself. This ensures that all parts of your resource are communicating correctly.
100
+
In `CLU`, use Azure OpenAI to suggest utterances to add to your project using generative language models. We recommended that you use an AI Foundry resource while using `CLU`, so you don't need to connect multiple resources. In order to use the AI Foundry resource, you need to provide your AI Foundry resource with elevated access. To do so, access the Azure portal. Within your Azure AI resource, provide access as a *Cognitive Services User* to itself. This step ensures that all parts of your resource are communicating correctly.
101
101
102
-
###Connecting with separate Language and AOAI Resources
102
+
###Connect with separate Language and Azure OpenAI Resources
103
103
104
-
You first need to get access and create a resource in Azure OpenAI. Next, you'll create a connection to the Azure OpenAI resource within the same AI Foundry project in the **Management center** in the left panel of the Azure AI Foundry page. You'll then need to create a deployment for the AOAI models within the connected AOAI resource. Follow the pre-requisite steps [here](../../../openai/how-to/create-resource.md) to create a new resource.
104
+
You first need to get access and create a resource in Azure OpenAI. Next, create a connection to the Azure OpenAI resource within the same AI Foundry project in the `Management center` in the left panel of the Azure AI Foundry page. You then need to create a deployment for the AOAI models within the connected AOAI resource. Follow the prerequisite steps [here](../../../openai/how-to/create-resource.md) to create a new resource.
105
105
106
-
Before you get started, the suggest utterances feature is only available if your Language resource is in the following regions:
106
+
Before you get started, the suggested utterances feature is only available if your Language resource is in the following regions:
107
107
* East US
108
108
* South Central US
109
109
* West Europe
110
110
111
-
In the Data Labeling page:
111
+
In the Data Labeling page:
112
112
113
-
1. Select the **Suggest utterances** button. A pane opens up on the right side prompting you to select your Azure OpenAI resource and deployment.
114
-
2. On selection of an Azure OpenAI resource, select **Connect**, which allows your Language resource to have direct access to your Azure OpenAI resource. It assigns your Language resource the role of `Cognitive Services User` to your Azure OpenAI resource, which allows your current Language resource to have access to Azure OpenAI's service. If the connection fails, follow these[steps](#add-required-configurations-to-azure-openai-resource)below to add the right role to your Azure OpenAI resource manually.
115
-
3. Once the resource is connected, select the deployment. The recommended model for the Azure OpenAI deployment is `gpt-35-turbo-instruct`.
116
-
4. Select the intent you'd like to get suggestions for. Make sure the intent you have selected has at least 5 saved utterances to be enabled for utterance suggestions. The suggestions provided by Azure OpenAI are based on the **most recent utterances** you've added for that intent.
117
-
5. Select **Generate utterances**. Once complete, the suggested utterances show up with a dotted line around it, with the note *Generated by AI*. Those suggestions need to be accepted or rejected. Accepting a suggestion simply adds it to your project, as if you had added it yourself. Rejecting it deletes the suggestion entirely. Only accepted utterances are part of your project and used for training or testing. You can accept or reject by clicking on the green check or red cancel buttons beside each utterance. You can also use the `Accept all` and `Reject all` buttons in the toolbar.
113
+
1. Select the `Suggest utterances` button. A pane opens up on the right side prompting you to select your Azure OpenAI resource and deployment.
114
+
1. On selection of an Azure OpenAI resource, select `Connect`, which allows your Language resource to have direct access to your Azure OpenAI resource. It assigns your Language resource the role of `Cognitive Services User` to your Azure OpenAI resource, which allows your current Language resource to have access to Azure OpenAI's service. If the connection fails, the following[steps](#add-required-configurations-to-azure-openai-resource)enable you to manually add the correct role to your Azure OpenAI resource.
115
+
1. Once the resource is connected, select the deployment. The recommended model for the Azure OpenAI deployment is `gpt-35-turbo-instruct`.
116
+
1. Select the intent you'd like to get suggestions for. Make sure the intent you selected has at least five saved utterances to be enabled for utterance suggestions. The suggestions provided by Azure OpenAI are based on the `most recent utterances` you added for that intent.
117
+
1. Select `Generate utterances`. Once complete, the suggested utterances show up with a dotted line around it, with the note *Generated by AI*. Those suggestions need to be accepted or rejected. Accepting a suggestion simply adds it to your project, as if you had added it yourself. Rejecting it deletes the suggestion entirely. Only accepted utterances are part of your project and used for training or testing. You can accept or reject by clicking on the green check or red cancel buttons beside each utterance. You can also use the `Accept all` and `Reject all` buttons in the toolbar.
118
118
119
-
:::image type="content" source="../media/suggest-utterances.png" alt-text="A screenshot showing utterance suggestions in Language Studio." lightbox="../media/suggest-utterances.png":::
Using this feature entails a charge to your Azure OpenAI resource for a similar number of tokens to the suggested utterances generated. Details for Azure OpenAI's pricing can be found [here](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/).
122
122
@@ -128,33 +128,33 @@ Enable identity management for your Language resource using the following option
128
128
129
129
Your Language resource must have identity management, to enable it using the [Azure portal](https://portal.azure.com):
130
130
131
-
1. Go to your Language resource
132
-
2. From left hand menu, under **Resource Management** section, select **Identity**
133
-
3. From **System assigned** tab, make sure to set **Status** to **On**
131
+
1. Go to your Language resource.
132
+
1. From left hand menu, under `Resource Management` section, select `Identity`.
133
+
1. From `System assigned` tab, make sure to set `Status` to `On`.
134
134
135
135
#### [Language Studio](#tab/studio)
136
136
137
137
Your Language resource must have identity management, to enable it using [Language Studio](https://aka.ms/languageStudio):
138
138
139
-
1. Select the settings icon in the top right corner of the screen
140
-
2. Select **Resources**
141
-
3. Select the check box **Managed Identity** for your Language resource.
139
+
1. Select the settings icon in the top right corner of the screen.
140
+
1. Select *`Resources`.
141
+
1. Select the check box `Managed Identity` for your Language resource.
142
142
143
143
---
144
144
145
-
After enabling managed identity, assign the role `Cognitive Services User` to your Azure OpenAI resource using the managed identity of your Language resource.
145
+
After enabling managed identity, assign the role `Cognitive Services User` to your Azure OpenAI resource using the managed identity of your Language resource.
146
146
147
147
1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your Azure OpenAI resource.
148
-
2. Select the Access Control (IAM) tab on the left.
149
-
3. Select Add > Add role assignment.
150
-
4. Select "Job function roles" and click Next.
151
-
5. Select `Cognitive Services User` from the list of roles and click Next.
152
-
6. Select Assign access to "Managed identity" and select "Select members".
153
-
7. Under "Managed identity" select "Language".
154
-
8. Search for your resource and select it. Then select the Select button below and next to complete the process.
155
-
9. Review the details and select Review + Assign.
156
-
157
-
:::image type="content" source="../media/add-role-azure-openai.gif" alt-text="Multiple screenshots showing the steps to add the required role to your Azure OpenAI resource." lightbox="../media/add-role-azure-openai.gif":::
148
+
1. Select the `Access Control (IAM)` tab.
149
+
1. Select `Add` > Add role assignment.
150
+
1. Select `Job function roles` and select `Next`.
151
+
1. Select `Cognitive Services User` from the list of roles and select `Next`.
152
+
1. Select Assign access to "Managed identity" and select `Select members`.
153
+
1. Under `Managed identity` select `Language`.
154
+
1. Search for your resource and select it. Then select `Next`and complete the process.
155
+
1. Review the details and select `Review + Assign`.
156
+
157
+
:::image type="content" source="../media/add-role-azure-openai.gif" alt-text="Multiple screenshots showing the steps to add the required role to your Azure OpenAI resource." lightbox="../media/add-role-azure-openai.gif":::
158
158
159
159
After a few minutes, refresh the AI Foundry, and you can successfully connect to Azure OpenAI.
0 commit comments