You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/language-service/conversational-language-understanding/how-to/tag-utterances.md
+65-61Lines changed: 65 additions & 61 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,38 +2,41 @@
2
2
title: How to tag utterances in Conversational Language Understanding
3
3
titleSuffix: Azure AI services
4
4
description: Use this article to tag your utterances in Conversational Language Understanding projects
5
-
author: jboback
5
+
author: laujan
6
6
manager: nitinme
7
7
ms.service: azure-ai-language
8
8
ms.topic: how-to
9
-
ms.date: 11/21/2024
10
-
ms.author: jboback
9
+
ms.date: 05/20/2025
10
+
ms.author: lajanuar
11
11
ms.custom: language-service-clu
12
12
---
13
13
14
-
# Label your utterances in Language Studio
14
+
# Label your utterances in AI Foundry
15
15
16
-
Once you have [built a schema](build-schema.md) for your project, you should add training utterances to your project. The utterances should be similar to what your users use when interacting with the project. When you add an utterance, you have to assign which intent it belongs to. After the utterance is added, label the words within your utterance that you want to extract as entities.
16
+
Once you [build a schema](build-schema.md) for your fine-tuning task, you should add training utterances to your project. The utterances should be similar to what your users use when interacting with the project. When you add an utterance, you have to assign which intent it belongs to. After the utterance is added, label the words within your utterance that you want to extract as entities.
17
17
18
-
Data labeling is a crucial step in development lifecycle; this data are used in the next step when training your model so that your model can learn from the labeled data. If you already have labeled utterances, you can directly [import it into your project](create-project.md#import-project), but you need to make sure that your data follows the [accepted data format](../concepts/data-formats.md). See [create project](create-project.md#import-project) to learn more about importing labeled data into your project. Labeled data informs the model how to interpret text, and is used for training and evaluation.
18
+
Data labeling is a crucial step in the conversational language understanding `CLU` trained development lifecycle; this data are used in the next step when training your model so that your model can learn from the labeled data. If you already labeled utterances, you can directly [import it into your project](create-project.md#import-project), IF your data follows the [accepted data format](../concepts/data-formats.md). See [create fine-tuning task](create-project.md#import-project) to learn more about importing labeled data. Labeled data informs the model how to interpret text and is used for training and evaluation.
19
+
20
+
> [!TIP]
21
+
> Use the `Quick Deploy` option to implement custom `CLU` intent routing, powered by your own `LLM` model deployment without adding or labeling any training data.
19
22
20
23
## Prerequisites
21
24
22
25
Before you can label your data, you need:
23
26
24
27
* A successfully [created project](create-project.md).
25
28
26
-
See the [project development lifecycle](../overview.md#project-development-lifecycle) for more information.
29
+
For more information, see the [conversational language understanding development lifecycle](../overview.md#project-development-lifecycle).
27
30
28
31
## Data labeling guidelines
29
32
30
-
After [building your schema](build-schema.md) and [creating your project](create-project.md), you need to label your data. Labeling your data is important so your model knows which words and sentences are associated with the intents and entities in your project. Spend time labeling your utterances - introducing and refining the data that is used in training your models.
33
+
After [building your schema](build-schema.md) and [creating your project](create-project.md), you need to label your data. Labeling your data is important so your model knows which sentences and words are associated with the intents and entities in your project. Spend time labeling your utterances - introducing and refining the data that is used in training your models.
31
34
32
35
As you add utterances and label them, keep in mind:
33
36
34
-
* The machine learning models generalize based on the labeled examples you provide it; the more examples you provide, the more data points the model has to make better generalizations.
37
+
* The machine learning models generalize based on the labeled examples you provide it; the more examples you provide, the more data points the model has to make better generalizations.
35
38
36
-
* The precision, consistency and completeness of your labeled data are key factors to determining model performance.
39
+
* The precision, consistency, and completeness of your labeled data are key factors to determining model performance.
37
40
38
41
***Label precisely**: Label each intent and entity to its right type always. Only include what you want classified and extracted, avoid unnecessary data in your labels.
39
42
***Label consistently**: The same entity should have the same label across all the utterances.
@@ -51,108 +54,109 @@ As you add utterances and label them, keep in mind:
51
54
52
55
Use the following steps to label your utterances:
53
56
54
-
1. Go to your project page in [Language Studio](https://aka.ms/languageStudio).
57
+
1. Go to your project page in [AI Foundry](https://ai.azure.com).
58
+
59
+
1. From the left side menu, select `Manage data`. In this page, you can start adding your utterances and labeling them. You can also upload your utterances directly by clicking on `Upload utterance file` from the top menu. Make sure it follows the [accepted format](../concepts/data-formats.md#utterance-file-format).
55
60
56
-
2. From the left side menu, select **Data labeling**. In this page, you can start adding your utterance and labeling them. You can also upload your utterance directly by clicking on **Upload utterance file** from the top menu, make sure it follows the [accepted format](../concepts/data-formats.md#utterance-file-format).
61
+
1. From the top pivots, you can change the view to be `training set` or `testing set`. Learn more about [training and testing sets](train-model.md#data-splitting) and how they're used for model training and evaluation.
57
62
58
-
3. From the top pivots, you can change the view to be **training set** or **testing set**. Learn more about [training and testing sets](train-model.md#data-splitting) and how they're used for model training and evaluation.
59
-
60
63
:::image type="content" source="../media/tag-utterances.png" alt-text="A screenshot of the page for tagging utterances in Language Studio." lightbox="../media/tag-utterances.png":::
61
-
64
+
62
65
> [!TIP]
63
-
> If you are planning on using **Automatically split the testing set from training data** splitting, add all your utterances to the training set.
64
-
65
-
66
-
4. From the **Select intent** dropdown menu, select one of the intents, the language of the utterance (for multilingual projects), and the utterance itself. Press the enter key in the utterance's text box to add the utterance.
66
+
> If you're planning on using `Automatically split the testing set from training data` splitting, add all your utterances to the training set.
67
+
68
+
69
+
1. From the `Select intent` dropdown menu, select one of the intents, the language of the utterance (for multilingual projects), and the utterance itself. Press enter key in the utterance's text box and add the utterance.
70
+
71
+
1. You have two options to label entities in an utterance:
67
72
68
-
5. You have two options to label entities in an utterance:
69
-
70
73
|Option |Description |
71
74
|---------|---------|
72
75
|Label using a brush | Select the brush icon next to an entity in the right pane, then highlight the text in the utterance you want to label. |
73
76
|Label using inline menu | Highlight the word you want to label as an entity, and a menu appears. Select the entity you want to label these words with. |
74
-
75
-
6. In the right side pane, under the **Labels** pivot, you can find all the entity types in your project and the count of labeled instances per each.
76
77
77
-
7. Under the **Distribution** pivot you can view the distribution across training and testing sets. You have two options for viewing:
78
+
1. In the right side pane, under the `Labels` pivot, you can find all the entity types in your project and the count of labeled instances per each.
79
+
80
+
1. Under the `Distribution` pivot, you can view the distribution across training and testing sets. You have two options for viewing:
78
81
**Total instances per labeled entity* where you can view count of all labeled instances of a specific entity.
79
82
**Unique utterances per labeled entity* where each utterance is counted if it contains at least one labeled instance of this entity.
80
83
**Utterances per intent* where you can view count of utterances per intent.
81
84
82
-
:::image type="content" source="../media/label-distribution.png" alt-text="A screenshot showing entity distribution in Language Studio." lightbox="../media/label-distribution.png":::
85
+
:::image type="content" source="../media/label-distribution.png" alt-text="A screenshot showing entity distribution in Language Studio." lightbox="../media/label-distribution.png":::
83
86
84
87
85
88
> [!NOTE]
86
-
> Listand prebuilt components are not shown in the data labeling page, and all labels here only apply to the **learned component**.
89
+
> List, regex, and prebuilt components aren't shown in the data labeling page, and all labels here only apply to the **learned component**.
87
90
88
91
To remove a label:
89
92
1. From within your utterance, select the entity you want to remove a label from.
90
-
3. Scroll through the menu that appears, and select **Remove label**.
93
+
1. Scroll through the menu that appears, and select **Remove label**.
91
94
92
95
To delete an entity:
93
-
1. Select the entity you want to edit in the right side pane.
94
-
2. Select the three dots next to the entity, and select the option you want from the drop-down menu.
96
+
1. Select the garbage bin icon next to the entity you want to edit in the right side pane. Then select **Delete** to confirm.
95
97
96
98
## Suggest utterances with Azure OpenAI
97
99
98
-
In CLU, use Azure OpenAI to suggest utterances to add to your project using GPT models. You first need to get access and create a resource in Azure OpenAI. You'll then need to create a deployment for the GPT models. Follow the pre-requisite steps [here](../../../openai/how-to/create-resource.md).
100
+
In `CLU`, use Azure OpenAI to suggest utterances to add to your project using generative language models. We recommended that you use an AI Foundry resource while using `CLU`, so you don't need to connect multiple resources. In order to use the AI Foundry resource, you need to provide your AI Foundry resource with elevated access. To do so, access the Azure portal. Within your Azure AI resource, provide access as a *Cognitive Services User* to itself. This step ensures that all parts of your resource are communicating correctly.
101
+
102
+
### Connect with separate Language and Azure OpenAI Resources
99
103
100
-
Before you get started, the suggest utterances feature is only available if your Language resource is in the following regions:
104
+
You first need to get access and create a resource in Azure OpenAI. Next, create a connection to the Azure OpenAI resource within the same AI Foundry project in the `Management center` in the left panel of the Azure AI Foundry page. You then need to create a deployment for the AOAI models within the connected AOAI resource. Follow the prerequisite steps [here](../../../openai/how-to/create-resource.md) to create a new resource.
105
+
106
+
Before you get started, the suggested utterances feature is only available if your Language resource is in the following regions:
101
107
* East US
102
108
* South Central US
103
109
* West Europe
104
110
105
-
In the Data Labeling page:
111
+
In the Data Labeling page:
106
112
107
-
1. Select the **Suggest utterances** button. A pane opens up on the right side prompting you to select your Azure OpenAI resource and deployment.
108
-
2. On selection of an Azure OpenAI resource, select **Connect**, which allows your Language resource to have direct access to your Azure OpenAI resource. It assigns your Language resource the role of `Cognitive Services User` to your Azure OpenAI resource, which allows your current Language resource to have access to Azure OpenAI's service. If the connection fails, follow these[steps](#add-required-configurations-to-azure-openai-resource)below to add the right role to your Azure OpenAI resource manually.
109
-
3. Once the resource is connected, select the deployment. The recommended model for the Azure OpenAI deployment is `gpt-35-turbo-instruct`.
110
-
4. Select the intent you'd like to get suggestions for. Make sure the intent you have selected has at least 5 saved utterances to be enabled for utterance suggestions. The suggestions provided by Azure OpenAI are based on the **most recent utterances** you've added for that intent.
111
-
5. Select **Generate utterances**. Once complete, the suggested utterances show up with a dotted line around it, with the note *Generated by AI*. Those suggestions need to be accepted or rejected. Accepting a suggestion simply adds it to your project, as if you had added it yourself. Rejecting it deletes the suggestion entirely. Only accepted utterances are part of your project and used for training or testing. You can accept or reject by clicking on the green check or red cancel buttons beside each utterance. You can also use the `Accept all` and `Reject all` buttons in the toolbar.
113
+
1. Select the `Suggest utterances` button. A pane opens up on the right side prompting you to select your Azure OpenAI resource and deployment.
114
+
1. On selection of an Azure OpenAI resource, select `Connect`, which allows your Language resource to have direct access to your Azure OpenAI resource. It assigns your Language resource the role of `Cognitive Services User` to your Azure OpenAI resource, which allows your current Language resource to have access to Azure OpenAI's service. If the connection fails, the following[steps](#add-required-configurations-to-azure-openai-resource)enable you to manually add the correct role to your Azure OpenAI resource.
115
+
1. Once the resource is connected, select the deployment. The recommended model for the Azure OpenAI deployment is `gpt-35-turbo-instruct`.
116
+
1. Select the intent you'd like to get suggestions for. Make sure the intent you selected has at least five saved utterances to be enabled for utterance suggestions. The suggestions provided by Azure OpenAI are based on the `most recent utterances` you added for that intent.
117
+
1. Select `Generate utterances`. Once complete, the suggested utterances show up with a dotted line around it, with the note *Generated by AI*. Those suggestions need to be accepted or rejected. Accepting a suggestion simply adds it to your project, as if you had added it yourself. Rejecting it deletes the suggestion entirely. Only accepted utterances are part of your project and used for training or testing. You can accept or reject by clicking on the green check or red cancel buttons beside each utterance. You can also use the `Accept all` and `Reject all` buttons in the toolbar.
112
118
113
-
:::image type="content" source="../media/suggest-utterances.png" alt-text="A screenshot showing utterance suggestions in Language Studio." lightbox="../media/suggest-utterances.png":::
Using this feature entails a charge to your Azure OpenAI resource for a similar number of tokens to the suggested utterances generated. Details for Azure OpenAI's pricing can be found [here](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/).
116
122
117
123
### Add required configurations to Azure OpenAI resource
118
124
119
-
If connecting your Language resource to an Azure OpenAI resource fails, follow these steps:
120
-
121
125
Enable identity management for your Language resource using the following options:
122
126
123
-
### [Azure portal](#tab/portal)
127
+
####[Azure portal](#tab/portal)
124
128
125
129
Your Language resource must have identity management, to enable it using the [Azure portal](https://portal.azure.com):
126
130
127
-
1. Go to your Language resource
128
-
2. From left hand menu, under **Resource Management** section, select **Identity**
129
-
3. From **System assigned** tab, make sure to set **Status** to **On**
131
+
1. Go to your Language resource.
132
+
1. From left hand menu, under `Resource Management` section, select `Identity`.
133
+
1. From `System assigned` tab, make sure to set `Status` to `On`.
130
134
131
-
### [Language Studio](#tab/studio)
135
+
####[Language Studio](#tab/studio)
132
136
133
137
Your Language resource must have identity management, to enable it using [Language Studio](https://aka.ms/languageStudio):
134
138
135
-
1. Select the settings icon in the top right corner of the screen
136
-
2. Select **Resources**
137
-
3. Select the check box **Managed Identity** for your Language resource.
139
+
1. Select the settings icon in the top right corner of the screen.
140
+
1. Select *`Resources`.
141
+
1. Select the check box `Managed Identity` for your Language resource.
138
142
139
143
---
140
144
141
-
After enabling managed identity, assign the role `Cognitive Services User` to your Azure OpenAI resource using the managed identity of your Language resource.
145
+
After enabling managed identity, assign the role `Cognitive Services User` to your Azure OpenAI resource using the managed identity of your Language resource.
142
146
143
147
1. Sign in to the [Azure portal](https://portal.azure.com) and navigate to your Azure OpenAI resource.
144
-
2. Select the Access Control (IAM) tab on the left.
145
-
3. Select Add > Add role assignment.
146
-
4. Select "Job function roles" and click Next.
147
-
5. Select `Cognitive Services User` from the list of roles and click Next.
148
-
6. Select Assign access to "Managed identity" and select "Select members".
149
-
7. Under "Managed identity" select "Language".
150
-
8. Search for your resource and select it. Then select the Select button below and next to complete the process.
151
-
9. Review the details and select Review + Assign.
152
-
153
-
:::image type="content" source="../media/add-role-azure-openai.gif" alt-text="Multiple screenshots showing the steps to add the required role to your Azure OpenAI resource." lightbox="../media/add-role-azure-openai.gif":::
154
-
155
-
After a few minutes, refresh the Language Studio and you are able to successfully connect to Azure OpenAI.
148
+
1. Select the `Access Control (IAM)` tab.
149
+
1. Select `Add` > Add role assignment.
150
+
1. Select `Job function roles` and select `Next`.
151
+
1. Select `Cognitive Services User` from the list of roles and select `Next`.
152
+
1. Select Assign access to "Managed identity" and select `Select members`.
153
+
1. Under `Managed identity` select `Language`.
154
+
1. Search for your resource and select it. Then select `Next`and complete the process.
155
+
1. Review the details and select `Review + Assign`.
156
+
157
+
:::image type="content" source="../media/add-role-azure-openai.gif" alt-text="Multiple screenshots showing the steps to add the required role to your Azure OpenAI resource." lightbox="../media/add-role-azure-openai.gif":::
158
+
159
+
After a few minutes, refresh the AI Foundry, and you can successfully connect to Azure OpenAI.
0 commit comments