You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/concepts/data-formats.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -147,7 +147,7 @@ Your Labels file should be in the `json` format below to be used when importing
147
147
|`prebuilts`|`MedicationName`| The name of the prebuilt component populating the prebuilt entity. [Prebuilt entities](../../text-analytics-for-health/concepts/health-entity-categories.md) are automatically loaded into your project by default but you can extend them with list components in your labels file. |`MedicationName`|
148
148
|`documents`|| Array containing all the documents in your project and list of the entities labeled within each document. |[]|
149
149
|`location`|`{DOCUMENT-NAME}`| The location of the documents in the storage container. Since all the documents are in the root of the container this should be the document name.|`doc1.txt`|
150
-
|`dataset`|`{DATASET}`| The test set to which this file goes to when split before training. <!--Learn more about data splitting [here](../how-to/train-model.md#data-splitting).--> Possible values for this field are `Train` and `Test`. |`Train`|
150
+
|`dataset`|`{DATASET}`| The test set to which this file goes to when split before training. Learn more about data splitting [here](../how-to/train-model.md#data-splitting). Possible values for this field are `Train` and `Test`. |`Train`|
151
151
|`regionOffset`|| The inclusive character position of the start of the text. |`0`|
152
152
|`regionLength`|| The length of the bounding box in terms of UTF16 characters. Training only considers the data in this region. |`500`|
153
153
|`category`|| The type of entity associated with the span of text specified. |`Entity1`|
@@ -166,4 +166,4 @@ Your Labels file should be in the `json` format below to be used when importing
166
166
## Next steps
167
167
* You can import your labeled data into your project directly. Learn how to [import project](../how-to/create-project.md#import-project)
168
168
* See the [how-to article](../how-to/label-data.md) more information about labeling your data.
169
-
<!--* When you're done labeling your data, you can [train your model](../how-to/train-model.md).-->
169
+
* When you're done labeling your data, you can [train your model](../how-to/train-model.md).
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/concepts/evaluation-metrics.md
# Evaluation metrics for custom Text Analytics for health models
17
17
18
-
<!--Your [dataset is split](../how-to/train-model.md#data-splitting) into two parts: a set for training, and a set for testing. The training set is used to train the model, while the testing set is used as a test for model after training to calculate the model performance and evaluation. The testing set is not introduced to the model through the training process, to make sure that the model is tested on new data.-->
18
+
Your [dataset is split](../how-to/train-model.md#data-splitting) into two parts: a set for training, and a set for testing. The training set is used to train the model, while the testing set is used as a test for model after training to calculate the model performance and evaluation. The testing set is not introduced to the model through the training process, to make sure that the model is tested on new data.
19
19
20
20
Model evaluation is triggered automatically after training is completed successfully. The evaluation process starts by using the trained model to predict user defined entities for documents in the test set, and compares them with the provided data labels (which establishes a baseline of truth). The results are returned so you can review the model’s performance. User defined entities are **included** in the evaluation factoring in Learned and List components; Text Analytics for health prebuilt entities are **not** factored in the model evaluation. For evaluation, custom Text Analytics for health uses the following metrics:
21
21
@@ -149,7 +149,5 @@ Similarly,
149
149
## Next steps
150
150
151
151
*[Custom text analytics for health overview](../overview.md)
152
-
<!--
153
152
*[View a model's performance in Language Studio](../how-to/view-model-evaluation.md)
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/how-to/label-data.md
+4-6Lines changed: 4 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ Data labeling is a crucial step in development lifecycle. In this step, you lab
22
22
Before you can label your data, you need:
23
23
24
24
* A successfully [created project](create-project.md) with a configured Azure blob storage account
25
-
<!--* Text data that [has been uploaded](design-schema.md#data-preparation) to your storage account.-->
25
+
* Text data that [has been uploaded](design-schema.md#data-preparation) to your storage account.
26
26
27
27
See the [project development lifecycle](../overview.md#project-development-lifecycle) for more information.
28
28
@@ -34,7 +34,7 @@ As you label your data, keep in mind:
34
34
35
35
* You can't add labels for Text Analytics for health entities as they're pretrained prebuilt entities. You can only add labels to new entity categories that you defined during schema definition.
36
36
37
-
<!--If you want to improve the recall for a prebuilt entity, you can extend it by adding a list component while you are [defining your schema](design-schema.md).-->
37
+
If you want to improve the recall for a prebuilt entity, you can extend it by adding a list component while you are [defining your schema](design-schema.md).
38
38
39
39
* In general, more labeled data leads to better results, provided the data is labeled accurately.
40
40
@@ -83,7 +83,7 @@ Use the following steps to label your data:
83
83
84
84
6. In the right side pane under the **Labels** pivot you can find all the entity types in your project and the count of labeled instances per each. The prebuilt entities will be shown for reference but you will not be able to label for these prebuilt entities as they are pretrained.
85
85
86
-
7. In the bottom section of the right side pane you can add the current document you are viewing to the training set or the testing set. By default all the documents are added to your training set. <!--Learn more about [training and testing sets](train-model.md#data-splitting) and how they are used for model training and evaluation.-->
86
+
7. In the bottom section of the right side pane you can add the current document you are viewing to the training set or the testing set. By default all the documents are added to your training set. See [training and testing sets](train-model.md#data-splitting)for information on how they are used for model training and evaluation.
87
87
88
88
> [!TIP]
89
89
> If you are planning on using **Automatic** data splitting, use the default option of assigning all the documents into your training set.
@@ -107,6 +107,4 @@ You cannot delete any of the Text Analytics for health pretrained entities becau
107
107
108
108
## Next steps
109
109
110
-
[Custom text analytics for health overview](../overview.md)
111
-
112
-
<!--After you've labeled your data, you can begin [training a model](train-model.md) that will learn based on your data.-->
110
+
After you've labeled your data, you can begin [training a model](train-model.md) that will learn based on your data.
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/how-to/view-model-evaluation.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ Before viewing model evaluation, you need:
28
28
* A successfully [created project](create-project.md) with a configured Azure blob storage account.
29
29
* Text data that [has been uploaded](design-schema.md#data-preparation) to your storage account.
30
30
*[Labeled data](label-data.md)
31
-
<!--* A [successfully trained model](train-model.md)-->
31
+
* A [successfully trained model](train-model.md)
32
32
33
33
34
34
## Model details
@@ -72,5 +72,5 @@ There are several metrics you can use to evaluate your mode. See the [performanc
72
72
73
73
## Next steps
74
74
75
-
<!--* [Deploy your model](deploy-model.md)-->
75
+
*[Deploy your model](deploy-model.md)
76
76
* Learn about the [metrics used in evaluation](../concepts/evaluation-metrics.md).
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/language-studio/test-model.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ To test your deployed models from within the [Language Studio](https://aka.ms/La
21
21
22
22
5. Click on **Run the test** from the top menu.
23
23
24
-
6. In the **Result** tab, you can see the extracted entities from your text and their types. You can also view the JSON response under the **JSON** tab. <!--[learn more](../rest-api/get-results.md#response-body) about the structure of the JSON response.-->
24
+
6. In the **Result** tab, you can see the extracted entities from your text and their types. You can also view the [JSON response](../../how-to/call-api.md#get-task-results) under the **JSON** tab.
25
25
26
26
:::image type="content" source="../../media/test-model-results.png" alt-text="A screenshot showing the deployment testing screen in Language Studio." lightbox="../../media/test-model-results.png":::
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/language-studio/train-model.md
+1-3Lines changed: 1 addition & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,9 +20,7 @@ To start training your model from within the [Language Studio](https://aka.ms/La
20
20
21
21
:::image type="content" source="../../media/train-model.png" alt-text="A screenshot showing the training job creation screen in Language Studio." lightbox="../../media/train-model.png":::
22
22
23
-
4. Select data splitting method. You can choose **Automatically splitting the testing set from training data** where the system will split your labeled data between the training and testing sets, according to the specified percentages. Or you can **Use a manual split of training and testing data**, this option is only enabled if you have added documents to your testing set.
24
-
25
-
<!-- see [data labeling](../../how-to/tag-data.md) and [how to train a model](../../how-to/train-model.md#data-splitting) for information about data splitting. -->
23
+
4. Select data splitting method. You can choose **Automatically splitting the testing set from training data** where the system will split your labeled data between the training and testing sets, according to the specified percentages. Or you can **Use a manual split of training and testing data**, this option is only enabled if you have added documents to your testing set. See [data labeling](../../how-to/label-data.md) and [how to train a model](../../how-to/train-model.md#data-splitting) for information about data splitting.
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/quickstarts/language-studio.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,13 +19,13 @@ ms.author: aahi
19
19
20
20
## Create a new Azure Language resource and Azure storage account
21
21
22
-
Before you can use custom Text Analytics for health, you'll need to create an Azure Language resource, which will give you the credentials that you need to create a project and start training a model. You'll also need an Azure storage account, where you can upload your dataset that will be used to build your model.
22
+
Before you can use custom Text Analytics for health, you need to create an Azure Language resource, which will give you the credentials that you need to create a project and start training a model. You'll also need an Azure storage account, where you can upload your dataset that is used to build your model.
23
23
24
24
> [!IMPORTANT]
25
25
> To quickly get started, we recommend creating a new Azure Language resource using the steps provided in this article. Using the steps in this article will let you create the Language resource and storage account at the same time, which is easier than doing it later.
26
26
>
27
-
<!-- If you have a pre-existing resource that you'd like to use, you will need to connect it to storage account. See [guidance to using a pre-existing resource](../../how-to/create-project.md#using-a-pre-existing-language-resource) for information.
28
-
-->
27
+
>If you have a pre-existing resource that you'd like to use, you will need to connect it to storage account. For more information, see [guidance to using a pre-existing resource](../../how-to/create-project.md#using-a-pre-existing-language-resource).
28
+
29
29
[!INCLUDE [create a new resource from the Azure portal](../resource-creation-azure-portal.md)]
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/quickstarts/rest-api.md
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,10 +22,9 @@ Before you can use custom Text Analytics for health, you'll need to create an Az
22
22
23
23
> [!IMPORTANT]
24
24
> To get started quickly, we recommend creating a new Azure Language resource using the steps provided in this article, which will let you create the Language resource, and create and/or connect a storage account at the same time, which is easier than doing it later.
25
+
>
26
+
> If you have a pre-existing resource that you'd like to use, you will need to connect it to storage account. See [create project](../../how-to/create-project.md#using-a-pre-existing-language-resource) for more information.
25
27
26
-
<!--
27
-
> If you have a pre-existing resource that you'd like to use, you will need to connect it to storage account. See [create project](../../how-to/create-project.md#using-a-pre-existing-language-resource) for information.
28
-
-->
29
28
[!INCLUDE [create a new resource from the Azure portal](../resource-creation-azure-portal.md)]
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/rest-api/create-project.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,9 +57,9 @@ Use the following JSON in your request. Replace the placeholder values below wit
57
57
|Key |Placeholder|Value | Example |
58
58
|---------|---------|---------|--|
59
59
| projectName |`{PROJECT-NAME}`| The name of your project. This value is case-sensitive. |`myProject`|
60
-
| language |`{LANGUAGE-CODE}`| A string specifying the language code for the documents used in your project. If your project is a multilingual project, choose the language code of the majority of the documents. <!--See [language support](../../language-support.md) to learn more about supported language codes.-->|`en-us`|
60
+
| language |`{LANGUAGE-CODE}`| A string specifying the language code for the documents used in your project. If your project is a multilingual project, choose the language code of the majority of the documents. See [language support](../../language-support.md) to learn more about supported language codes. |`en-us`|
61
61
| projectKind |`CustomHealthcare`| Your project kind. |`CustomHealthcare`|
62
-
| multilingual |`true`| A boolean value that enables you to have documents in multiple languages in your dataset and when your model is deployed you can query the model in any supported language (not necessarily included in your training documents. <!--See [language support](../../language-support.md) to learn more about multilingual support.-->|`true`|
62
+
| multilingual |`true`| A boolean value that enables you to have documents in multiple languages in your dataset and when your model is deployed you can query the model in any supported language (not necessarily ones included in your training documents). See [language support](../../language-support.md) to learn more about multilingual support. |`true`|
63
63
| storageInputContainerName |`{CONTAINER-NAME`| The name of your Azure storage container where you have uploaded your documents. |`myContainer`|
Copy file name to clipboardExpand all lines: articles/cognitive-services/language-service/custom-text-analytics-for-health/includes/rest-api/get-results.md
+6-7Lines changed: 6 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -270,7 +270,7 @@ The response is a JSON document with the following parameters
270
270
}
271
271
272
272
```
273
-
<!--
273
+
274
274
|Key|Sample Value|Description|
275
275
|--|--|--|
276
276
|entities|[]|An array containing all the extracted entities.|
@@ -280,12 +280,11 @@ The response is a JSON document with the following parameters
280
280
|text|`first entity`| The text that was extracted for a specific entity.|
281
281
|category|`MedicationName`| The name of the entity type or category corresponding to the extracted text.|
282
282
|confidenceScore|`0.9`| A number denoting the model's certainty level of the extracted entity ranging from 0 to 1 with higher number denoting higher certainty.|
283
-
|assertion|`certainty`| [Assertions](../../../text-analytics-for-health/concepts/assertion-detection.md) associated with the extracted entity. Assertions are only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
284
-
|name|`Ibuprofen`| The normalized name for the [entity linking](../../../text-analytics-for-health/includes/features.md#entity-linking) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
285
-
|links| [] | An array containing all the results from the [entity linking](../../../text-analytics-for-health/includes/features.md#entity-linking) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
286
-
|dataSource| `UMLS` | The reference standard resulting from the [entity linking](../../../text-analytics-for-health/includes/features.md#entity-linking) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
287
-
|id| `C0020740` | The reference code resulting from the [entity linking](../../../text-analytics-for-health/includes/features.md#entity-linking) associated with the extracted entity belonging to the extracted data source. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
283
+
|assertion|`certainty`|[Assertions](../../../text-analytics-for-health/concepts/assertion-detection.md) associated with the extracted entity. Assertions are only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/overview.md?tabs=entity-linking#text-analytics-for-health-features).|
284
+
|name|`Ibuprofen`| The normalized name for the [entity linking](../../../text-analytics-for-health/overview.md?tabs=entity-linking#text-analytics-for-health-features) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
285
+
|links|[]| An array containing all the results from the [entity linking](../../../text-analytics-for-health/overview.md?tabs=entity-linking#text-analytics-for-health-features) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
286
+
|dataSource|`UMLS`| The reference standard resulting from the [entity linking](../../../text-analytics-for-health/overview.md?tabs=entity-linking#text-analytics-for-health-features) associated with the extracted entity. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
287
+
|ID|`C0020740`| The reference code resulting from the [entity linking](../../../text-analytics-for-health/overview.md?tabs=entity-linking#text-analytics-for-health-features) associated with the extracted entity belonging to the extracted data source. Entity linking is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
288
288
|relations|[]| Array containing all the extracted relationships. [Relationship extraction](../../../text-analytics-for-health/concepts/relation-extraction.md) is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
289
289
|relationType|`DosageOfMedication`| The category of the extracted [relationship](../../../text-analytics-for-health/concepts/relation-extraction.md). Relationship extraction is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
290
290
|entities|`"Dosage", "Medication"`| The entities associated with the extracted relationship. Relationship extraction is only supported for prebuilt [Text Analytics for health entities](../../../text-analytics-for-health/concepts/health-entity-categories.md).|
0 commit comments