Merge pull request #199120 from aahill/fast-follow-updates

terminology, json, toc updates

Author: Court72

articles/cognitive-services/language-service/concepts/model-lifecycle.md

@@ -44,6 +44,8 @@ For asynchronous endpoints, use the `model-version` property in the request body
 
 The model-version used in your API request will be included in the response object.
 
+> [!NOTE]
+> If you are using an model version that is not listed in the table, then it was subjected to the expiration policy.
 
 Use the table below to find which model versions are supported by each feature:
 

articles/cognitive-services/language-service/custom-named-entity-recognition/concepts/data-formats.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.service: cognitive-services
 ms.subservice: language-service
 ms.topic: conceptual
-ms.date: 05/06/2022
+ms.date: 05/24/2022
 ms.author: aahi
 ms.custom: language-service-custom-ner, ignite-fall-2021, event-tier1-build-2022
 ---
@@ -23,62 +23,79 @@ Your Labels file should be in the `json` format below to be used in [importing](
 
 ```json
 {
+  "projectFileVersion": "2022-05-01",
+  "stringIndexType": "Utf16CodeUnit",
+  "metadata": {
+    "projectKind": "CustomEntityRecognition",
+    "storageInputContainerName": "{CONTAINER-NAME}",
+    "projectName": "{PROJECT-NAME}",
+    "multilingual": false,
+    "description": "Project-description",
+    "language": "en-us"
+  },
+  "assets": {
+    "projectKind": "CustomEntityRecognition",
     "entities": [
-        {
-            "category": "Entity1"
-        },
-        {
-            "category": "Entity2"
-        }
+      {
+        "category": "Entity1"
+      },
+      {
+        "category": "Entity2"
+      }
     ],
     "documents": [
-        {
-            "location": "{DOCUMENT-NAME}",
-            "language": "{LANGUAGE-CODE}",
-            "dataset": "{DATASET}",
-            "entities": [
-                {
-                    "regionOffset": 0,
-                    "regionLength": 500,
-                    "labels": [
-                        {
-                            "category": "Entity1",
-                            "offset": 25,
-                            "length": 10
-                        },
-                        {
-                            "category": "Entity2",
-                            "offset": 120,
-                            "length": 8
-                        }
-                    ]
-                }
+      {
+        "location": "{DOCUMENT-NAME}",
+        "language": "{LANGUAGE-CODE}",
+        "dataset": "{DATASET}",
+        "entities": [
+          {
+            "regionOffset": 0,
+            "regionLength": 500,
+            "labels": [
+              {
+                "category": "Entity1",
+                "offset": 25,
+                "length": 10
+              },
+              {
+                "category": "Entity2",
+                "offset": 120,
+                "length": 8
+              }
             ]
-        },
-        {
-            "location": "{DOCUMENT-NAME}",
-            "language": "{LANGUAGE-CODE}",
-            "dataset": "{DATASET}",
-            "entities": [
-                {
-                    "regionOffset": 0,
-                    "regionLength": 100,
-                    "labels": [
-                        {
-                            "category": "Entity2",
-                            "offset": 20,
-                            "length": 5
-                        }
-                    ]
-                }
+          }
+        ]
+      },
+      {
+        "location": "{DOCUMENT-NAME}",
+        "language": "{LANGUAGE-CODE}",
+        "dataset": "{DATASET}",
+        "entities": [
+          {
+            "regionOffset": 0,
+            "regionLength": 100,
+            "labels": [
+              {
+                "category": "Entity2",
+                "offset": 20,
+                "length": 5
+              }
             ]
-        }
+          }
+        ]
+      }
     ]
+  }
 }
+
 ```
 
 |Key  |Placeholder  |Value  | Example |
 |---------|---------|----------|--|
+| `multilingual` | `true`| A boolean value that enables you to have documents in multiple languages in your dataset and when your model is deployed you can query the model in any supported language (not necessarily included in your training documents). See [language support](../language-support.md#multi-lingual-option) to learn more about multilingual support. | `true`|
+|`projectName`|`{PROJECT-NAME}`|Project name|`myproject`|
+| storageInputContainerName|`{CONTAINER-NAME}`|Container name|`mycontainer`|
 | `entities` | | Array containing all the entity types you have in the project. These are the entity types that will be extracted from your documents into.|  |
 | `documents` | | Array containing all the documents in your project and list of the entities labeled within each document. | [] |
 | `location` | `{DOCUMENT-NAME}` |  The location of the documents in the storage container. Since all the documents are in the root of the container this should be the document name.|`doc1.txt`|

articles/cognitive-services/language-service/custom-named-entity-recognition/concepts/evaluation-metrics.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.service: cognitive-services
 ms.subservice: language-service
 ms.topic: conceptual 
-ms.date: 05/06/2022
+ms.date: 05/24/2022
 ms.author: aahi
 ms.custom: language-service-custom-ner, ignite-fall-2021, event-tier1-build-2022
 ---
@@ -133,5 +133,5 @@ Similarly,
 
 ## Next steps
 
-* [View a model's evaluation in Language Studio](../how-to/view-model-evaluation.md)
+* [View a model's performance in Language Studio](../how-to/view-model-evaluation.md)
 * [Train a model](../how-to/train-model.md)

articles/cognitive-services/language-service/custom-named-entity-recognition/glossary.md

@@ -30,7 +30,7 @@ For example, in the sentence "*John borrowed 25,000 USD from Fred.*" the entitie
 | Loan Amount | *25,000 USD* |
 
 ## F1 score
-The F1 score is a function of Precision and Recall. It's needed when you seek a balance between [precision](#precision) and [recall](#recall].
+The F1 score is a function of Precision and Recall. It's needed when you seek a balance between [precision](#precision) and [recall](#recall).
 
 ## Model
 

articles/cognitive-services/language-service/custom-named-entity-recognition/how-to/call-api.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.service: cognitive-services
 ms.subservice: language-service
 ms.topic: how-to
-ms.date: 05/09/2022
+ms.date: 05/24/2022
 ms.author: aahi
 ms.devlang: csharp, python
 ms.custom: language-service-custom-ner, event-tier1-build-2022
@@ -17,7 +17,7 @@ ms.custom: language-service-custom-ner, event-tier1-build-2022
 # Query deployment to extract entities
 
 After the deployment is added successfully, you can query the deployment to extract entities from your text based on the model you assigned to the deployment.
-You can query the deployment programmatically using the [Prediction API](https://aka.ms/ct-runtime-swagger) or through the [Client libraries (Azure SDK)](#get-task-results). 
+You can query the deployment programmatically using the [Prediction API](https://aka.ms/ct-runtime-api) or through the [Client libraries (Azure SDK)](#get-task-results). 
 
 ## Test deployed model
 
@@ -80,4 +80,7 @@ First you will need to get your resource key and endpoint:
 
 ## Next steps
 
-* [Custom NER overview](../overview.md)
+* [Enrich a Cognitive Search index tutorial](../tutorials/cognitive-search.md)
+
+
+

articles/cognitive-services/language-service/custom-named-entity-recognition/how-to/create-project.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.service: cognitive-services
 ms.subservice: language-service
 ms.topic: how-to
-ms.date: 05/06/2022
+ms.date: 05/24/2022
 ms.author: aahi
 ms.custom: language-service-custom-ner, references_regions, ignite-fall-2021, event-tier1-build-2022
 ---
@@ -112,4 +112,4 @@ If you have already labeled data, you can use it to get started with the service
 
 * You should have an idea of the [project schema](design-schema.md) you will use to label your data.
 
-* After your project is created, you can start [tagging your data](tag-data.md), which will inform your entity extraction model how to interpret text, and is used for training and evaluation.
+* After your project is created, you can start [labeling your data](tag-data.md), which will inform your entity extraction model how to interpret text, and is used for training and evaluation.

articles/cognitive-services/language-service/custom-named-entity-recognition/how-to/design-schema.md

@@ -29,13 +29,13 @@ The schema defines the entity types/categories that you need your model to extra
 
 * Avoid entity types ambiguity.
 
-    **Ambiguity** happens when entity types you select are similar to each other. The more ambiguous your schema the more tagged data you will need to differentiate between different entity types.
+    **Ambiguity** happens when entity types you select are similar to each other. The more ambiguous your schema the more labeled data you will need to differentiate between different entity types.
 
     For example, if you are extracting data from a legal contract, to extract "Name of first party" and "Name of second party" you will need to add more examples to overcome ambiguity since the names of both parties look similar. Avoid ambiguity as it saves time, effort, and yields better results.
 
 * Avoid complex entities. Complex entities can be difficult to pick out precisely from text, consider breaking it down into multiple entities.
 
-    For example, extracting "Address" would be challenging if it's not broken down to smaller entities. There are so many variations of how addresses appear, it would take large number of tagged entities to teach the model to extract an address, as a whole, without breaking it down. However, if you replace "Address" with "Street Name", "PO Box", "City", "State" and "Zip", the model will require fewer tags per entity.
+    For example, extracting "Address" would be challenging if it's not broken down to smaller entities. There are so many variations of how addresses appear, it would take large number of labeled entities to teach the model to extract an address, as a whole, without breaking it down. However, if you replace "Address" with "Street Name", "PO Box", "City", "State" and "Zip", the model will require fewer labels per entity.
 
 ## Data selection
 
@@ -61,10 +61,14 @@ As a prerequisite for creating a project, your training data needs to be uploade
 * [Create and upload documents from Azure](../../../../storage/blobs/storage-quickstart-blobs-portal.md#create-a-container)
 * [Create and upload documents using Azure Storage Explorer](../../../../vs-azure-tools-storage-explorer-blobs.md)
 
-You can only use `.txt` documents. If your data is in other format, you can use [CLUtils parse command](https://github.com/microsoft/CognitiveServicesLanguageUtilities/blob/main/CustomTextAnalytics.CLUtils/Solution/CogSLanguageUtilities.ViewLayer.CliCommands/Commands/ParseCommand/README.md) to change your file format.
+You can only use `.txt` documents. If your data is in other format, you can use [CLUtils parse command](https://github.com/microsoft/CognitiveServicesLanguageUtilities/blob/main/CustomTextAnalytics.CLUtils/Solution/CogSLanguageUtilities.ViewLayer.CliCommands/Commands/ParseCommand/README.md) to change your document format.
 
-You can upload an annotated dataset, or you can upload an unannotated one and [tag your data](../how-to/tag-data.md) in Language studio. 
+You can upload an annotated dataset, or you can upload an unannotated one and [label your data](../how-to/tag-data.md) in Language studio. 
 
+## Test set
+
+When defining the testing set, make sure to include example documents that are not present in the training set. Defining the testing set is an important step to calculate the [model performance](view-model-evaluation.md#model-details). Also, make sure that the testing set include documents that represent all entities used in your project.
+
 ## Next steps
 
 If you haven't already, create a custom NER project. If it's your first time using custom NER, consider following the [quickstart](../quickstart.md) to create an example project. You can also see the [how-to article](../how-to/create-project.md) for more details on what you need to create a project.

articles/cognitive-services/language-service/custom-named-entity-recognition/how-to/improve-model.md

@@ -15,13 +15,13 @@ ms.custom: language-service-custom-ner, ignite-fall-2021, event-tier1-build-2022
 
 # Improve model performance
 
-In some cases, the model is expected to extract entities that are inconsistent with your tagged ones. In this page you can observe these inconsistencies and decide on the needed changes needed to improve your model performance.
+In some cases, the model is expected to extract entities that are inconsistent with your labeled ones. In this page you can observe these inconsistencies and decide on the needed changes needed to improve your model performance.
 
 ## Prerequisites
 
 * A successfully [created project](create-project.md) with a configured Azure blob storage account
     * Text data that [has been uploaded](design-schema.md#data-preparation) to your storage account.
-* [Tagged data](tag-data.md)
+* [Labeled data](tag-data.md)
 * A [successfully trained model](train-model.md)
 * Reviewed the [model evaluation details](view-model-evaluation.md) to determine how your model is performing.
 * Familiarized yourself with the [evaluation metrics](../concepts/evaluation-metrics.md).
@@ -31,7 +31,7 @@ See the [project development lifecycle](../overview.md#project-development-lifec
 
 ## Review test set predictions
 
-After you have viewed your [model's evaluation](view-model-evaluation.md), you'll have formed an idea on your model performance. In this page, you can view how your model performs vs how it's expected to perform. You can view predicted and tagged entities side by side for each document in your test set. You can review entities that were extracted differently than they were originally tagged.
+After you have viewed your [model's evaluation](view-model-evaluation.md), you'll have formed an idea on your model performance. In this page, you can view how your model performs vs how it's expected to perform. You can view predicted and labeled entities side by side for each document in your test set. You can review entities that were extracted differently than they were originally labeled.
 
 
 To review inconsistent predictions in the [test set](train-model.md) from within the [Language Studio](https://aka.ms/LanguageStudio):
@@ -42,15 +42,15 @@ To review inconsistent predictions in the [test set](train-model.md) from within
 
 3. For easier analysis, you can toggle **Show incorrect predictions only** to view entities that were incorrectly predicted only. You should see all documents that include entities that were incorrectly predicted.
 
-5. You can expand each document to see more details about predicted and tagged entities.
+5. You can expand each document to see more details about predicted and labeled entities.
 
     Use the following information to help guide model improvements. 
 
-    * If entity `X` is constantly identified as entity `Y`, it means that there is ambiguity between these entity types and you need to reconsider your schema. Learn more about [data selection and schema design](design-schema.md#schema-design). Another solution is to consider tagging more instances of these entities, to help the model improve and differentiate between them.
+    * If entity `X` is constantly identified as entity `Y`, it means that there is ambiguity between these entity types and you need to reconsider your schema. Learn more about [data selection and schema design](design-schema.md#schema-design). Another solution is to consider labeling more instances of these entities, to help the model improve and differentiate between them.
 
     * If a complex entity is repeatedly not predicted, consider [breaking it down to simpler entities](design-schema.md#schema-design) for easier extraction. 
 
-    * If an entity is predicted while it was not tagged in your data, this means to you need to review your tags. Be sure that all instances of an entity are properly tagged in all documents.
+    * If an entity is predicted while it was not labeled in your data, this means to you need to review your labels. Be sure that all instances of an entity are properly labeled in all documents.
 
 
     :::image type="content" source="../media/review-predictions.png" alt-text="A screenshot showing model predictions in Language Studio." lightbox="../media/review-predictions.png":::