Merge pull request #214615 from aahill/hazem-oct-update

Hazem oct update

Author: 19BMG00

articles/cognitive-services/language-service/conversational-language-understanding/concepts/data-formats.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.service: cognitive-services
 ms.subservice: language-service
 ms.topic: conceptual
-ms.date: 05/13/2022
+ms.date: 10/14/2022
 ms.author: aahi
 ms.custom: language-service-custom-clu
 ---
@@ -24,14 +24,17 @@ If you're [importing a project](../how-to/create-project.md#import-project) into
 
 ```json
 {
-  "projectFileVersion": "2022-05-01",
+  "projectFileVersion": "2022-10-01-preview",
   "stringIndexType": "Utf16CodeUnit",
   "metadata": {
     "projectKind": "Conversation",
     "projectName": "{PROJECT-NAME}",
     "multilingual": true,
     "description": "DESCRIPTION",
-    "language": "{LANGUAGE-CODE}"
+    "language": "{LANGUAGE-CODE}",
+    "settings": {
+            "confidenceThreshold": 0
+        }
   },
   "assets": {
     "projectKind": "Conversation",
@@ -43,7 +46,7 @@ If you're [importing a project](../how-to/create-project.md#import-project) into
     "entities": [
       {
         "category": "entity1",
-        "compositionSetting": "requireExactOverlap",
+        "compositionSetting": "{COMPOSITION-SETTING}",
         "list": {
           "sublists": [
             {
@@ -61,8 +64,20 @@ If you're [importing a project](../how-to/create-project.md#import-project) into
         },
         "prebuilts": [
           {
-            "category": "PREBUILT1"
+            "category": "{PREBUILT-COMPONENTS}"
           }
+        ],
+        "regex": {
+          "expressions": [
+              {
+                  "regexKey": "regex1",
+                  "language": "{LANGUAGE-CODE}",
+                  "regexPattern": "{REGEX-PATTERN}"
+              }
+          ]
+        },
+        "requiredComponents": [
+            "{REQUIRED-COMPONENTS}"
         ]
       }
     ],
@@ -89,19 +104,28 @@ If you're [importing a project](../how-to/create-project.md#import-project) into
 |Key  |Placeholder  |Value  | Example |
 |---------|---------|----------|--|
 | `api-version` | `{API-VERSION}`     | The version of the API you're calling. The value referenced here is for the latest released [model version](../../concepts/model-lifecycle.md#choose-the-model-version-used-on-your-data) released.  | `2022-05-01` |
-|`confidenceThreshold`|`{CONFIDENCE-THRESHOLD}`|This is the threshold score below which the intent will be predicted as [none intent](none-intent.md)|`0.7`|
+|`confidenceThreshold`|`{CONFIDENCE-THRESHOLD}`|This is the threshold score below which the intent will be predicted as [none intent](none-intent.md). Values are from `0` to `1`|`0.7`|
 | `projectName` | `{PROJECT-NAME}` | The name of your project. This value is case-sensitive. | `EmailApp` |
-| `multilingual` | `true`| A boolean value that enables you to have documents in multiple languages in your dataset and when your model is deployed you can query the model in any supported language (not necessarily included in your training documents. See [Language support](../language-support.md#multi-lingual-option) for more information about supported language codes.  | `true`|
-|`sublists`|`[]`|Array containing a sublists|`[]`|
+| `multilingual` | `true`| A boolean value that enables you to have utterances in multiple languages in your dataset and when your model is deployed you can query the model in any supported language (not necessarily included in your training documents. See [Language support](../language-support.md#multi-lingual-option) for more information about supported language codes.  | `true`|
+|`sublists`|`[]`|Array containing sublists. Each sublist is a key and its associated values.|`[]`|
+|`compositionSetting`|`{COMPOSITION-SETTING}`|Rule that defines how to manage multiple components in your entity. Options are `combineComponents` or `separateComponents`. |`combineComponents`|
 |`synonyms`|`[]`|Array containing all the synonyms|synonym|
-| `language` | `{LANGUAGE-CODE}` |  A string specifying the language code for the utterances used in your project. If your project is a  multilingual project, choose the [language code](../language-support.md) of the majority of the utterances. |`en-us`|
-| `intents` | `[]` | Array containing all the intents you have in the project. These are the intent types that will be extracted from your utterances.| `[]` |
-| `entities` | `[]` | Array containing all the entities in your project. These are the entities that will be extracted from your utterances.| `[]` |
+| `language` | `{LANGUAGE-CODE}` |  A string specifying the language code for the utterances, synonyms, and regular expressions used in your project. If your project is a  multilingual project, choose the [language code](../language-support.md) of the majority of the utterances. |`en-us`|
+| `intents` | `[]` | Array containing all the intents you have in the project. These are the intents that will be classified from your utterances.| `[]` |
+| `entities` | `[]` | Array containing all the entities in your project. These are the entities that will be extracted from your utterances. Every entity can have additional optional components defined with them: list, prebuilt, or regex. | `[]` |
 | `dataset` | `{DATASET}` |  The test set to which this utterance will go to when split before training. Learn more about data splitting [here](../how-to/train-model.md#data-splitting) . Possible values for this field are `Train` and `Test`.      |`Train`|
 | `category` | ` ` |  The type of entity associated with the span of text specified. | `Entity1`|
 | `offset` | ` ` |  The inclusive character position of the start of the entity.      |`5`|
 | `length` | ` ` |  The character length of the entity.      |`5`|
-| `language` | `{LANGUAGE-CODE}` |  A string specifying the language code for the utterances used in your project. If your project is a multilingual project, choose the [language code](../language-support.md) of the majority of the utterances. |`en-us`|
+| `listKey`| ` ` | A normalized value for the list of synonyms to map back to in prediction. | `Microsoft` |
+| `values`| `{VALUES-FOR-LIST}` | A list of comma separated strings that will be matched exactly for extraction and map to the list key. | `"msft", "microsoft", "MS"` |
+| `regexKey`| `{REGEX-PATTERN}` | A regular expression. | `ProductPattern1` |
+| `regexPattern`| `{REGEX-PATTERN}` | A regular expression. | `^pre` |
+| `prebuilts`| `{PREBUILT-COMPONENTS}` | The prebuilt components that can extract common types. You can find the list of prebuilts you can add [here](../prebuilt-component-reference.md). | `Quantity.Number` |
+| `requiredComponents` | `{REQUIRED-COMPONENTS}` |  A setting that specifies a requirement that a specific component be present to return the entity. You can learn more [here](./entity-components.md#required-components). The possible values are `learned`, `regex`, `list`, or `prebuilts`   |`"learned", "prebuilt"`|
+
+
+
 
 ## Utterance file format