Merge pull request #259298 from MicrosoftDocs/main

11/22 11:00 AM IST Publish

Author: Saisang

articles/ai-services/document-intelligence/concept-add-on-capabilities.md

@@ -8,7 +8,7 @@ ms.service: azure-ai-document-intelligence
 ms.custom:
   - ignite-2023
 ms.topic: conceptual
-ms.date: 11/15/2023
+ms.date: 11/21/2023
 ms.author: lajanuar
 monikerRange: '>=doc-intel-3.1.0'
 ---
@@ -36,23 +36,27 @@ monikerRange: '>=doc-intel-3.1.0'
 
 Document Intelligence supports more sophisticated and modular analysis capabilities. Use the add-on features to extend the results to include more features extracted from your documents. Some add-on features incur an extra cost. These optional features can be enabled and disabled depending on the scenario of the document extraction. The following add-on capabilities are available for `2023-07-31 (GA)` and later releases:
 
-* [`ocr.highResolution`](#high-resolution-extraction)
+* [`ocrHighResolution`](#high-resolution-extraction)
 
-* [`ocr.formula`](#formula-extraction)
+* [`formulas`](#formula-extraction)
 
-* [`ocr.font`](#font-property-extraction)
+* [`styleFont`](#font-property-extraction)
+
+* [`barcodes`](#barcode-property-extraction)
+
+* [`languages`](#language-detection)
 
-* [`ocr.barcode`](#barcode-property-extraction)
 :::moniker-end
 
 :::moniker range="doc-intel-4.0.0"
 
 > [!NOTE]
 >
-> Add-on capabilities are available within all models except for the [Read model](concept-read.md).
+> Not all add-on capabilities are supported by all models. For more information, *see* [model data extraction](concept-model-overview.md#model-data-extraction).
 
 The following add-on capability is available for `2023-10-31-preview` and later releases:
 
+* [`keyValuePairs`](#key-value-pairs)
 * [`queryFields`](#query-fields)
 
 > [!NOTE]
@@ -159,8 +163,33 @@ The `ocr.barcode` capability extracts all identified barcodes in the `barcodes`
 | `ITF` |:::image type="content" source="media/barcodes/interleaved-two-five.png" alt-text="Screenshot of the interleaved-two-of-five barcode (ITF).":::|
 | `Data Matrix` |:::image type="content" source="media/barcodes/datamatrix.gif" alt-text="Screenshot of the Data Matrix.":::|
 
+## Language detection
+
+It predicts the detected primary language for each text line along with the `confidence` in the `languages` collection under `analyzeResult`.
+
+```json
+"languages": [
+    {
+        "spans": [
+            {
+                "offset": 0,
+                "length": 131
+            }
+        ],
+        "locale": "en",
+        "confidence": 0.7
+    },
+]
+```
+
 :::moniker range="doc-intel-4.0.0"
 
+## Key-value Pairs
+
+Key-value pairs are specific spans within the document that identify a label or key and its associated response or value. In a structured form, these pairs could be the label and the value the user entered for that field. In an unstructured document, they could be the date a contract was executed on based on the text in a paragraph. The AI model is trained to extract identifiable keys and values based on a wide variety of document types, formats, and structures.
+
+Keys can also exist in isolation when the model detects that a key exists, with no associated value or when processing optional fields. For example, a middle name field can be left blank on a form in some instances. Key-value pairs are spans of text contained in the document. For documents where the same value is described in different ways, for example, customer/user, the associated key is either customer or user (based on context).
+
 ## Query Fields
 
 * Document Intelligence now supports query field extractions. With query field extraction, you can add fields to the extraction process using a query request without the need for added training.
@@ -185,7 +214,7 @@ For query field extraction, specify the fields you want to extract and Document
 
     :::image type="content" source="media/studio/query-fields.png" alt-text="Screenshot of the query fields button in Document Intelligence Studio.":::
 
-* You can pass a list of field labels like `Party1`, `Party2`, `TermsOfUse`, `PaymentTerms`, `PaymentDate`, and `TermEndDate`" as part of the analyze document request.
+* You can pass a list of field labels like `Party1`, `Party2`, `TermsOfUse`, `PaymentTerms`, `PaymentDate`, and `TermEndDate`" as part of the `analyze document` request.
 
    :::image type="content" source="media/studio/query-field-select.png" alt-text="Screenshot of query fields selection window in Document Intelligence Studio.":::
 

articles/ai-services/document-intelligence/concept-model-overview.md

@@ -88,12 +88,15 @@ The following table shows the available models for each current preview and stab
 | [Custom classification model](#custom-classifier)| The **Custom classification model** can classify each page in an input file to identify the document(s) within and can also identify multiple documents or multiple instances of a single document within an input file.
 | [Composed models](#composed-models) | Combine several custom models into a single model to automate processing of diverse document types with a single composed model.
 
-For all models, except Business card model, Document Intelligence now supports add-on capabilities to allow for more sophisticated analysis. These optional capabilities can be enabled and disabled depending on the scenario of the document extraction. There are four add-on capabilities available for the `2023-07-31` (GA) and later API version:
+For all models, except Business card model, Document Intelligence now supports add-on capabilities to allow for more sophisticated analysis. These optional capabilities can be enabled and disabled depending on the scenario of the document extraction. There are seven add-on capabilities available for the `2023-07-31` (GA) and later API version:
 
-* [`ocr.highResolution`](concept-add-on-capabilities.md#high-resolution-extraction)
-* [`ocr.formula`](concept-add-on-capabilities.md#formula-extraction)
-* [`ocr.font`](concept-add-on-capabilities.md#font-property-extraction)
-* [`ocr.barcode`](concept-add-on-capabilities.md#barcode-property-extraction)
+* [`ocrHighResolution`](concept-add-on-capabilities.md#high-resolution-extraction)
+* [`formulas`](concept-add-on-capabilities.md#formula-extraction)
+* [`styleFont`](concept-add-on-capabilities.md#font-property-extraction)
+* [`barcodes`](concept-add-on-capabilities.md#barcode-property-extraction)
+* [`languages`](concept-add-on-capabilities.md#language-detection)
+* [`keyValuePairs`](concept-add-on-capabilities.md#key-value-pairs) (2023-10-31-preview)
+* [`queryFields`](concept-add-on-capabilities.md#query-fields) (2023-31-preview)
 
 ## Analysis features
 
@@ -146,7 +149,6 @@ The Layout analysis model analyzes and extracts text, tables, selection marks, a
 >
 > [Learn more: layout model](concept-layout.md)
 
-
 ### Health insurance card
 
 :::image type="icon" source="media/studio/health-insurance-logo.png":::
@@ -174,7 +176,6 @@ The US tax document models analyze and extract key fields and line items from a
   |US Tax 1098-T|Extract qualified tuition details.|**prebuilt-tax.us.1098T**|
   |US Tax 1099|Extract Information from 1099 forms.|**prebuilt-tax.us.1099(variations)**|
 
-
 ***Sample W-2 document processed using [Document Intelligence Studio](https://formrecognizer.appliedai.azure.com/studio/prebuilt?formType=tax.us.w2)***:
 
 :::image type="content" source="./media/studio/w-2.png" alt-text="Screenshot of a sample W-2.":::

articles/ai-services/language-service/summarization/how-to/document-summarization.md

@@ -87,8 +87,6 @@ You can also use the `sortby` parameter to specify in what order the extracted s
 
 ### Try document abstractive summarization
 
-<!-- [Reference documentation](https://go.microsoft.com/fwlink/?linkid=2211684) -->
-
 The following example gets you started with document abstractive summarization:
 
 1. Copy the command below into a text editor. The BASH example uses the `\` line continuation character. If your console or terminal uses a different line continuation character, use that character instead.

articles/ai-services/language-service/summarization/includes/quickstarts/rest-api.md

@@ -9,16 +9,8 @@ ms.author: aahi
 ms.custom: ignite-fall-2021, event-tier1-build-2022, ignite-2022
 ---
 
-# [Document summarization](#tab/document-summarization)
-
-[Reference documentation](https://go.microsoft.com/fwlink/?linkid=2211684)
-
 # [Conversation summarization](#tab/conversation-summarization)
 
-[Reference documentation](https://go.microsoft.com/fwlink/?linkid=2195178)
-
----
-
 Use this quickstart to send text summarization requests using the REST API. In the following example, you will use cURL to summarize documents or text-based customer service conversations.
 
 [!INCLUDE [Use Language Studio](../use-language-studio.md)]
@@ -54,8 +46,6 @@ Choose the type of summarization you would like to perform, and select one of th
 
 # [Document summarization](#tab/document-summarization)
 
-[Reference documentation](https://go.microsoft.com/fwlink/?linkid=2211684)
-
 |parameter  |Description  |
 |---------|---------|
 |`-X POST <endpoint>`     | Specifies your endpoint for accessing the API.        |
@@ -199,8 +189,6 @@ curl -X GET $LANGUAGE_ENDPOINT/language/analyze-text/jobs/<my-job-id>?api-versio
 }
 ```
 
-# [Conversation summarization](#tab/conversation-summarization)
-
 ## Conversation issue and resolution summarization
 
 The following example will get you started with conversation issue and resolution summarization:

articles/ai-services/language-service/summarization/overview.md

@@ -147,8 +147,6 @@ As you use document summarization in your applications, see the following refere
 |JavaScript     | [JavaScript documentation](/javascript/api/overview/azure/ai-text-analytics-readme?view=azure-node-preview&preserve-view=true)        | [JavaScript samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/textanalytics/ai-text-analytics/samples/v5) |
 |Python | [Python documentation](/python/api/overview/azure/ai-textanalytics-readme?view=azure-python-preview&preserve-view=true)        | [Python samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/textanalytics/azure-ai-textanalytics/samples) |
 
-<!-- |REST API     | [REST API documentation](https://go.microsoft.com/fwlink/?linkid=2211684)        |         | -->
-
 ## Responsible AI
 
 An AI system includes not only the technology, but also the people who will use it, the people who will be affected by it, and the environment in which it’s deployed. Read the [transparency note for summarization](/legal/cognitive-services/language-service/transparency-note-extractive-summarization?context=/azure/ai-services/language-service/context/context) to learn about responsible AI use and deployment in your systems. You can also see the following articles for more information: