You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-services/content-understanding/overview.md
+14-20Lines changed: 14 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,8 @@ ms.author: lajanuar
7
7
manager: nitinme
8
8
ms.service: azure-ai-content-understanding
9
9
ms.topic: overview
10
-
ms.date: 05/19/2025
10
+
ms.date: 03/06/2025
11
+
ms.custom: ignite-2024-understanding-release
11
12
12
13
#customer intent: As a user, I want to learn more about Content Understanding solutions.
13
14
---
@@ -50,16 +51,14 @@ Content Understanding offers a streamlined process to reason over large amounts
50
51
51
52
Common applications for Content Understanding include:
52
53
53
-
|Application|Description|Quickstart|
54
-
|:---------|:----------|:----------|
55
-
|Post-call analytics| Businesses and call centers can generate insights from call recordings to track key KPIs, improve product experience, generate business insights, create differentiated customer experiences, and answer queries faster and more accurately.|[**Post-call analytics quickstart**](concepts/analyzer-templates.md#modality-templates)|
56
-
|Media asset management| Software and media vendors can use Content Understanding to extract richer, targeted information from videos for media asset management solutions.|[**Media asset management quickstart**](concepts/analyzer-templates.md#modality-templates)|
57
-
|Tax automation| Tax preparation companies can use Content Understanding to generate a unified view of information from various documents and create comprehensive tax returns.|[**Tax automation quickstart**](concepts/analyzer-templates.md#modality-templates)|
58
-
|Chart understanding| Businesses can enhance chart understanding by automating the analysis and interpretation of various types of charts and diagrams using Content Understanding.|[**Chart understanding quickstart**](concepts/analyzer-templates.md#modality-templates)|
59
-
|Mortgage application processing|Analyze supplementary supporting documentation and mortgage applications to determine whether a prospective home buyer provided all the necessary documentation to secure a mortgage.|[**Content Understanding Pro quickstart**](concepts/standard-pro-modes.md#apply-standard-or-pro-mode-to-your-scenarios)|
60
-
|Invoice contract verification|Review invoices and contractual agreements with clients carefully. Apply a multi-step reasoning process to analyze the data. Ensure that conclusions, such as validating the consistency between the invoice and the contract, are accurate and thorough.|[**Content Understanding Pro quickstart**](concepts/standard-pro-modes.md#apply-standard-or-pro-mode-to-your-scenarios)|
61
-
62
-
See [Quickstart](quickstart/use-ai-foundry.md) for more examples.
54
+
|Application|Description|
55
+
|:---------|:----------|
56
+
|Post-call analytics| Businesses and call centers can generate insights from call recordings to track key KPIs, improve product experience, generate business insights, create differentiated customer experiences, and answer queries faster and more accurately.|
57
+
|Media asset management| Software and media vendors can use Content Understanding to extract richer, targeted information from videos for media asset management solutions.|
58
+
|Tax automation| Tax preparation companies can use Content Understanding to generate a unified view of information from various documents and create comprehensive tax returns.|
59
+
|Chart understanding| Businesses can enhance chart understanding by automating the analysis and interpretation of various types of charts and diagrams using Content Understanding.|
60
+
|Mortgage application processing|Analyze supplementary supporting documentation and mortgage applications to determine whether a prospective home buyer provided all the necessary documentation to secure a mortgage.|
61
+
|Invoice contract verification|Review invoices and contractual agreements with clients carefully. Apply a multi-step reasoning process to analyze the data. Ensure that conclusions, such as validating the consistency between the invoice and the contract, are accurate and thorough.|
63
62
64
63
## Components
65
64
@@ -68,34 +67,29 @@ See [Quickstart](quickstart/use-ai-foundry.md) for more examples.
68
67
|Component|Description|
69
68
|:---------|:----------|
70
69
|Analyzer|The analyzer is the core component of Content Understanding. It allows customers to configure content extraction settings and field extraction schema. Once configured, the analyzer consistently applies these settings to process all incoming data.|
71
-
|Content extraction|Content extraction enables users to specify the types of information to be identified and extracted from incoming content. User-specified information includes options such as `OCR` for text, layout analysis, barcodes, tables, and more, allowing users to focus on the most relevant content elements.|
72
-
|Add-ons| Content Understanding add-ons enhance content extraction by incorporating added elements like barcodes, tables, and detected faces.|
70
+
|Content extraction|With content extraction, you specify the types of information to identify and extract from incoming content. You can target text (such as optical character recognition (OCR) results), selection marks, barcodes, formulas, and layout elements like paragraphs, sections, and tables. This approach lets you focus on extracting the most relevant information for your needs.|
73
71
|Field extraction|Field extraction allows users to define the structure and schema of the desired fields to extract from input files. See [service limits](service-limits.md) for a complete list of field types supported. Fields can be generated via one of the following methods:</br></br> •**Extract**: Directly extract values as they appear in the input content, such as dates from receipts or item details from invoices.</br></br>•**Classify**: Classify content from a predefined set of categories, such as call sentiment or chart type.</br></br>•**Generate**: Generate values freely from input data, such as summarizing an audio conversation or creating scene descriptions from videos.|
74
72
|Grounding source| Content Understanding identifies the specific regions in the content where the value was generated from. Source grounding allows users in automation scenarios to quickly verify the correctness of the field values, leading to higher confidence in the extracted data. |
75
73
|Confidence score | Content Understanding provides confidence scores from 0 to 1 to estimate the reliability of the results. High scores indicate accurate data extraction, enabling straight-through processing in automation workflows.|
76
-
|Reference dataset (offered in Pro mode)|The service can reference documents at inference time to aid in providing context. For example, if you're looking to analyze invoices to ensure they're consistent with a contractual agreement, you can supply the invoice and other relevant documents (for example, purchase order) as inputs, and supply the contract files as reference data.|
77
-
|Multi-step reasoning (offered in Pro mode)|Multi-step reasoning takes data analysis a step further than extracting and aggregating structured data and allows you to draw conclusions on that data, minimizing the need for human review.|
78
74
79
75
## Responsible AI
80
76
81
77
Azure AI Content Understanding is designed to guard against processing harmful content, such as graphic violence and gore, hateful speech and bullying, exploitation, abuse, and more. For more information and a full list of prohibited content, *see* our [**Transparency note**](/legal/cognitive-services/content-understanding/transparency-note?toc=/azure/ai-services/content-understanding/toc.json&bc=/azure/ai-services/content-understanding/breadcrumb/toc.json) and our [**Code of Conduct**](https://aka.ms/AI-CoC).
82
78
83
79
### Modified content filtering
84
80
85
-
Content Understanding now supports modified content filtering for approved customers. The subscription IDs with approved modified content filtering impacts Content Understanding output. By default, Content Understanding employs a content filtering system that identifies specific risk categories for potentially harmful content in both submitted prompts and generated outputs. Modified content filtering allows the system to annotate rather than block potentially harmful output, giving you the ability to determine how to handle potentially harmful content. For more information on content filter types, *see*[Content filtering: filter types](../openai/concepts/content-filter.md#content-filter-types).
81
+
Content Understanding now supports modified content filtering for approved customers. The subscription IDs with approved modified content filtering impacts Content Understanding output. By default, Content Understanding employs a content filtering system that identifies specific risk categories for potentially harmful content in both submitted prompts and generated outputs. Modified content filtering allows the system to annotate rather than block potentially harmful output, giving you the ability to determine how to handle potentially harmful content. For more information on content filter types, *see*[Content filter types](../openai/concepts/content-filter.md#content-filter-types).
86
82
87
83
> [!IMPORTANT]
88
84
>
89
85
> * Apply for modified content filters via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR).
90
-
> * For more information, *see*[**Content Filtering**](../openai/concepts/content-filter.md).
91
-
92
-
To learn more about how to add modified content filtering to your requests, *see* our [REST API quickstart](quickstart/use-rest-api.md).
86
+
> * For more information, *see*[**Content filtering**](../openai/concepts/content-filter.md).
93
87
94
88
## Data privacy and security
95
89
Developers using the Content Understanding service should review Microsoft's policies on customer data. For more information, visit our [**Data, protection and privacy**](https://www.microsoft.com/trust-center/privacy) page.
96
90
97
91
> [!IMPORTANT]
98
-
> If you're using Microsoft products or services to process Biometric Data, you're responsible for: (i) providing notice to data subjects, including with respect to retention periods and destruction; (ii) obtaining consent from data subjects; and (iii) deleting the Biometric Data, all as appropriate, and required under applicable Data Protection Requirements. "Biometric Data" has the meaning articulated in Article 4 of the GDPR and, if applicable, equivalent terms in other data protection requirements. For related information, see [Data and Privacy for Face](/legal/cognitive-services/face/data-privacy-security).
92
+
> If you're using Microsoft products or services to process Biometric Data, you're responsible for: (i) providing notice to data subjects, including with respect to retention periods and destruction; (ii) obtaining consent from data subjects; and (iii) deleting the Biometric Data, all as appropriate, and required under applicable Data Protection Requirements. "Biometric Data" has the meaning articulated in Article 4 of the GDPR and, if applicable, equivalent terms in other data protection requirements. For related information, see [Data and privacy for Face](/legal/cognitive-services/face/data-privacy-security).
Copy file name to clipboardExpand all lines: articles/ai-services/content-understanding/whats-new.md
+23-27Lines changed: 23 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,44 +20,40 @@ Azure AI Content Understanding service is updated on an ongoing basis. Bookmark
20
20
21
21
The Azure AI Content Understanding **`2025-05-01-preview`** REST API is now available. This update introduces the following updates and enhanced capabilities:
22
22
23
-
***Processing modes**: With the **`2025-05-01-preview`** release, we introduce two modes: `standard` and `pro`. The default mode for all analyzers is `standard`. Content Understanding pro mode adds reasoning, support for multiple input documents, the ability to configure an external knowledge base for linking, enrichment, and validation. These features enable automating complex tasks to extend field extraction capabilities to include tasks that required custom code or human effort. The `pro` mode is currently limited to documents as inputs. Common challenges that the pro mode addresses include aggregating a schema across content from different input files. It also involves validating results across documents. Additionally, it uses external knowledge, such as guidelines, standard operating procedures, and other context, to generate an output schema. Learn more about the [pro mode](concepts/standard-pro-modes.md).
23
+
### Processing modes
24
24
25
-
***Improvements to document processing** :
25
+
With the **`2025-05-01-preview`** release, we introduce two modes: `standard` and `pro`.
26
+
The default mode for all analyzers is `standard`.
27
+
Content Understanding pro mode adds reasoning, support for multiple input documents, the ability to configure an external knowledge base for linking, enrichment, and validation.
28
+
These features automate complex tasks by extending field extraction capabilities to cover scenarios that previously required custom code or human effort.
26
29
27
-
***Document classification and splitting** with a [Classification API](concepts/classifier.md). This API supports classifying and logical splitting a single file containing multiple documents with optional routing to field extraction analyzers. The API enables you to define a workflow to classify and split a file into multiple logical documents and route the individual documents to a downstream field extraction model in a single API call.
28
-
* Improvements in **content extraction**:
29
-
* Added support for extracting table spanning multiple pages as a single logical table. Learn more about [structure extraction updates in documents](document/elements.md).
30
-
* Selection mark support for checkmark and radio buttons as unicode characters. Learn more about [structure extraction updates in documents](document/elements.md).
31
-
* Bar code extraction as part of the default content extraction along with `OCR`. Learn more about [structure extraction updates in documents](document/elements.md).
32
-
* Support for embedded figures in Markdown as base64 string format output. Learn more about [structure extraction updates in documents](document/elements.md).
33
-
* Improvements in **field extraction**
34
-
* Confidence score improvements with better grounding results for extractive fields.
35
-
* New file format support extended for following document types `.rtf`,`.txt`,`.xml`,`.json`, `.msg`,`.eml`,`.csv`, and `.xlsx`.
30
+
The `pro` mode is currently limited to documents as inputs, with support other types of content types coming soon!
31
+
Common challenges that the pro mode addresses are aggregating a schema across content from different input files, validating results across documents, and using external knowledge to generate an output schema.
32
+
Learn more about the [pro mode](concepts/standard-pro-modes.md).
36
33
34
+
### Document classification and splitting
37
35
38
-
***Improvements to video processing**:
36
+
This release introduces a new [classification API](concepts/classifier.md). This API supports classifying and logically splitting a single file containing multiple documents with optional routing to field extraction analyzers. You can create a custom classifier to split and classify a file into multiple logical documents and route the individual documents to a downstream field extraction model in a single API call.
39
37
40
-
* Added Support for whole video fields. Learn more about [video processing improvements](video/overview.md#segmentation-mode).
41
-
* Added Support for video chapters via segmentation. Learn more about [video processing improvements](video/overview.md#segmentation-mode).
42
-
* Added Support for face identification on extracted face thumbnails. The identity enhances the description and downstream tasks like search and retrieval. Learn more about [face detection in videos](video/overview.md#content-extraction---grouping-and-identification)
43
-
* Added Support for disabling face blurring in analyzer configuration. Learn more about [video processing improvements](video/overview.md#field-extraction--face-description).
38
+
### Improvements to document processing
44
39
45
-
***Improvements in audio processing**:
40
+
* Added support for extracting table spanning multiple pages as a single logical table. Learn more about [structure extraction updates in documents](document/elements.md).
41
+
* Selection mark support for checkmark and radio buttons as unicode characters. Learn more about [structure extraction updates in documents](document/elements.md).
42
+
* Barcode extraction as part of the default content extraction along with `OCR`. Learn more about [structure extraction updates in documents](document/elements.md).
43
+
* Confidence score improvements with better grounding results for extractive fields.
44
+
* New file format support extended for following document types: `docx`, `xslx`, `pptx`, `msg`, `eml`, `rtf`, `html`, `md`, and `xml`.
46
45
47
-
* Support for multi-speaker call center role detection to allow detection of multiple speakers.
46
+
### Improvements to video processing
48
47
49
-
***Face API preview**:
48
+
* Added support for whole video fields. Learn more about [video processing improvements](video/overview.md#segmentation-mode).
49
+
* Added support for video chapters via segmentation. Learn more about [video processing improvements](video/overview.md#segmentation-mode).
50
+
* Added support for face identification on extracted face thumbnails. The identity enhances the description and downstream tasks like search and retrieval. Learn more about [face detection in videos](video/overview.md#content-extraction---grouping-and-identification)
51
+
* Added support for disabling face blurring in analyzer configuration. Learn more about [video processing improvements](video/overview.md#field-extraction--face-description).
50
52
51
-
This release adds new face detection and recognition capabilities to Content Understanding. You can create a database of faces and recognize the faces in the processed content.
52
-
* Detection, recognition, and enrollment of faces. Learn more about [detecting and recognizing faces](face/overview.md).
53
+
### Face API
53
54
54
-
***Billing Changes**:
55
+
This release adds new face detection and recognition capabilities to Content Understanding. You can create a directory of faces and persons. The directory can be used to recognize the faces in the processed content. Learn more about [detecting and recognizing faces](face/overview.md).
55
56
56
-
* New simpler pricing model that lowers processing costs when compared to the **2024-12-01-preview** REST API across many of the features. Learn more about the [updated pricing model](https://azure.microsoft.com/pricing/details/content-understanding/)
57
-
58
-
***User experience improvements**:
59
-
60
-
The [Azure AI Foundry](https://ai.azure.com/) experience continues to improve with a streamlined project creation flow, improved performance experience, and a try-out experience. Get started with Content Understanding in the [Azure AI Foundry](https://aka.ms/cu-landing) today.
0 commit comments