You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/tutorial-document-extraction-image-verbalization.md
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ In this tutorial, you use:
26
26
27
27
+ The [Document Extraction skill](cognitive-search-skill-document-extraction.md) for extracting normalized images and text.
28
28
29
-
+ The [GenAI Prompt skill (preview)](cognitive-search-skill-genai-prompt.md)to generate image captions, which are text-based descriptions of visual content, for search and grounding.
29
+
+ The [GenAI Prompt skill (preview)](cognitive-search-skill-genai-prompt.md)that calls a chat completion model to create descriptions of visual content.
30
30
31
31
+ A search index configured to store text and image embeddings and support for vector-based similarity search.
32
32
@@ -35,28 +35,29 @@ This tutorial demonstrates a lower-cost approach for indexing multimodal content
35
35
For a more comprehensive solution that includes structured text layout and spatial metadata, see [Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md).
36
36
37
37
> [!NOTE]
38
-
> Setting `imageAction` to `generateNormalizedImages`is required for this tutorial and incurs an additional charge for image extraction according to [Azure AI Search pricing](https://azure.microsoft.com/pricing/details/search/).
38
+
> Setting `imageAction` to `generateNormalizedImages`results in image extraction, which is an extra charge. For more information, see [Azure AI Search pricing](https://azure.microsoft.com/pricing/details/search/) for image extraction.
39
39
40
-
Using a REST client and the [Search REST APIs](/rest/api/searchservice/) you will:
40
+
<!--Using a REST client and the [Search REST APIs](/rest/api/searchservice/) you will:
41
41
42
42
> [!div class="checklist"]
43
43
> + Set up sample data and configure an `azureblob` data source
44
44
> + Create an index with support for text and image embeddings
45
45
> + Define a skillset with extraction, captioning, and embedding steps
46
46
> + Create and run an indexer to process and index content
47
47
> + Search the index you just created
48
+
-->
48
49
49
50
## Prerequisites
50
51
51
52
+ An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).
+[Azure Storage](/azure/storage/common/storage-account-create), used for storing sample data.
54
55
55
56
+[Azure AI Search](search-what-is-azure-search.md), Basic pricing tier or higher, with a managed identity. [Create a service](search-create-service-portal.md) or [find an existing service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your current subscription.
56
57
57
58
+[Visual Studio Code](https://code.visualstudio.com/download) with a [REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client).
Copy file name to clipboardExpand all lines: articles/search/tutorial-document-extraction-multimodal-embeddings.md
+9-11Lines changed: 9 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,40 +22,38 @@ In this tutorial, you use:
22
22
23
23
+ A 36-page PDF document that combines rich visual content, such as charts, infographics, and scanned pages, with traditional text.
24
24
25
-
+Extract text and normalized images using the [Document Extraction skill](cognitive-search-skill-document-extraction.md).
25
+
+The [Document Extraction skill](cognitive-search-skill-document-extraction.md) for extracting normalized images and text.
26
26
27
-
+Vectorize text and images using the [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md).
27
+
+The [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) to vectorize text and images.
28
28
29
29
+ A search index configured to store text and image embeddings and support for vector-based similarity search.
30
30
31
-
This tutorial demonstrates a lower-cost approach for indexing multimodal content using Document Extraction skill and image captioning. It enables extraction and search over both text and images from documents in Azure Blob Storage. However, it doesn't include locational metadata for text, such as page numbers or bounding regions.
32
-
33
-
For a more comprehensive solution that includes structured text layout and spatial metadata, see [Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md).
31
+
This tutorial demonstrates a lower-cost approach for indexing multimodal content using the Document Extraction skill and image captioning. It enables extraction and search over both text and images from documents in Azure Blob Storage. However, it doesn't include locational metadata for text, such as page numbers or bounding regions. For a more comprehensive solution that includes structured text layout and spatial metadata, see [Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md).
34
32
35
33
> [!NOTE]
36
-
> Setting `imageAction` to `generateNormalizedImages` results in image extraction, which is an extra charge. For more information, see [Azure AI Search pricing](https://azure.microsoft.com/pricing/details/search/) for image extraction pricing.
34
+
> Setting `imageAction` to `generateNormalizedImages` results in image extraction, which is an extra charge. For more information, see [Azure AI Search pricing](https://azure.microsoft.com/pricing/details/search/) for image extraction.
37
35
38
-
Using a REST client and the [Search REST APIs](/rest/api/searchservice/) you will:
36
+
<!--Using a REST client and the [Search REST APIs](/rest/api/searchservice/) you will:
39
37
40
38
> [!div class="checklist"]
41
39
> + Set up sample data and configure an `azureblob` data source
42
40
> + Create an index with support for text and image embeddings
43
41
> + Define a skillset with extraction and embedding steps
44
42
> + Create and run an indexer to process and index content
45
43
> + Search the index you just created
44
+
-->
46
45
47
46
## Prerequisites
48
47
49
48
+[Azure Storage](/azure/storage/common/storage-account-create), used for storing sample data.
50
49
51
-
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills) for image vectorization. Image vectorization requires Azure AI Vision multimodal embeddings. For an updated list of regions, see the [Azure AI Vision documentation](/azure/ai-services/computer-vision/overview-image-analysis#region-availability).
52
-
50
+
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills) that provides Azure AI Vision for multimodal embeddings. You must use an Azure AI multi-service account for this task. For an updated list of regions that provide multimodal embeddings, see the [Azure AI Vision documentation](/azure/ai-services/computer-vision/overview-image-analysis#region-availability).
53
51
+[Azure AI Search](search-create-service-portal.md), with a managed identity. [Create a service](search-create-service-portal.md) or [find an existing service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your current subscription.
54
52
> Your service must be on the Basic tier or higher—this tutorial isn't supported on the Free tier. It must also be in the same region as your multi-service account.
55
53
56
54
+[Visual Studio Code](https://code.visualstudio.com/download) with a [REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client).
57
55
58
-
### Download files
56
+
##Prepare data
59
57
60
58
Download the following sample PDF:
61
59
@@ -67,7 +65,7 @@ Download the following sample PDF:
67
65
68
66
1.[Upload the sample data file](/azure/storage/blobs/storage-quickstart-blobs-portal).
69
67
70
-
1.[Create a role assignment in Azure Storage and Specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
68
+
1.[Create a **Storage Blob Data Reader** role assignment and specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
71
69
72
70
1. For connections made using a system-assigned managed identity. Provide a connection string that contains a ResourceId, with no account key or password. The ResourceId must include the subscription ID of the storage account, the resource group of the storage account, and the storage account name. The connection string is similar to the following example:
Copy file name to clipboardExpand all lines: articles/search/tutorial-document-layout-image-verbalization.md
+11-14Lines changed: 11 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,37 +24,34 @@ In this tutorial, you use:
24
24
25
25
+ A 36-page PDF document that combines rich visual content, such as charts, infographics, and scanned pages, with traditional text.
26
26
27
-
+ The [Document Layout skill (preview)](cognitive-search-skill-document-intelligence-layout.md) for extracting text and normalized images with its locationMetadata from various documents, such as page numbers or bounding regions.
27
+
+ The [Document Layout skill (preview)](cognitive-search-skill-document-intelligence-layout.md) for extracting text and normalized images with its `locationMetadata` from various documents, such as page numbers or bounding regions.
28
28
29
-
The [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) has limited regional availability, is bound to Azure AI services, and requires a [billable resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. For a lower-cost solution to indexing multimodal content, see [Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md).
30
-
31
-
+ The [GenAI Prompt skill (preview)](cognitive-search-skill-genai-prompt.md) to generate image captions, which are text-based descriptions of visual content, for search and grounding.
29
+
+ The [GenAI Prompt skill (preview)](cognitive-search-skill-genai-prompt.md) that calls a chat completion model to create descriptions of visual content.
32
30
33
31
+ A search index configured to store text and image embeddings and support for vector-based similarity search.
34
32
35
-
> [!NOTE]
36
-
> Setting `imageAction` to `generateNormalizedImages` is required for this tutorial and incurs an additional charge for image extraction according to [Azure AI Search pricing](https://azure.microsoft.com/pricing/details/search/).
37
-
38
-
Using a REST client and the [Search REST APIs](/rest/api/searchservice/), you will:
33
+
<!-- Using a REST client and the [Search REST APIs](/rest/api/searchservice/), you will:
39
34
40
35
> [!div class="checklist"]
41
36
> + Set up sample data and configure an `azureblob` data source
42
37
> + Create an index with support for text and image embeddings
43
38
> + Define a skillset with extraction, captioning, embedding and knowleage store file projection steps
44
39
> + Create and run an indexer to process and index content
45
-
> + Search the index you just created
40
+
> + Search the index you just created-->
46
41
47
42
## Prerequisites
48
43
49
-
+ An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).
+[Azure Storage](/azure/storage/common/storage-account-create), used for storing sample data.
52
45
53
46
+[Azure AI Search](search-what-is-azure-search.md). [Create a service](search-create-service-portal.md) or [find an existing service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your current subscription. Your service must be on the Basic tier or higher. This tutorial isn't supported on the Free tier.
54
47
55
48
+[Visual Studio Code](https://code.visualstudio.com/download) with a [REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client).
56
49
57
-
### Download files
50
+
## Limitations
51
+
52
+
The [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) has limited regional availability, is bound to Azure AI services, and requires a [billable resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. For a lower-cost solution to indexing multimodal content, see [Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md).
53
+
54
+
## Prepare data
58
55
59
56
Download the following sample PDF:
60
57
@@ -66,7 +63,7 @@ Download the following sample PDF:
66
63
67
64
1.[Upload the sample data file](/azure/storage/blobs/storage-quickstart-blobs-portal).
68
65
69
-
1.[Create a role assignment in Azure Storage and Specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
66
+
1.[Create a **Storage Blob Data Reader** role assignment and specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
70
67
71
68
1. For connections made using a system-assigned managed identity. Provide a connection string that contains a ResourceId, with no account key or password. The ResourceId must include the subscription ID of the storage account, the resource group of the storage account, and the storage account name. The connection string is similar to the following example:
Copy file name to clipboardExpand all lines: articles/search/tutorial-document-layout-multimodal-embeddings.md
+12-13Lines changed: 12 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,49 +23,48 @@ In this tutorial, you use:
23
23
24
24
+ A 36-page PDF document that combines rich visual content, such as charts, infographics, and scanned pages, with traditional text.
25
25
26
-
+ The [Document Layout skill (preview)](cognitive-search-skill-document-intelligence-layout.md) for extracting text and normalized images with its locationMetadata from various documents, such as page numbers or bounding regions.
26
+
+ The [Document Layout skill (preview)](cognitive-search-skill-document-intelligence-layout.md) for extracting text and normalized images with its `locationMetadata` from various documents, such as page numbers or bounding regions.
27
27
28
-
The [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) has limited regional availability, is bound to Azure AI services, and requires a [billable resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. For a lower-cost solution to indexing multimodal content, see [Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md).
29
-
30
-
+ Vectorization using the [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md), which generates embeddings for both text and images.
28
+
+ The [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) to vectorize text and images.
31
29
32
30
+ A search index configured to store text and image embeddings and support for vector-based similarity search.
33
31
34
-
Using a REST client and the [Search REST APIs](/rest/api/searchservice/), you will:
32
+
<!--Using a REST client and the [Search REST APIs](/rest/api/searchservice/), you will:
35
33
36
34
> [!div class="checklist"]
37
35
> + Set up sample data and configure an `azureblob` data source
38
36
> + Create an index with support for text and image embeddings
39
37
> + Define a skillset with extraction, embedding and knowleage store file projection steps
40
38
> + Create and run an indexer to process and index content
41
-
> + Search the index you just created
39
+
> + Search the index you just created-->
42
40
43
41
## Prerequisites
44
42
45
-
+ An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).
+[Azure Storage](/azure/storage/common/storage-account-create), used for storing sample data.
48
44
49
-
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills) for image vectorization. Image vectorization requires Azure AI Vision multimodal embeddings. For an updated list of regions, see the [Azure AI Vision documentation](/azure/ai-services/computer-vision/overview-image-analysis#region-availability).
45
+
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills)that provides Azure AI Vision for multimodal embeddings. You must use an Azure AI multi-service account for this task. For an updated list of regions that provide multimodal embeddings, see the [Azure AI Vision documentation](/azure/ai-services/computer-vision/overview-image-analysis#region-availability).
50
46
51
47
+[Azure AI Search](search-what-is-azure-search.md), with a managed identity. [Create a service](search-create-service-portal.md) or [find an existing service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your current subscription. Your service must be on the Basic tier or higher—this tutorial isn't supported on the Free tier. It must also be in the same region as your multi-service account.
52
48
53
49
+[Visual Studio Code](https://code.visualstudio.com/download) with a [REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client).
54
50
55
-
### Download files
51
+
## Limitations
52
+
53
+
The [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) has limited regional availability, is bound to Azure AI services, and requires a [billable resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. For a lower-cost solution to indexing multimodal content, see [Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md).
1. In Azure Storage, create a new container named **doc-intelligence-multimodality-container**.
65
64
66
65
1.[Upload the sample data file](/azure/storage/blobs/storage-quickstart-blobs-portal).
67
66
68
-
1.[Create a role assignment in Azure Storage and specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
67
+
1.[Create a **Storage Blob Data Reader** role assignment and specify a managed identity in a connection string](search-howto-managed-identities-storage.md)
69
68
70
69
1. For connections made using a system-assigned managed identity, provide a connection string that contains a ResourceId, with no account key or password. The ResourceId must include the subscription ID of the storage account, the resource group of the storage account, and the storage account name. The connection string is similar to the following example:
0 commit comments