You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/search-get-started-portal-image-search.md
+21-24Lines changed: 21 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@ ms.custom:
15
15
16
16
In this quickstart, you use the **Import and vectorize data** wizard in the Azure portal to get started with [multimodal search](multimodal-search-overview.md). The wizard simplifies the process of extracting, chunking, vectorizing, and loading both text and images into a searchable index.
17
17
18
-
Unlike [Quickstart: Vector search in the Azure portal](search-get-started-portal-import-vectors.md), which processes simple images, this quickstart supports advanced image processing for multimodal RAG scenarios.
18
+
Unlike [Quickstart: Vector search in the Azure portal](search-get-started-portal-import-vectors.md), which processes simple text-containing images, this quickstart supports advanced image processing for multimodal RAG scenarios.
19
19
20
20
This quickstart uses a multimodal PDF from the [azure-search-sample-data](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/sustainable-ai-pdf) repo. However, you can use different files and still complete this quickstart.
21
21
@@ -55,7 +55,7 @@ For content embedding, you can choose either image verbalization (followed by te
55
55
56
56
<sup>1</sup> The endpoint of your Azure OpenAI resource must have a [custom subdomain](/azure/ai-services/cognitive-services-custom-subdomains), such as `https://my-unique-name.openai.azure.com`. If you created your resource in the [Azure portal](https://portal.azure.com/), this subdomain was automatically generated during resource setup.
57
57
58
-
<sup>2</sup> Azure OpenAI resources (with access to embedding models) that were created in the [Azure AI Foundry portal](https://ai.azure.com/) aren't supported. You must create an Azure OpenAI resource in the Azure portal.
58
+
<sup>2</sup> Azure OpenAI resources (with access to embedding models) that were created in the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) aren't supported. You must create an Azure OpenAI resource in the Azure portal.
59
59
60
60
<sup>3</sup> For billing purposes, you must [attach your multi-service resource](cognitive-search-attach-cognitive-services.md) to the skillset in your Azure AI Search service. Unless you use a [keyless connection](cognitive-search-attach-cognitive-services.md#bill-through-a-keyless-connection) to create the skillset, both resources must be in the same region.
61
61
@@ -101,7 +101,7 @@ On your Azure AI Search service:
101
101
102
102
### [**Azure Storage**](#tab/storage-perms)
103
103
104
-
Azure Storage is both the data source for your documents and the destination for extracted images. Your search service requires access to these storage containers, which you create in the next section of this quickstart.
104
+
Azure Storage is both the data source for your documents and the destination for extracted images. Your search service requires access to these storage containers, which you create in the next section.
105
105
106
106
On your Azure Storage account:
107
107
@@ -111,7 +111,7 @@ On your Azure Storage account:
111
111
112
112
### Conditional roles
113
113
114
-
The following tabs cover all wizard-compatible resources for multimodal search. Only select the tabs that apply to your chosen [extraction method](#supported-extraction-methods) and [embedding method](#supported-embedding-methods).
114
+
The following tabs cover all wizard-compatible resources for multimodal search. Select only the tabs that apply to your chosen [extraction method](#supported-extraction-methods) and [embedding method](#supported-embedding-methods).
115
115
116
116
### [**Azure OpenAI**](#tab/openai-perms)
117
117
@@ -123,7 +123,7 @@ On your Azure OpenAI resource:
123
123
124
124
### [**Azure AI Foundry**](#tab/ai-foundry-perms)
125
125
126
-
The Azure AI Foundry model catalog provides large language models (LLMs) for image verbalization and embedding models for text and image vectorization. Your search service requires access to call the [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) and [AML skill](cognitive-search-aml-skill.md).
126
+
The Azure AI Foundry model catalog provides LLMs for image verbalization and embedding models for text and image vectorization. Your search service requires access to call the [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) and [AML skill](cognitive-search-aml-skill.md).
127
127
128
128
On your Azure AI Foundry project:
129
129
@@ -162,7 +162,7 @@ The wizard offers several options for content embedding. Image verbalization req
162
162
163
163
To deploy the models for this quickstart:
164
164
165
-
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
165
+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs).
166
166
167
167
1. Select your Azure OpenAI resource or Azure AI Foundry project.
168
168
@@ -210,9 +210,9 @@ Depending on your chosen [extraction method](#supported-extraction-methods), the
The default method calls the [Document Extraction skill](cognitive-search-skill-document-extraction.md) to extract content and metadata from documents, which includes generating normalized images. The [Text Split skill](cognitive-search-skill-textsplit.md) is then used to split the extracted content into pages.
213
+
The default method calls the [Document Extraction skill](cognitive-search-skill-document-extraction.md) to extract text content and generate normalized images from your documents. The [Text Split skill](cognitive-search-skill-textsplit.md) is then called to split the extracted text content into pages.
214
214
215
-
To use Document Extraction skill:
215
+
To use the Document Extraction skill:
216
216
217
217
1. On the **Content extraction** page, select **Default**.
218
218
@@ -222,13 +222,13 @@ To use Document Extraction skill:
Your Azure AI multi-service resource provides access to the [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md), which extracts page numbers, bounding polygons, and other location metadata from both text and images. The Document Layout skill also breaks documents into smaller, more manageable chunks.
225
+
Your Azure AI multi-service resource provides access to [Azure AI Document Intelligence](/azure/ai-services/document-intelligence/overview), which calls the [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) to recognize document structure and extract text and images relationally. It does so by attaching location metadata, such as page numbers and bounding polygons, to each image. The Document Layout skill also breaks text content into smaller, more manageable chunks.
226
226
227
227
To use the Document Layout skill:
228
228
229
229
1. On the **Content extraction** page, select **AI Document Intelligence**.
230
230
231
-
1. Specify your Azure subscription and Azure AI multi-service resource.
231
+
1. Specify your Azure subscription and multi-service resource.
232
232
233
233
1. For the authentication type, select **System assigned identity**.
234
234
@@ -248,7 +248,7 @@ During this step, the wizard uses your chosen [embedding method](#supported-embe
248
248
249
249
The wizard calls one skill to create descriptive text for images (image verbalization) and another skill to create vector embeddings for both text and images.
250
250
251
-
For image verbalization, the [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) uses the LLM you deployed to analyze each extracted image and produce a natural-language description.
251
+
For image verbalization, the [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) uses your deployed LLM to analyze each extracted image and produce a natural-language description.
252
252
253
253
For embeddings, the [Azure OpenAI Embedding skill](cognitive-search-skill-azure-openai-embedding.md), [AML skill](cognitive-search-aml-skill.md), or [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) uses your deployed embedding model to convert text chunks and verbalized descriptions into high-dimensional vectors. These vectors enable similarity and hybrid retrieval.
254
254
@@ -332,14 +332,14 @@ On the **Advanced settings** page, you can optionally add fields to the index sc
332
332
333
333
| Field | Applies to | Description | Attributes |
334
334
|--|--|--|--|
335
-
| content_id | Text and image vectors | String field. Document key for the index. |Searchable, retrievable, sortable, filterable, and facetable. |
336
-
| document_title | Text and image vectors | String field. Human-readable document title, page title, or page number. |Searchable, retrievable, sortable, filterable, and facetable. |
335
+
| content_id | Text and image vectors | String field. Document key for the index. |Retrievable, sortable, and searchable. |
336
+
| document_title | Text and image vectors | String field. Human-readable document title. |Retrievable and searchable. |
337
337
| text_document_id | Text vectors | String field. Identifies the parent document from which the text chunk originates. | Retrievable and filterable. |
338
-
| image_document_id | Image vectors | String field. Identifies the parent document from which the image originates. |Searchable, retrievable, sortable, filterable, and facetable. |
339
-
| content_text | Text vectors | String field. Human-readable version of the text chunk. |Searchable, retrievable, sortable, filterable, and facetable. |
340
-
| content_embedding |Image vectors | Collection(Edm.Single). Vector representation of the image verbalization. |Searchable and retrievable. |
341
-
| content_path | Text and image vectors | String field. Path to the content in the storage container. | Retrievable, sortable, filterable, and facetable. |
342
-
| locationMetadata |Text and image vectors | Edm.ComplexType. Contains metadata about the content's location. | Varies by field. |
338
+
| image_document_id | Image vectors | String field. Identifies the parent document from which the image originates. |Retrievable and filterable. |
339
+
| content_text | Text vectors | String field. Human-readable version of the text chunk. |Retrievable and searchable. |
340
+
| content_embedding |Text and image vectors | Collection(Edm.Single). Vector representation of text and images. |Retrievable and searchable. |
341
+
| content_path | Text and image vectors | String field. Path to the content in the storage container. | Retrievableand searchable. |
342
+
| locationMetadata |Image vectors | Edm.ComplexType. Contains metadata about the image's location in the documents. | Varies by field. |
343
343
344
344
You can't modify the generated fields or their attributes, but you can add fields if your data source provides them. For example, Azure Blob Storage provides a collection of metadata fields.
345
345
@@ -351,9 +351,6 @@ To add fields to the index schema:
351
351
352
352
1. Select a source field from the available fields, enter a field name for the index, and accept (or override) the default data type.
353
353
354
-
> [!NOTE]
355
-
> Metadata fields are searchable but not retrievable, filterable, facetable, or sortable.
356
-
357
354
1. If you want to restore the schema to its original version, select **Reset**.
358
355
359
356
## Schedule indexing
@@ -392,9 +389,9 @@ When the wizard completes the configuration, it creates the following objects:
392
389
393
390
+ The [Document Extraction skill](cognitive-search-skill-document-extraction.md) or [Document Layout skill](cognitive-search-skill-document-intelligence-layout.md) extracts text and images from source documents. The [Text Split skill](cognitive-search-skill-textsplit.md) accompanies the Document Extraction skill for data chunking, while the Document Layout skill has built-in chunking.
394
391
395
-
+ The [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) verbalizes images. If you're using direct multimodal embeddings, this skill is absent.
392
+
+ The [GenAI Prompt skill](cognitive-search-skill-genai-prompt.md) verbalizes images in natural language. If you're using direct multimodal embeddings, this skill is absent.
396
393
397
-
+ The [Azure OpenAI Embedding skill](cognitive-search-skill-azure-openai-embedding.md), [AML skill](cognitive-search-aml-skill.md), or [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) is called twice to vectorize text and image content.
394
+
+ The [Azure OpenAI Embedding skill](cognitive-search-skill-azure-openai-embedding.md), [AML skill](cognitive-search-aml-skill.md), or [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) is called once for text vectorization and once for image vectorization.
398
395
399
396
+ The [Shaper skill](cognitive-search-skill-shaper.md) enriches the output with metadata and creates new images with contextual information.
400
397
@@ -403,7 +400,7 @@ When the wizard completes the configuration, it creates the following objects:
403
400
404
401
## Check results
405
402
406
-
This quickstart creates a multimodal index that supports [hybrid search](hybrid-search-overview.md) over both text and verbalized images. However, it doesn't support images as query inputs, which requires integrated vectorization using an embedding skill and an equivalent vectorizer. For more information, see [Query with Search explorer](search-explorer.md).
403
+
This quickstart creates a multimodal index that supports [hybrid search](hybrid-search-overview.md) over both text and images. Unless you use direct multimodal embeddings, the index doesn't accept images as query inputs, which requires the [AML skill](cognitive-search-aml-skill.md) or [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md) with an equivalent vectorizer. For more information, see [Query with Search explorer](search-explorer.md).
407
404
408
405
Hybrid search is a combination of full-text queries and vector queries. When you issue a hybrid query, the search engine computes the semantic similarity between your query and the indexed vectors and ranks the results accordingly. For the index created in this quickstart, the results surface content from the `content_text` field that closely aligns with your query.
Copy file name to clipboardExpand all lines: articles/search/search-get-started-portal-import-vectors.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,7 +52,7 @@ For integrated vectorization, you must use one of the following embedding models
52
52
53
53
<sup>1</sup> The endpoint of your Azure OpenAI resource must have a [custom subdomain](/azure/ai-services/cognitive-services-custom-subdomains), such as `https://my-unique-name.openai.azure.com`. If you created your resource in the [Azure portal](https://portal.azure.com/), this subdomain was automatically generated during resource setup.
54
54
55
-
<sup>2</sup> Azure OpenAI resources (with access to embedding models) that were created in the [Azure AI Foundry portal](https://ai.azure.com/) aren't supported. Only Azure OpenAI resources created in the Azure portal are compatible with the [Azure OpenAI Embedding skill](cognitive-search-skill-azure-openai-embedding.md).
55
+
<sup>2</sup> Azure OpenAI resources (with access to embedding models) that were created in the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) aren't supported. Only Azure OpenAI resources created in the Azure portal are compatible with the [Azure OpenAI Embedding skill](cognitive-search-skill-azure-openai-embedding.md).
56
56
57
57
<sup>3</sup> For billing purposes, you must [attach your multi-service resource](cognitive-search-attach-cognitive-services.md) to the skillset in your Azure AI Search service. Unless you use a [keyless connection](cognitive-search-attach-cognitive-services.md#bill-through-a-keyless-connection) to create the skillset, both resources must be in the same region.
58
58
@@ -182,7 +182,7 @@ This section points you to the content that works for this quickstart. Before yo
182
182
183
183
## Prepare embedding model
184
184
185
-
The wizard can use embedding models deployed from Azure OpenAI, Azure AI Vision, or from the model catalog in the [Azure AI Foundry portal](https://ai.azure.com/). Before you proceed, make sure you completed the prerequisites for [role-based access](#role-based-access).
185
+
The wizard can use embedding models deployed from Azure OpenAI, Azure AI Vision, or from the model catalog in the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs). Before you proceed, make sure you completed the prerequisites for [role-based access](#role-based-access).
186
186
187
187
### [Azure OpenAI](#tab/model-aoai)
188
188
@@ -204,7 +204,7 @@ The wizard supports text-embedding-ada-002, text-embedding-3-large, and text-emb
204
204
205
205
1. To deploy an embedding model:
206
206
207
-
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com/) and select your Azure OpenAI resource.
207
+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) and select your Azure OpenAI resource.
208
208
209
209
1. From the left pane, select **Model catalog**.
210
210
@@ -255,7 +255,7 @@ For the model catalog, you should have an [Azure AI Foundry project](/azure/ai-f
255
255
256
256
1. To deploy an embedding model:
257
257
258
-
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com/) and select your project.
258
+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) and select your project.
259
259
260
260
1. From the left pane, select **Model catalog**.
261
261
@@ -409,7 +409,7 @@ However, if you work with content that includes useful images, you can apply AI
409
409
410
410
Azure AI Search and your Azure AI resource must be in the same region or configured for [keyless billing connections](cognitive-search-attach-cognitive-services.md).
411
411
412
-
1. On the **Vectorize your images** page, specify the kind of connection the wizard should make. For image vectorization, the wizard can connect to embedding models in the [Azure AI Foundry portal](https://ai.azure.com/) or Azure AI Vision.
412
+
1. On the **Vectorize your images** page, specify the kind of connection the wizard should make. For image vectorization, the wizard can connect to embedding models in the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) or Azure AI Vision.
0 commit comments