You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/search-get-started-portal-image-search.md
+37-35Lines changed: 37 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,19 @@
1
1
---
2
2
title: "Quickstart: Multimodal Search in the Azure portal"
3
3
titleSuffix: Azure AI Search
4
-
description: Learn how to search for multimodal content on an Azure AI Search index in the Azure portal. Run a wizard to vectorize text and images, and then use Search Explorer to provide an image as your query input.
4
+
description: Learn how to search for multimodal content on an Azure AI Search index in the Azure portal. Run a wizard to generate natural-language descriptions of images and vectorize both text and images, and then use Search Explorer to query your multimodal index.
5
5
author: haileytap
6
6
ms.author: haileytapia
7
7
ms.service: azure-ai-search
8
8
ms.topic: quickstart
9
-
ms.date: 05/20/2025
9
+
ms.date: 05/21/2025
10
10
ms.custom:
11
11
- references_regions
12
12
---
13
13
14
14
# Quickstart: Search for multimodal content in the Azure portal
15
15
16
-
In this quickstart, you use the **Import and vectorize data wizard** in the Azure portal to get started with [multimodal search](multimodal-search-overview.md). Multimodality refers to the ability to process and query over multiple types of data, such as text and images.
16
+
In this quickstart, you use the **Import and vectorize data**wizard in the Azure portal to get started with [multimodal search](multimodal-search-overview.md). The wizard simplifies the process of extracting page text and inline images from documents, describing images in natural language, vectorizing both text and image descriptions, and storing images for later retrieval.
17
17
18
18
The sample data consists of a multimodal PDF in the [azure-search-sample-data](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/sustainable-ai-pdf) repo, but you can use different files and still follow this quickstart.
19
19
@@ -25,7 +25,7 @@ The sample data consists of a multimodal PDF in the [azure-search-sample-data](h
25
25
26
26
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills) in East US, West Europe, or North Central US.
27
27
28
-
+ An [Azure AI Search service](search-create-service-portal.md) in the same region as your Azure AI multi-service account. You can use any pricing tier.
28
+
+ An [Azure AI Search service](search-create-service-portal.md) in the same region as your Azure AI multi-service account.
29
29
30
30
+ An [Azure OpenAI resource](/azure/ai-services/openai/how-to/create-resource).
31
31
@@ -63,23 +63,31 @@ To prepare the sample data for this quickstart:
63
63
64
64
1. Create another container to store images extracted from the PDF.
65
65
66
-
## Prepare models
66
+
## Deploy models
67
67
68
-
The wizard requires a large language model (LLM) to verbalize images and an embedding model to generate vector representations of text and images. Both models are available through Azure OpenAI.
68
+
The wizard requires a large language model (LLM) to verbalize images and an embedding model to generate vector representations of text and verbalized text content. Both models are available through Azure OpenAI.
69
69
70
-
Multimodal search supports several models, but this quickstart assumes `gpt-4o` for the LLM and `text-embedding-3-large` for the embedding model.
71
-
72
-
To prepare the models for this quickstart:
70
+
To deploy the models for this quickstart:
73
71
74
72
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com) and select your Azure OpenAI resource.
75
73
76
74
1. From the left pane, select **Model catalog**.
77
75
78
-
1. Deploy `gpt-4o` and `text-embedding-3-large`.
76
+
1. Deploy one of the following LLMs:
79
77
80
-
## Start the wizard
78
+
+ gpt-4o
79
+
80
+
+ gpt-4o-mini
81
+
82
+
1. Deploy one of the following embedding models:
83
+
84
+
+ text-embedding-ada-002
85
+
86
+
+ text-embedding-3-small
81
87
82
-
If your Azure Storage blob container has the default configuration, and if your Azure AI Search service and Azure AI multi-service account are in the [same supported region](cognitive-search-skill-document-intelligence-layout.md) and tenant, you're ready to proceed.
88
+
+ text-embedding-3-large
89
+
90
+
## Start the wizard
83
91
84
92
To start the wizard for multimodal search:
85
93
@@ -151,7 +159,7 @@ To use the GenAI Prompt skill and Azure OpenAI Embedding skill:
151
159
152
160
1. For the kind, select **Azure OpenAI**.
153
161
154
-
1. Specify your Azure subscription, Azure OpenAI resource, and LLM deployment. This quickstart assumes `gpt-4o`.
162
+
1. Specify your Azure subscription, Azure OpenAI resource, and LLM deployment.
155
163
156
164
1. For the authentication type, select **System assigned identity**.
157
165
@@ -163,7 +171,7 @@ To use the GenAI Prompt skill and Azure OpenAI Embedding skill:
163
171
164
172
1. For the kind, select **Azure OpenAI**.
165
173
166
-
1. Specify your Azure subscription, Azure OpenAI resource, and embedding model deployment. This quickstart assumes `text-embedding-3-large`.
174
+
1. Specify your Azure subscription, Azure OpenAI resource, and embedding model deployment.
167
175
168
176
1. For the authentication type, select **System assigned identity**.
169
177
@@ -198,9 +206,9 @@ On the **Advanced settings** page, you can optionally add fields to the index sc
198
206
| content_id | Text and image vectors | String field. Document key for the index. | Searchable, retrievable, sortable, filterable, and facetable. |
199
207
| document_title | Text and image vectors | String field. Human-readable document title, page title, or page number. | Searchable, retrievable, sortable, filterable, and facetable. |
200
208
| text_document_id | Text vectors | String field. Identifies the parent document from which the text chunk originates. | Retrievable and filterable. |
201
-
| image_document_id | Image vectors | String field. Identifies the parent document from which the image chunk originates. | Searchable, retrievable, sortable, filterable, and facetable. |
209
+
| image_document_id | Image vectors | String field. Identifies the parent document from which the image originates. | Searchable, retrievable, sortable, filterable, and facetable. |
202
210
| content_text | Text vectors | String field. Human-readable version of the text chunk. | Searchable, retrievable, sortable, filterable, and facetable. |
203
-
| content_embedding | Image vectors | Collection(Edm.Single). Vector representation of the image chunk. | Searchable and retrievable. |
211
+
| content_embedding | Image vectors | Collection(Edm.Single). Vector representation of the image verbalization. | Searchable and retrievable. |
204
212
| content_path | Text and image vectors | String field. Path to the content in the storage container. | Retrievable, sortable, filterable, and facetable. |
205
213
| locationMetadata | Text and image vectors | Edm.ComplexType. Contains metadata about the content's location. | Varies by field. |
206
214
@@ -217,7 +225,7 @@ To add fields to the index schema:
217
225
> [!NOTE]
218
226
> Metadata fields are searchable but not retrievable, filterable, facetable, or sortable.
219
227
220
-
1.Select **Reset** if you want to restore the schema to its original version.
228
+
1.If you want to restore the schema to its original version, select **Reset**.
221
229
222
230
## Schedule indexing
223
231
@@ -233,7 +241,7 @@ To schedule indexing:
233
241
234
242
## Finish the wizard
235
243
236
-
The final step is to review your configuration and create the objects required for multimodal search. If necessary, return to previous pages in the wizard to adjust your configuration.
244
+
The final step is to review your configuration and create the necessary objects for multimodal search. If necessary, return to the previous pages in the wizard to adjust your configuration.
237
245
238
246
To finish the wizard:
239
247
@@ -268,35 +276,29 @@ When the wizard completes the configuration, it creates the following objects:
268
276
269
277
## Check results
270
278
271
-
Search Explorer accepts text, images, and vectors as query inputs. For images, Search Explorer vectorizes the image and sends the vector as a query input to the search engine. Image vectorization assumes that your index has a vectorizer definition, which the **Import and vectorize data wizard** creates based on your embedding model inputs.
279
+
This quickstart creates a multimodal index that supports [hybrid search](hybrid-search-overview.md) over both text and verbalized images. However, it doesn't support images as query inputs, which requires integrated vectorization using an embedding skill and an equivalent vectorizer. For more information, see [Query with Search explorer](search-explorer.md).
272
280
273
-
The following steps assume that you're searching for images. For the other two query types, see [Quickstart: Keyword search](search-get-started-portal.md#query-with-search-explorer) and [Quickstart: Vector search](search-get-started-portal-import-vectors.md#check-results).
281
+
Hybrid search is a combination of full-text queries and vector queries. When you issue a hybrid query, the search engine computes the semantic similarity between your query and the indexed vectors and ranks the results accordingly. For the index created in this quickstart, the results surface content from the `content_text` field that closely aligns with your query.
274
282
275
-
To use Search Explorer for image search:
283
+
To query your multimodal index:
276
284
277
285
1. Sign in to the [Azure portal](https://portal.azure.com/) and select your Azure AI Search service.
278
286
279
-
1. From the left pane, select **Search management** > **Indexes**, and then select your index.
280
-
281
-
1. Select the **Search explorer** tab.
282
-
283
-
1. From the **View** menu, select **Image view**.
284
-
285
-
:::image type="content" source="media/search-get-started-portal-images/select-image-view.png" alt-text="Screenshot of the command for selecting image view." border="true" lightbox="media/search-get-started-portal-images/select-image-view.png":::
287
+
1. From the left pane, select **Search management** > **Indexes**.
286
288
287
-
1.Drag or select a [sample PNG](https://github.com/Azure-Samples/azure-search-sample-data/blob/main/sustainable-ai-pdf) from your local folder. The PNGs come directly from the sample PDF used in this quickstart.
289
+
1.Select your index.
288
290
289
-
1. Select **Search** to run the query.
291
+
1. Select **Query options**, and then select **Hide vector values in search results**. This step makes the results more readable.
290
292
291
-
The top match should be the image for which you searched. Because a [vector search](vector-search-overview.md) matches on similar vectors, the search engine returns any document that's sufficiently similar to the query input, up to the `k` number of results. For more advanced queries that include relevance tuning, switch to the JSON view.
293
+
:::image type="content" source="media/search-get-started-portal-images/query-options.png" alt-text="Screenshot of the Query Options menu in Search Explorer." border="true" lightbox="media/search-get-started-portal-images/query-options.png":::
292
294
293
-
:::image type="content" source="media/search-get-started-portal-images/image-search.png" alt-text="Screenshot of the search results for image search." border="true" lightbox="media/search-get-started-portal-images/image-search.png":::
295
+
1. Enter text for which you want to search. Our example uses `energy`.
294
296
295
-
1.Try other query options to compare search outcomes:
297
+
1.To run the query, select **Search**.
296
298
297
-
+ (Recommended) Hide vectors for more readable results.
299
+
:::image type="content" source="media/search-get-started-portal-images/search-button.png" alt-text="Screenshot of the Search button in Search Explorer." border="true" lightbox="media/search-get-started-portal-images/search-button.png":::
298
300
299
-
+ Select a vector field to query over. The default is text vectors, but you can specify the image vector to exclude text vectors from query execution.
301
+
The results should include text and image content related to `energy` in your index. Highlights from relevant passages and image verbalizations appear in `@search.captions`, helping you quickly identify matches to your query.
Copy file name to clipboardExpand all lines: articles/search/search-get-started-portal-import-vectors.md
+6-7Lines changed: 6 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,12 +9,12 @@ ms.custom:
9
9
- build-2024
10
10
- ignite-2024
11
11
ms.topic: quickstart
12
-
ms.date: 05/20/2025
12
+
ms.date: 05/21/2025
13
13
---
14
14
15
15
# Quickstart: Vectorize text in the Azure portal
16
16
17
-
In this quickstart, you use the **Import and vectorize data wizard** in the Azure portal to get started with [integrated vectorization](vector-search-integrated-vectorization.md). The wizard chunks your content and calls an embedding model to vectorize content during indexing and for queries.
17
+
In this quickstart, you use the **Import and vectorize data** wizard in the Azure portal to get started with [integrated vectorization](vector-search-integrated-vectorization.md). The wizard chunks your content and calls an embedding model to vectorize content during indexing and for queries.
18
18
19
19
The sample data for this quickstart consists of text-based PDFs, but you can also use images and follow this quickstart to vectorize them.
20
20
@@ -447,13 +447,12 @@ You can't modify the generated fields or their attributes, but you can add new f
447
447
448
448
1. Select **Add field**.
449
449
450
-
451
450
1. Select a source field from the available fields, enter a field name for the index, and accept (or override) the default data type.
452
451
453
452
> [!NOTE]
454
453
> Metadata fields are searchable but not retrievable, filterable, facetable, or sortable.
455
454
456
-
1.Select **Reset** if you want to restore the schema to its original version.
455
+
1.If you want to restore the schema to its original version, select **Reset**.
457
456
458
457
## Schedule indexing
459
458
@@ -475,16 +474,16 @@ When the wizard completes the configuration, it creates the following objects:
475
474
476
475
+ An indexer with field mappings and output field mappings (if applicable).
477
476
478
-
> [!NOTE]
477
+
> [!TIP]
479
478
> Wizard-created objects have configurable JSON definitions. To view or modify these definitions, select **Search management** from the left pane, where you can view your indexes, indexers, data sources, and skillsets.
480
479
481
480
## Check results
482
481
483
482
Search Explorer accepts text strings as input and then vectorizes the text for vector query execution.
484
483
485
-
1. In the Azure portal, go to **Search Management** > **Indexes**, and then select the index that you created.
484
+
1. In the Azure portal, go to **Search Management** > **Indexes**, and then select your index.
486
485
487
-
1. Select **Query options** and hide vector values in search results. This step makes your search results easier to read.
486
+
1. Select **Query options**, and then select **Hide vector values in search results**. This step makes the results more readable.
488
487
489
488
:::image type="content" source="media/search-get-started-portal-import-vectors/query-options.png" alt-text="Screenshot of the button for query options.":::
0 commit comments