Skip to content

Commit 059966e

Browse files
committed
Updated RAG quickstarts
1 parent 5c5d01b commit 059966e

File tree

5 files changed

+43
-42
lines changed

5 files changed

+43
-42
lines changed
9.76 KB
Loading
65.7 KB
Loading
29.6 KB
Loading

articles/search/search-get-started-portal-image-search.md

Lines changed: 37 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
---
22
title: "Quickstart: Multimodal Search in the Azure portal"
33
titleSuffix: Azure AI Search
4-
description: Learn how to search for multimodal content on an Azure AI Search index in the Azure portal. Run a wizard to vectorize text and images, and then use Search Explorer to provide an image as your query input.
4+
description: Learn how to search for multimodal content on an Azure AI Search index in the Azure portal. Run a wizard to generate natural-language descriptions of images and vectorize both text and images, and then use Search Explorer to query your multimodal index.
55
author: haileytap
66
ms.author: haileytapia
77
ms.service: azure-ai-search
88
ms.topic: quickstart
9-
ms.date: 05/20/2025
9+
ms.date: 05/21/2025
1010
ms.custom:
1111
- references_regions
1212
---
1313

1414
# Quickstart: Search for multimodal content in the Azure portal
1515

16-
In this quickstart, you use the **Import and vectorize data wizard** in the Azure portal to get started with [multimodal search](multimodal-search-overview.md). Multimodality refers to the ability to process and query over multiple types of data, such as text and images.
16+
In this quickstart, you use the **Import and vectorize data** wizard in the Azure portal to get started with [multimodal search](multimodal-search-overview.md). The wizard simplifies the process of extracting page text and inline images from documents, describing images in natural language, vectorizing both text and image descriptions, and storing images for later retrieval.
1717

1818
The sample data consists of a multimodal PDF in the [azure-search-sample-data](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/sustainable-ai-pdf) repo, but you can use different files and still follow this quickstart.
1919

@@ -25,7 +25,7 @@ The sample data consists of a multimodal PDF in the [azure-search-sample-data](h
2525

2626
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills) in East US, West Europe, or North Central US.
2727

28-
+ An [Azure AI Search service](search-create-service-portal.md) in the same region as your Azure AI multi-service account. You can use any pricing tier.
28+
+ An [Azure AI Search service](search-create-service-portal.md) in the same region as your Azure AI multi-service account.
2929

3030
+ An [Azure OpenAI resource](/azure/ai-services/openai/how-to/create-resource).
3131

@@ -63,23 +63,31 @@ To prepare the sample data for this quickstart:
6363

6464
1. Create another container to store images extracted from the PDF.
6565

66-
## Prepare models
66+
## Deploy models
6767

68-
The wizard requires a large language model (LLM) to verbalize images and an embedding model to generate vector representations of text and images. Both models are available through Azure OpenAI.
68+
The wizard requires a large language model (LLM) to verbalize images and an embedding model to generate vector representations of text and verbalized text content. Both models are available through Azure OpenAI.
6969

70-
Multimodal search supports several models, but this quickstart assumes `gpt-4o` for the LLM and `text-embedding-3-large` for the embedding model.
71-
72-
To prepare the models for this quickstart:
70+
To deploy the models for this quickstart:
7371

7472
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com) and select your Azure OpenAI resource.
7573

7674
1. From the left pane, select **Model catalog**.
7775

78-
1. Deploy `gpt-4o` and `text-embedding-3-large`.
76+
1. Deploy one of the following LLMs:
7977

80-
## Start the wizard
78+
+ gpt-4o
79+
80+
+ gpt-4o-mini
81+
82+
1. Deploy one of the following embedding models:
83+
84+
+ text-embedding-ada-002
85+
86+
+ text-embedding-3-small
8187

82-
If your Azure Storage blob container has the default configuration, and if your Azure AI Search service and Azure AI multi-service account are in the [same supported region](cognitive-search-skill-document-intelligence-layout.md) and tenant, you're ready to proceed.
88+
+ text-embedding-3-large
89+
90+
## Start the wizard
8391

8492
To start the wizard for multimodal search:
8593

@@ -151,7 +159,7 @@ To use the GenAI Prompt skill and Azure OpenAI Embedding skill:
151159

152160
1. For the kind, select **Azure OpenAI**.
153161

154-
1. Specify your Azure subscription, Azure OpenAI resource, and LLM deployment. This quickstart assumes `gpt-4o`.
162+
1. Specify your Azure subscription, Azure OpenAI resource, and LLM deployment.
155163

156164
1. For the authentication type, select **System assigned identity**.
157165

@@ -163,7 +171,7 @@ To use the GenAI Prompt skill and Azure OpenAI Embedding skill:
163171

164172
1. For the kind, select **Azure OpenAI**.
165173

166-
1. Specify your Azure subscription, Azure OpenAI resource, and embedding model deployment. This quickstart assumes `text-embedding-3-large`.
174+
1. Specify your Azure subscription, Azure OpenAI resource, and embedding model deployment.
167175

168176
1. For the authentication type, select **System assigned identity**.
169177

@@ -198,9 +206,9 @@ On the **Advanced settings** page, you can optionally add fields to the index sc
198206
| content_id | Text and image vectors | String field. Document key for the index. | Searchable, retrievable, sortable, filterable, and facetable. |
199207
| document_title | Text and image vectors | String field. Human-readable document title, page title, or page number. | Searchable, retrievable, sortable, filterable, and facetable. |
200208
| text_document_id | Text vectors | String field. Identifies the parent document from which the text chunk originates. | Retrievable and filterable. |
201-
| image_document_id | Image vectors | String field. Identifies the parent document from which the image chunk originates. | Searchable, retrievable, sortable, filterable, and facetable. |
209+
| image_document_id | Image vectors | String field. Identifies the parent document from which the image originates. | Searchable, retrievable, sortable, filterable, and facetable. |
202210
| content_text | Text vectors | String field. Human-readable version of the text chunk. | Searchable, retrievable, sortable, filterable, and facetable. |
203-
| content_embedding | Image vectors | Collection(Edm.Single). Vector representation of the image chunk. | Searchable and retrievable. |
211+
| content_embedding | Image vectors | Collection(Edm.Single). Vector representation of the image verbalization. | Searchable and retrievable. |
204212
| content_path | Text and image vectors | String field. Path to the content in the storage container. | Retrievable, sortable, filterable, and facetable. |
205213
| locationMetadata | Text and image vectors | Edm.ComplexType. Contains metadata about the content's location. | Varies by field. |
206214

@@ -217,7 +225,7 @@ To add fields to the index schema:
217225
> [!NOTE]
218226
> Metadata fields are searchable but not retrievable, filterable, facetable, or sortable.
219227
220-
1. Select **Reset** if you want to restore the schema to its original version.
228+
1. If you want to restore the schema to its original version, select **Reset**.
221229

222230
## Schedule indexing
223231

@@ -233,7 +241,7 @@ To schedule indexing:
233241

234242
## Finish the wizard
235243

236-
The final step is to review your configuration and create the objects required for multimodal search. If necessary, return to previous pages in the wizard to adjust your configuration.
244+
The final step is to review your configuration and create the necessary objects for multimodal search. If necessary, return to the previous pages in the wizard to adjust your configuration.
237245

238246
To finish the wizard:
239247

@@ -268,35 +276,29 @@ When the wizard completes the configuration, it creates the following objects:
268276
269277
## Check results
270278

271-
Search Explorer accepts text, images, and vectors as query inputs. For images, Search Explorer vectorizes the image and sends the vector as a query input to the search engine. Image vectorization assumes that your index has a vectorizer definition, which the **Import and vectorize data wizard** creates based on your embedding model inputs.
279+
This quickstart creates a multimodal index that supports [hybrid search](hybrid-search-overview.md) over both text and verbalized images. However, it doesn't support images as query inputs, which requires integrated vectorization using an embedding skill and an equivalent vectorizer. For more information, see [Query with Search explorer](search-explorer.md).
272280

273-
The following steps assume that you're searching for images. For the other two query types, see [Quickstart: Keyword search](search-get-started-portal.md#query-with-search-explorer) and [Quickstart: Vector search](search-get-started-portal-import-vectors.md#check-results).
281+
Hybrid search is a combination of full-text queries and vector queries. When you issue a hybrid query, the search engine computes the semantic similarity between your query and the indexed vectors and ranks the results accordingly. For the index created in this quickstart, the results surface content from the `content_text` field that closely aligns with your query.
274282

275-
To use Search Explorer for image search:
283+
To query your multimodal index:
276284

277285
1. Sign in to the [Azure portal](https://portal.azure.com/) and select your Azure AI Search service.
278286

279-
1. From the left pane, select **Search management** > **Indexes**, and then select your index.
280-
281-
1. Select the **Search explorer** tab.
282-
283-
1. From the **View** menu, select **Image view**.
284-
285-
:::image type="content" source="media/search-get-started-portal-images/select-image-view.png" alt-text="Screenshot of the command for selecting image view." border="true" lightbox="media/search-get-started-portal-images/select-image-view.png":::
287+
1. From the left pane, select **Search management** > **Indexes**.
286288

287-
1. Drag or select a [sample PNG](https://github.com/Azure-Samples/azure-search-sample-data/blob/main/sustainable-ai-pdf) from your local folder. The PNGs come directly from the sample PDF used in this quickstart.
289+
1. Select your index.
288290

289-
1. Select **Search** to run the query.
291+
1. Select **Query options**, and then select **Hide vector values in search results**. This step makes the results more readable.
290292

291-
The top match should be the image for which you searched. Because a [vector search](vector-search-overview.md) matches on similar vectors, the search engine returns any document that's sufficiently similar to the query input, up to the `k` number of results. For more advanced queries that include relevance tuning, switch to the JSON view.
293+
:::image type="content" source="media/search-get-started-portal-images/query-options.png" alt-text="Screenshot of the Query Options menu in Search Explorer." border="true" lightbox="media/search-get-started-portal-images/query-options.png":::
292294

293-
:::image type="content" source="media/search-get-started-portal-images/image-search.png" alt-text="Screenshot of the search results for image search." border="true" lightbox="media/search-get-started-portal-images/image-search.png":::
295+
1. Enter text for which you want to search. Our example uses `energy`.
294296

295-
1. Try other query options to compare search outcomes:
297+
1. To run the query, select **Search**.
296298

297-
+ (Recommended) Hide vectors for more readable results.
299+
:::image type="content" source="media/search-get-started-portal-images/search-button.png" alt-text="Screenshot of the Search button in Search Explorer." border="true" lightbox="media/search-get-started-portal-images/search-button.png":::
298300

299-
+ Select a vector field to query over. The default is text vectors, but you can specify the image vector to exclude text vectors from query execution.
301+
The results should include text and image content related to `energy` in your index. Highlights from relevant passages and image verbalizations appear in `@search.captions`, helping you quickly identify matches to your query.
300302

301303
## Clean up resources
302304

articles/search/search-get-started-portal-import-vectors.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ ms.custom:
99
- build-2024
1010
- ignite-2024
1111
ms.topic: quickstart
12-
ms.date: 05/20/2025
12+
ms.date: 05/21/2025
1313
---
1414

1515
# Quickstart: Vectorize text in the Azure portal
1616

17-
In this quickstart, you use the **Import and vectorize data wizard** in the Azure portal to get started with [integrated vectorization](vector-search-integrated-vectorization.md). The wizard chunks your content and calls an embedding model to vectorize content during indexing and for queries.
17+
In this quickstart, you use the **Import and vectorize data** wizard in the Azure portal to get started with [integrated vectorization](vector-search-integrated-vectorization.md). The wizard chunks your content and calls an embedding model to vectorize content during indexing and for queries.
1818

1919
The sample data for this quickstart consists of text-based PDFs, but you can also use images and follow this quickstart to vectorize them.
2020

@@ -447,13 +447,12 @@ You can't modify the generated fields or their attributes, but you can add new f
447447

448448
1. Select **Add field**.
449449

450-
451450
1. Select a source field from the available fields, enter a field name for the index, and accept (or override) the default data type.
452451

453452
> [!NOTE]
454453
> Metadata fields are searchable but not retrievable, filterable, facetable, or sortable.
455454
456-
1. Select **Reset** if you want to restore the schema to its original version.
455+
1. If you want to restore the schema to its original version, select **Reset**.
457456

458457
## Schedule indexing
459458

@@ -475,16 +474,16 @@ When the wizard completes the configuration, it creates the following objects:
475474

476475
+ An indexer with field mappings and output field mappings (if applicable).
477476

478-
> [!NOTE]
477+
> [!TIP]
479478
> Wizard-created objects have configurable JSON definitions. To view or modify these definitions, select **Search management** from the left pane, where you can view your indexes, indexers, data sources, and skillsets.
480479
481480
## Check results
482481

483482
Search Explorer accepts text strings as input and then vectorizes the text for vector query execution.
484483

485-
1. In the Azure portal, go to **Search Management** > **Indexes**, and then select the index that you created.
484+
1. In the Azure portal, go to **Search Management** > **Indexes**, and then select your index.
486485

487-
1. Select **Query options** and hide vector values in search results. This step makes your search results easier to read.
486+
1. Select **Query options**, and then select **Hide vector values in search results**. This step makes the results more readable.
488487

489488
:::image type="content" source="media/search-get-started-portal-import-vectors/query-options.png" alt-text="Screenshot of the button for query options.":::
490489

0 commit comments

Comments
 (0)