You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/search-get-started-portal-image-search.md
+39-23Lines changed: 39 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,37 +8,43 @@ ms.author: heidist
8
8
ms.service: cognitive-search
9
9
ms.topic: quickstart
10
10
ms.date: 06/14/2024
11
+
ms.custom:
12
+
- references_regions
11
13
---
12
14
13
-
# Quickstart: Image search in Azure portal (preview)
15
+
# Quickstart: Image search using Search Explorer in Azure portal
14
16
15
17
> [!IMPORTANT]
16
-
> **Import and vectorize data** wizard and Azure AI Vision vectorizers are in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). By default, it targets the [2024-05-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true).
18
+
> Image vectors are supported in stable API versions, but the wizard and vectorizers are in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). By default, the wizard targets the [2024-05-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true).
17
19
18
-
Get started with image search using the **Import and vectorize data** wizard in the Azure portal and image search in **Search explorer**.
20
+
Get started with image search using the **Import and vectorize data** wizard in the Azure portal and use **Search explorer** to run image-based queries.
19
21
20
-
You need three Azure resources and sample data to complete this walkthrough:
22
+
You need three Azure resources and some sample image files to complete this walkthrough:
21
23
22
24
> [!div class="checklist"]
23
25
> + Azure Storage to store image files as blobs
24
-
> + Azure AI services multiservice account, used for image vectorization and image analysis
26
+
> + Azure AI services multiservice account, used for image vectorization and Optical Character Recognition (OCR)
25
27
> + Azure AI Search for indexing and queries
26
28
27
-
Sample data consists of image files in the azure-search-sample-data repo, but you can use different images and still follow this walkthrough.
29
+
Sample data consists of image files in the [azure-search-sample-data](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/unsplash-images) repo, but you can use different images and still follow this walkthrough.
28
30
29
31
## Prerequisites
30
32
31
33
+ An Azure subscription. [Create one for free](https://azure.microsoft.com/free/).
32
34
33
-
+ Azure AI services, a multiservice account, in a region that provides Azure AI Vision multimodal embeddings. Currently, those regions are: SwedenCentral, EastUS, NorthEurope, WestEurope, WestUS, SoutheastAsia, KoreaCentral, FranceCentral, AustraliaEast, WestUS2, SwitzerlandNorth, JapanEast. [Check the documentation](/azure/ai-services/computer-vision/how-to/image-retrieval) for an updated list.
35
+
+ Azure AI services, a multiservice account, in a region that provides Azure AI Vision multimodal embeddings.
34
36
35
-
+ Azure AI Search, on any tier, but in the same region as Azure AI services. The tier of your search service determines how many blobs you can index. We used the free tier to create this walkthrough and we limited the content to 10 JPG files.
37
+
Currently, those regions are: SwedenCentral, EastUS, NorthEurope, WestEurope, WestUS, SoutheastAsia, KoreaCentral, FranceCentral, AustraliaEast, WestUS2, SwitzerlandNorth, JapanEast. [Check the documentation](/azure/ai-services/computer-vision/how-to/image-retrieval) for an updated list.
36
38
37
-
+ Azure Storage, use a standard performance (general-purpose v2) account. Access tiers can be hot, cool, and cold.
39
+
+ Azure AI Search, on any tier, but in the same region as Azure AI services.
40
+
41
+
Service tier determines how many blobs you can index. We used the free tier to create this walkthrough and limited the content to 10 JPG files.
42
+
43
+
+ Azure Storage, a standard performance (general-purpose v2) account. Access tiers can be hot, cool, and cold.
38
44
39
45
All of the above resources must have public access enabled for the portal nodes to be able to access them. Otherwise, the wizard fails. After the wizard runs, firewalls and private endpoints can be enabled on the different integration components for security.
40
46
41
-
A free search service supports role-based access control on connections to Azure AI Search, but it doesn't support managed identities on outbound connections to Azure Storage or Azure AI Vision. This means you must use key-based authentication on free search service connections to other Azure services. Or, you can use basic or above and [configure a managed identity](search-howto-managed-identities-data-sources.md) and role assignment to admit requests from Azure AI Search.
47
+
A free search service supports role-based access control on connections to Azure AI Search, but it doesn't support managed identities on outbound connections to Azure Storage or Azure AI Vision. This means you must use key-based authentication on free search service connections to other Azure services. For more secure connections, use basic tier or above and [configure a managed identity](search-howto-managed-identities-data-sources.md) and role assignments to admit requests from Azure AI Search on other Azure services.
42
48
43
49
## Check for space
44
50
@@ -56,7 +62,7 @@ If you're starting with the free service, you're limited to three indexes, three
56
62
57
63
## Start the wizard
58
64
59
-
If your Azure AI multiservice account and Azure AI Search are in the same supported region and tenant, and if your Azure Storage blob container is using the default configuration, you're ready to start the wizard.
65
+
If your search service and Azure AI service are located in the same [supported region](/azure/ai-services/computer-vision/how-to/image-retrieval) and tenant, and if your Azure Storage blob container is using the default configuration, you're ready to start the wizard.
60
66
61
67
1. Sign in to the [Azure portal](https://portal.azure.com/) with your Azure account, and go to your Azure AI Search service.
62
68
@@ -66,9 +72,9 @@ If your Azure AI multiservice account and Azure AI Search are in the same suppor
66
72
67
73
## Connect to your data
68
74
69
-
The next step is to connect to a data source to use for the search index.
75
+
The next step is to connect to a data source that provides the images.
70
76
71
-
1.In the **Import and vectorize data** wizard on the **Connect to your data** tab, expand the **Data Source** dropdown list and select **Azure Blob Storage**.
77
+
1.On the **Connect to your data** tab, select **Azure Blob Storage**.
72
78
73
79
1. Specify the Azure subscription.
74
80
@@ -82,9 +88,9 @@ The next step is to connect to a data source to use for the search index.
82
88
83
89
If raw content includes text, or if the skillset produces text, the wizard calls a text embedding model to generate vectors for that content. In this exercise, text will be produced from the Optical Character Recognition (OCR) skill that you add in the next step.
84
90
85
-
Azure AI Vision model provides text embeddings, so we'll use that model for text vectorization.
91
+
Azure AI Vision provides text embeddings, so we'll use that resource for text vectorization.
86
92
87
-
1. On the **Vectorize text** page, select **AI Vision vectorization**. If it's not selectable, make sure Azure AI Search and Azure AI multiservice account are together in a region that [supports AI Vision multimodal APIs](/azure/ai-services/computer-vision/how-to/image-retrieval).
93
+
1. On the **Vectorize your text** page, select **AI Vision vectorization**. If it's not selectable, make sure Azure AI Search and your Azure AI multiservice account are together in a region that [supports AI Vision multimodal APIs](/azure/ai-services/computer-vision/how-to/image-retrieval).
88
94
89
95
:::image type="content" source="media/search-get-started-portal-images/vectorize-your-text.png" alt-text="Screenshot of the Vectorize your text page in the wizard.":::
90
96
@@ -94,9 +100,15 @@ Azure AI Vision model provides text embeddings, so we'll use that model for text
94
100
95
101
Use Azure AI Vision to generate a vector representation of the image files.
96
102
97
-
In this step, you can also set enrichment options to extract text from images. The wizard uses OCR from Azure AI services to recognize text in image files. Two more outputs appear in the index when OCR is added to the workflow. First, the "chunk" field is populated with the OCR-generated string. Second, the "text_vector" field is populated with an embedding that represents the string. The inclusion of plain text in an index is useful if you want to use relevance features that operate on strings, such as semantic ranker and scoring profiles.
103
+
In this step, you can also apply AI to extract text from images. The wizard uses OCR from Azure AI services to recognize text in image files. Two more outputs appear in the index when OCR is added to the workflow:
104
+
105
+
+ First, the "chunk" field is populated with the OCR-generated string.
106
+
107
+
+ Second, the "text_vector" field is populated with an embedding that represents the string.
108
+
109
+
The inclusion of plain text in an index is useful if you want to use relevance features that operate on strings, such as semantic ranking and scoring profiles.
98
110
99
-
1. On the **Vectorize images** page, select the **Vectorize images** checkbox, and then select **AI Vision vectorization**.
111
+
1. On the **Vectorize your images** page, select the **Vectorize images** checkbox, and then select **AI Vision vectorization**.
100
112
101
113
1. Select **Use same AI service selected for text vectorization**.
102
114
@@ -131,16 +143,20 @@ In this step, you can also set enrichment options to extract text from images. T
131
143
+ Skillset with the following five skills:
132
144
133
145
+[OCR skill](cognitive-search-skill-ocr.md) recognizes text in image files.
134
-
+[Text Merger skill](cognitive-search-skill-textmerger.md) reunites the various outputs of OCR processing.
135
-
+[Text Split skill](cognitive-search-skill-textsplit.md) adds data chunking. This skill is part of the wizard workflow, although for this data, chunking isn't technically necessary.
136
-
+[Azure AI Vision multimodal](cognitive-search-skill-vision-vectorize.md) is used to vectorize OCR-generated text.
146
+
147
+
+[Text Merger skill](cognitive-search-skill-textmerger.md) unifies the various outputs of OCR processing.
148
+
149
+
+[Text Split skill](cognitive-search-skill-textsplit.md) adds data chunking. This skill is built into the wizard workflow.
150
+
151
+
+[Azure AI Vision multimodal](cognitive-search-skill-vision-vectorize.md) is used to vectorize text generated from OCR.
152
+
137
153
+[Azure AI Vision multimodal](cognitive-search-skill-vision-vectorize.md) is called again to vectorize images.
138
154
139
155
+ Indexer with field mappings and output field mappings.
140
156
141
157
## Check results
142
158
143
-
Search explorer accepts text, vectors, and images as query inputs. You can drag or select an image into the search area, and it will be vectorized for search. Image vectorization assumes that your index has a vectorizer definition, which the **Import and vectorize data**wizard creates using your selections.
159
+
Search explorer accepts text, vectors, and images as query inputs. You can drag or select an image into the search area, and it will be vectorized for search. Image vectorization assumes that your index has a vectorizer definition, which **Import and vectorize data** creates from your inputs.
144
160
145
161
1. In the Azure portal, under **Search Management** and **Indexes**, select the index your created. An embedded Search Explorer is the first tab.
146
162
@@ -154,9 +170,9 @@ Search explorer accepts text, vectors, and images as query inputs. You can drag
154
170
155
171
:::image type="content" source="media/search-get-started-portal-images/image-search.png" alt-text="Screenshot of search results.":::
156
172
157
-
The top match should be the image you searched for.
173
+
The top match should be the image you searched for. Because a vector search matches on similar vectors, the search engine returns any document that is sufficiently similar to the query input. You can switch to JSON view for more advanced queries that include relevance tuning.
158
174
159
-
1. Try the query options to compare search outcomes:
175
+
1. Try other query options to compare search outcomes:
160
176
161
177
+ Hide vectors for more readable results.
162
178
+ Select a vector field to query over. The default is text vectors, but you can specify the image vector to exclude text vectors from query execution.
0 commit comments