Skip to content

Commit a547aee

Browse files
authored
Merge pull request #229722 from PatrickFarley/comvis-4
[cog svcs] Comvis 4
2 parents 7dd85e2 + 79b15a3 commit a547aee

File tree

6 files changed

+64
-8
lines changed

6 files changed

+64
-8
lines changed
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
title: Image Retrieval concepts - Image Analysis 4.0
3+
titleSuffix: Azure Cognitive Services
4+
description: Concepts related to image vectorization using the Image Analysis 4.0 API.
5+
services: cognitive-services
6+
author: PatrickFarley
7+
manager: nitinme
8+
9+
ms.service: cognitive-services
10+
ms.subservice: computer-vision
11+
ms.topic: conceptual
12+
ms.date: 03/06/2023
13+
ms.author: pafarley
14+
---
15+
16+
# Image retrieval (version 4.0 preview)
17+
18+
Image retrieval is the process of searching a large collection of images to find those that are most similar to a given query image. Image retrieval systems have traditionally used features extracted from the images, such as content labels, tags, and image descriptors, to compare images and rank them by similarity. However, vector similarity search is gaining more popularity due to a number of benefits over traditional keyword-based search and is becoming a vital component in popular content search services.
19+
20+
## What's the difference between vector search and keyword-based search?
21+
22+
Keyword search is the most basic and traditional method of information retrieval. In this approach, the search engine looks for the exact match of the keywords or phrases entered by the user in the search query and compares with labels and tags provided for the images. The search engine then returns images that contain those exact keywords as content tags and image labels. Keyword search relies heavily on the user's ability to input relevant and specific search terms.
23+
24+
Vector search, on the other hand, searches large collections of vectors in high-dimensional space to find vectors that are similar to a given query. Vector search looks for semantic similarities by capturing the context and meaning of the search query. This approach is often more efficient than traditional image retrieval techniques, as it can reduce search space and improve the accuracy of the results.
25+
26+
## Business Applications
27+
28+
Image retrieval has a variety of applications in different fields, including:
29+
30+
- Digital asset management: Image retrieval can be used to manage large collections of digital images, such as in museums, archives, or online galleries. Users can search for images based on visual features and retrieve the images that match their criteria.
31+
- Medical image retrieval: Image retrieval can be used in medical imaging to search for images based on their diagnostic features or disease patterns. This can help doctors or researchers to identify similar cases or track disease progression.
32+
- Security and surveillance: Image retrieval can be used in security and surveillance systems to search for images based on specific features or patterns, such as in, people & object tracking, or threat detection.
33+
- Forensic image retrieval: Image retrieval can be used in forensic investigations to search for images based on their visual content or metadata, such as in cases of cyber-crime.
34+
- E-commerce: Image retrieval can be used in online shopping applications to search for similar products based on their features or descriptions or provide recommendations based on previous purchases.
35+
- Fashion and design: Image retrieval can be used in fashion and design to search for images based on their visual features, such as color, pattern, or texture. This can help designers or retailers to identify similar products or trends.
36+
37+
## What are vector embeddings?
38+
39+
Vector embeddings are a way of representing content—text or images—as vectors of real numbers in a high-dimensional space. Vector embeddings are often learned from large amounts of textual and visual data using machine learning algorithms, such as neural networks. Each dimension of the vector corresponds to a different feature or attribute of the content, such as its semantic meaning, syntactic role, or context in which it commonly appears.
40+
41+
> [!NOTE]
42+
> Vector embeddings can only be meaningfully compared if they are from the same model type.
43+
44+
## How does it work?
45+
46+
1. Vectorize Images and Text: the Image Retrieval APIs, **VectorizeImage** and **VectorizeText**, can be used to extract feature vectors out of an image or text respectively. The APIs return a single feature vector representing the entire input.
47+
- Measure similarity: Vector search systems typically use distance metrics, such as cosine distance or Euclidean distance, to compare vectors and rank them by similarity. The [Vision studio](https://portal.vision.cognitive.azure.com/) demo uses [cosine distance](./how-to/image-retrieval.md#calculate-vector-similarity) to measure similarity.
48+
- Retrieve Images: Use the top _N_ vectors similar to the search query and retrieve images corresponding to those vectors from your photo library to provide as the final result.
49+
50+
## Next steps
51+
52+
Enable image retrieval for your search service and follow the steps to generate vector embeddings for text and images.
53+
* [Call the Image retrieval APIs](./how-to/image-retrieval.md)
54+

articles/cognitive-services/Computer-vision/concept-model-customization.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,10 @@ Below are the possible reasons:
174174

175175
We do not recommend you use custom models for business critical environments due to potential high latency. When customers train custom models in Vision Studio, those custom models belong to the Computer Vision resource that they were trained under, and the customer is able to make calls to those models using the **Analyze Image** API. When they make these calls, the custom model is loaded in memory, and the prediction infrastructure is initialized. While this happens, customers might experience longer than expected latency to receive prediction results.
176176

177+
## Data privacy and security
178+
179+
As with all of the Cognitive Services, developers using Image Analysis model customization should be aware of Microsoft's policies on customer data. See the [Cognitive Services page](https://www.microsoft.com/trustcenter/cloudservices/cognitiveservices) on the Microsoft Trust Center to learn more.
180+
177181
## Next steps
178182

179183
[Create and train a custom model](./how-to/model-customization.md)

articles/cognitive-services/Computer-vision/how-to/blob-storage-search.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ To get started with the **Search photos with natural language** or **Video summa
2525

2626
## Create a new storage account
2727

28-
To get started, [create a new storage account](https://ms.portal.azure.com/#create/Microsoft.StorageAccount).
28+
To get started, <a href="https://ms.portal.azure.com/#create/Microsoft.StorageAccount" title="create a new storage account" target="_blank">create a new storage account</a>.
2929

3030
:::image type="content" source="../media/storage-instructions/create-storage.png" alt-text="Screenshot of Blob storage creation.":::
3131

articles/cognitive-services/Computer-vision/how-to/image-retrieval.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,4 +106,4 @@ public static float GetCosineSimilarity(float[] vector1, float[] vector2)
106106

107107
## Next steps
108108

109-
[Image Analysis overview](../overview-image-analysis.md)
109+
[Image retrieval concepts](../concept-image-retrieval.md)

articles/cognitive-services/Computer-vision/overview-image-analysis.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ The Image Retrieval APIs enable the _vectorization_ of images and text queries.
126126

127127
These APIs are only available in the following geographic regions: East US, France Central, Korea Central, North Europe, Southeast Asia, West Europe, West US.
128128

129-
[Do Image Retrieval using vectorization](./how-to/image-retrieval.md)
129+
[Image Retrieval](./concept-image-retrieval.md)
130130

131131
## Background removal (v4.0 preview only)
132132

articles/cognitive-services/Computer-vision/toc.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -114,10 +114,6 @@ items:
114114
items:
115115
- name: Transparency note
116116
href: /legal/cognitive-services/computer-vision/imageanalysis-transparency-note?context=/azure/cognitive-services/computer-vision/context/context
117-
- name: Characteristics and limitations
118-
href: /legal/cognitive-services/computer-vision/imageanalysis-characteristics-and-limitations?context=/azure/cognitive-services/computer-vision/context/context
119-
- name: Integration and responsible use
120-
href: /legal/cognitive-services/computer-vision/imageanalysis-guidance-for-integration?context=/azure/cognitive-services/computer-vision/context/context
121117
- name: Data, privacy, and security
122118
href: /legal/cognitive-services/computer-vision/imageanalysis-data-privacy-security?context=/azure/cognitive-services/computer-vision/context/context
123119
- name: Limited Access features
@@ -132,7 +128,7 @@ items:
132128
href: how-to/model-customization.md
133129
- name: Migrate a Custom Vision project to Image Analysis
134130
href: how-to/migrate-from-custom-vision.md
135-
- name: Call the Image Retrieval APIs
131+
- name: Call the Image retrieval APIs
136132
href: how-to/image-retrieval.md
137133
- name: Call the Background removal API
138134
href: how-to/background-removal.md
@@ -164,6 +160,8 @@ items:
164160
href: concept-people-detection.md
165161
- name: Model customization
166162
href: concept-model-customization.md
163+
- name: Image retrieval
164+
href: concept-image-retrieval.md
167165
- name: Background removal
168166
href: concept-background-removal.md
169167
- name: Version 3.2

0 commit comments

Comments
 (0)