You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This article describes vector indexes in Azure Machine Learning that you can use to perform retrieval-augmented generation (RAG). A vector index stores embeddings that are numerical representations of _concepts_ (data) converted to number sequences. Embeddings enable large language models (LLMs) to understand the relationships between the concepts. You can create vector stores to connect your data with LLMs like GPT-4, and retrieve the data efficiently.
24
+
This article describes vector stores in Azure Machine Learning that you can use to perform retrieval-augmented generation (RAG). A vector store contains embeddings, which are numerical representations of concepts (data) converted to number sequences. Embeddings enable large language models (LLMs) to understand relationships between concepts. You can create vector stores to connect your data with LLMs like GPT-4 and retrieve data efficiently.
25
25
26
26
Azure Machine Learning supports two vector stores that contain your supplemental data used in a RAG workflow:
27
27
28
28
| Vector store | Description | Features and usage |
29
29
| --- | --- | --- |
30
-
|**Faiss**| Open source library | - Use local file-based store <br> - Incur minimal costs <br> - Support vector-only data <br> - Support development and testing |
31
-
|**Azure AI Search**| Azure PaaS resource | - Store text data in search index <br> - Host large number of indexes with single service <br> - Support enterprise-level business requirements <br> - Access hybrid information retrieval |
30
+
|**Faiss**| Open source library | - Uses a local file-based store <br> - Incurs minimal costs <br> - Supports vector-only data <br> - Supports development and testing |
31
+
|**Azure AI Search**| Azure PaaS resource | - Stores text data in search indexes <br> - Hosts large numbers of indexes with a single service <br> - Supports enterprise-level business requirements <br> - Provides hybrid information retrieval |
32
32
33
33
The following sections explore considerations for working with these vector stores.
34
34
35
35
## Faiss library
36
36
37
-
[Faiss](https://github.com/facebookresearch/faiss) is an open source library that provides a local file-based store. The vector index is stored in the Azure storage account of your Azure Machine Learning workspace. To work with Faiss, you download the library and use it as a component of your solution. Because the index is stored locally, the costs are minimal.
37
+
[Faiss](https://github.com/facebookresearch/faiss) is an open source library that provides a local file-based store. The vector index is stored in the Azure storage account of your Azure Machine Learning workspace. To work with Faiss, you download the library and use it as a component of your solution. Because the index is stored locally, costs are minimal.
38
38
39
-
You can use the Faiss library as your vector store and perform the following actions:
39
+
You can use the Faiss library as your vector store to perform the following actions:
40
40
41
-
- Store vector data locally, with no costs for creating an index (only storage cost)
41
+
- Store vector data locally with no costs for creating an index (only storage costs apply)
42
42
43
43
- Build and query an index in memory
44
44
45
-
- Share copies for individual use, and configure hosting of the index for an application
45
+
- Share copies for individual use and configure hosting of the index for an application
46
46
47
-
- Scale with underlying compute loading index
47
+
- Scale with underlying compute when loading the index
48
48
49
49
## Azure AI Search
50
50
51
-
[Azure AI Search](/azure/search/search-what-is-azure-search)(formerly Cognitive Search) is a dedicated Azure PaaS resource that you create in an Azure subscription. The resource supports information retrieval over your vector and textual data stored in search indexes. A prompt flow can create, populate, and query your vector data stored in Azure AI Search. A single search service can host a large number of indexes, which can be queried and used in a RAG pattern.
51
+
[Azure AI Search](/azure/search/search-what-is-azure-search) is a dedicated Azure PaaS resource that you create in an Azure subscription. The resource supports information retrieval over your vector and textual data stored in search indexes. A prompt flow can create, populate, and query your vector data stored in Azure AI Search. A single search service can host many indexes, which can be queried and used in a RAG pattern.
52
52
53
-
Here are some key points about using Azure AI Search for your vector store:
53
+
Here are key points about using Azure AI Search for your vector store:
54
54
55
-
-Support enterpriselevel business requirements for scale, security, and availability.
55
+
-Supports enterprise-level business requirements for scale, security, and availability.
56
56
57
-
-Access hybrid information retrieval. Vector data can coexist with nonvector data, which means you can use any of the [features of Azure AI Search](/azure/search/search-features-list) for indexing and queries, including [hybrid search](/azure/search/vector-search-how-to-query) and [semantic reranking](/azure/search/semantic-search-overview).
57
+
-Provides hybrid information retrieval. Vector data can coexist with nonvector data, which means you can use any of the [features of Azure AI Search](/azure/search/search-features-list) for indexing and queries, including [hybrid search](/azure/search/vector-search-how-to-query) and [semantic reranking](/azure/search/semantic-search-overview).
58
58
59
-
-Keep in mind that [vector support is in preview](/azure/search/vector-search-overview). Currently, vectors must be generated externally and then passed to Azure AI Search for indexing and query encoding. The prompt flow handles these transitions for you.
59
+
-[Vector support is in preview](/azure/search/vector-search-overview). Currently, vectors must be generated externally and then passed to Azure AI Search for indexing and query encoding. The prompt flow handles these transitions for you.
60
60
61
-
To use AI Search as a vector store for Azure Machine Learning, [you must have a search service](/azure/search/search-create-service-portal). After the service exists, and you grant access to developers, you can choose **Azure AI Search** as a vector index in a prompt flow. The prompt flow creates the index on Azure AI Search, generates vectors from your source data, sends the vectors to the index, invokes similarity search on AI Search, and returns the response.
61
+
To use Azure AI Search as a vector store for Azure Machine Learning, [you must have a search service](/azure/search/search-create-service-portal). After the service exists and you grant access to developers, you can choose **Azure AI Search** as a vector index in a prompt flow. The prompt flow creates the index on Azure AI Search, generates vectors from your source data, sends the vectors to the index, invokes similarity search on Azure AI Search, and returns the response.
62
62
63
63
## Related content
64
64
65
-
-[Create vector index in Azure Machine Learning prompt flow (preview)](how-to-create-vector-index.md)
65
+
-[Create a vector index in Azure Machine Learning prompt flow (preview)](how-to-create-vector-index.md)
66
66
-[Vectors in Azure AI Search](/azure/search/vector-search-overview)
0 commit comments