Skip to content

Commit 0c0f343

Browse files
jcodellajcodella
authored
Update vector-search.md
1 parent 899fff3 commit 0c0f343

File tree

1 file changed

+154
-5
lines changed

1 file changed

+154
-5
lines changed

‎articles/cosmos-db/mongodb/vcore/vector-search.md

Lines changed: 154 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,9 @@ You can create (Hierarchical Navigable Small World) indexes on M40 cluster tiers
5555

5656
|Field |Type |Description |
5757
|---------|---------|---------|
58-
| `kind` | string | Type of vector index to create. Type of vector index to create. `vector-hnsw` is available on M40 cluster tiers and higher.|
58+
| `index_name` | string | Unique name of the index. |
59+
| `path_to_property` | string | Path to the property that contains the vector. This path can be a top-level property or a dot notation path to the property. If a dot notation path is used, then all the nonleaf elements can't be arrays. Vectors must be a `number[]` to be indexed and return in vector search results.|
60+
| `kind` | string | Type of vector index to create. The options are `vector-ivf` and `vector-hnsw`. Note `vector-ivf` is available on all cluster tiers and `vector-hnsw` is available on M40 cluster tiers and higher. |
5961
|`m` |integer |The max number of connections per layer (`16` by default, minimum value is `2`, maximum value is `100`). Higher m is suitable for datasets with high dimensionality and/or high accuracy requirements. |
6062
|`efConstruction` |integer |the size of the dynamic candidate list for constructing the graph (`64` by default, minimum value is `4`, maximum value is `1000`). Higher `efConstruction` will result in better index quality and higher accuracy, but it will also increase the time required to build the index. `efConstruction` has to be at least `2 * m` |
6163
|`similarity` |string |Similarity metric to use with the index. Possible options are `COS` (cosine distance), `L2` (Euclidean distance), and `IP` (inner product). |
@@ -113,9 +115,9 @@ To create a vector index using the IVF (Inverted File) algorithm, use the follow
113115
| --- | --- | --- |
114116
| `index_name` | string | Unique name of the index. |
115117
| `path_to_property` | string | Path to the property that contains the vector. This path can be a top-level property or a dot notation path to the property. If a dot notation path is used, then all the nonleaf elements can't be arrays. Vectors must be a `number[]` to be indexed and return in vector search results.|
116-
| `kind` | string | Type of vector index to create. IVF, `vector-ivf`, is shown in this example. |
118+
| `kind` | string | Type of vector index to create. The options are `vector-ivf` and `vector-hnsw`. Note `vector-ivf` is available on all cluster tiers and `vector-hnsw` is available on M40 cluster tiers and higher. |
117119
| `numLists` | integer | This integer is the number of clusters that the inverted file (IVF) index uses to group the vector data. We recommend that `numLists` is set to `documentCount/1000` for up to 1 million documents and to `sqrt(documentCount)` for more than 1 million documents. Using a `numLists` value of `1` is akin to performing brute-force search, which has limited performance. |
118-
| `similarity` | string | Similarity metric to use with the IVF index. Possible options are `COS` (cosine distance), `L2` (Euclidean distance), and `IP` (inner product). |
120+
| `similarity` | string | Similarity metric to use with the index. Possible options are `COS` (cosine distance), `L2` (Euclidean distance), and `IP` (inner product). |
119121
| `dimensions` | integer | Number of dimensions for vector similarity. The maximum number of supported dimensions is `2000`. |
120122

121123
> [!IMPORTANT]
@@ -153,7 +155,7 @@ To retrieve the similarity score (`searchScore`) along with the documents found
153155
> [!IMPORTANT]
154156
> Vectors must be a `number[]` to be indexed. Using another type, such as `double[]`, prevents the document from being indexed. Non-indexed documents won't be returned in the result of a vector search.
155157
156-
## Examples
158+
## Example using an HNSW index.
157159

158160
The following examples show you how to index vectors, add documents that have vector properties, perform a vector search, and retrieve the index configuration.
159161

@@ -271,6 +273,153 @@ In this example, `vectorIndex` is returned with all the `cosmosSearch` parameter
271273
]
272274
```
273275

276+
## Example using an IVF Index
277+
278+
The following examples show you how to index vectors, add documents that have vector properties, perform a vector search, and retrieve the index configuration.
279+
280+
### Create a vector index
281+
282+
```javascript
283+
use test;
284+
285+
db.createCollection("exampleCollection");
286+
287+
db.runCommand({
288+
createIndexes: 'exampleCollection',
289+
indexes: [
290+
{
291+
name: 'vectorSearchIndex',
292+
key: {
293+
"vectorContent": "cosmosSearch"
294+
},
295+
cosmosSearchOptions: {
296+
kind: 'vector-ivf',
297+
numLists: 3,
298+
similarity: 'COS',
299+
dimensions: 3
300+
}
301+
}
302+
]
303+
});
304+
```
305+
306+
This command creates a `vector-ivf` index against the `vectorContent` property in the documents that are stored in the specified collection, `exampleCollection`. The `cosmosSearchOptions` property specifies the parameters for the IVF vector index. If your document has the vector stored in a nested property, you can set this property by using a dot notation path. For example, you might use `text.vectorContent` if `vectorContent` is a subproperty of `text`.
307+
308+
### Add vectors to your database
309+
310+
To add vectors to your database's collection, you first need to create the [embeddings](../../../ai-services/openai/concepts/understand-embeddings.md) by using your own model, [Azure OpenAI Embeddings](../../../cognitive-services/openai/tutorials/embeddings.md), or another API (such as [Hugging Face on Azure](https://azure.microsoft.com/solutions/hugging-face-on-azure/)). In this example, new documents are added through sample embeddings:
311+
312+
```javascript
313+
db.exampleCollection.insertMany([
314+
{name: "Eugenia Lopez", bio: "Eugenia is the CEO of AdvenureWorks.", vectorContent: [0.51, 0.12, 0.23]},
315+
{name: "Cameron Baker", bio: "Cameron Baker CFO of AdvenureWorks.", vectorContent: [0.55, 0.89, 0.44]},
316+
{name: "Jessie Irwin", bio: "Jessie Irwin is the former CEO of AdventureWorks and now the director of the Our Planet initiative.", vectorContent: [0.13, 0.92, 0.85]},
317+
{name: "Rory Nguyen", bio: "Rory Nguyen is the founder of AdventureWorks and the president of the Our Planet initiative.", vectorContent: [0.91, 0.76, 0.83]},
318+
]);
319+
```
320+
321+
### Perform a vector search
322+
323+
To perform a vector search, use the `$search` aggregation pipeline stage in a MongoDB query. To use the `cosmosSearch` index, use the new `cosmosSearch` operator.
324+
325+
```json
326+
{
327+
{
328+
"$search": {
329+
"cosmosSearch": {
330+
"vector": <vector_to_search>,
331+
"path": "<path_to_property>",
332+
"k": <num_results_to_return>,
333+
},
334+
"returnStoredSource": True }},
335+
{
336+
"$project": { "<custom_name_for_similarity_score>": {
337+
"$meta": "searchScore" },
338+
"document" : "$$ROOT"
339+
}
340+
}
341+
}
342+
```
343+
To retrieve the similarity score (`searchScore`) along with the documents found by the vector search, use the `$project` operator to include `searchScore` and rename it as `<custom_name_for_similarity_score>` in the results. Then the document is also projected as nested object. Note that the similarity score is calculated using the metric defined in the vector index.
344+
345+
### Query vectors and vector distances (aka similarity scores) using $search"
346+
347+
Continuing with the last example, create another vector, `queryVector`. Vector search measures the distance between `queryVector` and the vectors in the `vectorContent` path of your documents. You can set the number of results that the search returns by setting the parameter `k`, which is set to `2` here. You can also set `nProbes`, which is an integer that controls the number of nearby clusters that are inspected in each search. A higher value may improve accuracy, however the search will be slower as a result. This is an optional parameter with a default value of 1 and cannot be larger than the `numLists` value specified in the vector index.
348+
349+
350+
```javascript
351+
const queryVector = [0.52, 0.28, 0.12];
352+
db.exampleCollection.aggregate([
353+
{
354+
$search: {
355+
"cosmosSearch": {
356+
"vector": queryVector,
357+
"path": "vectorContent",
358+
"k": 2
359+
},
360+
"returnStoredSource": true }},
361+
{
362+
"$project": { "similarityScore": {
363+
"$meta": "searchScore" },
364+
"document" : "$$ROOT"
365+
}
366+
}
367+
]);
368+
```
369+
370+
In this example, a vector search is performed by using `queryVector` as an input via the Mongo shell. The search result is a list of two items that are most similar to the query vector, sorted by their similarity scores.
371+
372+
```javascript
373+
[
374+
{
375+
similarityScore: 0.9465376,
376+
document: {
377+
_id: ObjectId("645acb54413be5502badff94"),
378+
name: 'Eugenia Lopez',
379+
bio: 'Eugenia is the CEO of AdvenureWorks.',
380+
vectorContent: [ 0.51, 0.12, 0.23 ]
381+
}
382+
},
383+
{
384+
similarityScore: 0.9006955,
385+
document: {
386+
_id: ObjectId("645acb54413be5502badff97"),
387+
name: 'Rory Nguyen',
388+
bio: 'Rory Nguyen is the founder of AdventureWorks and the president of the Our Planet initiative.',
389+
vectorContent: [ 0.91, 0.76, 0.83 ]
390+
}
391+
}
392+
]
393+
```
394+
395+
### Get vector index definitions
396+
397+
To retrieve your vector index definition from the collection, use the `listIndexes` command:
398+
399+
``` javascript
400+
db.exampleCollection.getIndexes();
401+
```
402+
403+
In this example, `vectorIndex` is returned with all the `cosmosSearch` parameters that were used to create the index:
404+
405+
```javascript
406+
[
407+
{ v: 2, key: { _id: 1 }, name: '_id_', ns: 'test.exampleCollection' },
408+
{
409+
v: 2,
410+
key: { vectorContent: 'cosmosSearch' },
411+
name: 'vectorSearchIndex',
412+
cosmosSearch: {
413+
kind: 'vector-ivf',
414+
numLists: 3,
415+
similarity: 'COS',
416+
dimensions: 3
417+
},
418+
ns: 'test.exampleCollection'
419+
}
420+
]
421+
```
422+
274423
## Filtered vector search (preview)
275424
You can now execute vector searches with any supported query filter such as `$lt, $lte, $eq, $neq, $gte, $gt, $in, $nin, and $regex`. Enable the "filtering vector search" feature in the "Preview Features" tab of your Azure Subscription. Learn more about preview features [here](../../../azure-resource-manager/management/preview-features.md).
276425

@@ -310,7 +459,7 @@ db.exampleCollection.aggregate([
310459
> [!IMPORTANT]
311460
> While in preview, filtered vector search may require you to adjust your vector index parameters to achieve higher accuracy. For example, increasing `m`, `efConstruction`, or `efSearch` when using HNSW, or `numLists`, or `nProbes` when using IVF, may lead to better results. You should test your configuration before use to ensure that the results are satisfactory.
312461
313-
## Use LLM Orchestration tools such
462+
## Use LLM Orchestration tools
314463

315464
### Use as a vector database with Semantic Kernel
316465
Use Semantic Kernel to orchestrate your information retrieval from Azure Cosmos DB for MongoDB vCore and your LLM. Learn more [here](https://github.com/microsoft/semantic-kernel/tree/main/python/semantic_kernel/connectors/memory/azure_cosmosdb).

0 commit comments

Comments
 (0)