Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ Where:

- `FLAT`: brute force algorithm.
- `HNSW`: hierarchical, navigable, small world algorithm.
- `SVS-VAMANA`: a graph-based nearest neighbor search algorithm.

The `{algorithm}` attribute specifies the algorithm to use when searching `k` most similar vectors in the index or filtering vectors by range.

Expand Down
62 changes: 61 additions & 1 deletion content/develop/ai/search-and-query/vectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To quickly get started, check out the [Redis vector quickstart guide]({{< relref

## Overview

1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}) and [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) vector index types.
1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}), [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) and [`SVS-VAMANA`]({{< relref "develop/ai/search-and-query/vectors#svs-vamana-index" >}}) vector index types.
1. [**Store and update vectors**]({{< relref "develop/ai/search-and-query/vectors#store-and-update-vectors" >}}): Redis stores vectors and metadata in hashes or JSON objects.
1. [**Search with vectors**]({{< relref "develop/ai/search-and-query/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/ai/search-and-query/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/ai/search-and-query/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/ai/search-and-query/vectors#filters" >}}).
1. [**Configure vector queries at runtime**]({{< relref "develop/ai/search-and-query/vectors#runtime-query-params" >}}).
Expand Down Expand Up @@ -129,6 +129,55 @@ FT.CREATE documents

In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `HNSW` vector field named `doc_embedding` with five index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `M`, and `EF_CONSTRUCTION`.

### SVS-VAMANA index

Scalable Vector Search (SVS) is an Intel project in which a new vector search library, VAMANA graph index, was created. SVS-VAMANA supports highly accurate compressed vector indexes. You can read more about the project [here](https://intel.github.io/ScalableVectorSearch/intro.html). Support for `SVS-VAMANA` indexing was added in Redis 8.2.

Choose the `SVS-VAMANA` index type when you need vector search

- on billions of high-dimensional vectors,
- at high accuracy and state-of-the-art speed,
- using less memory than alternatives.

**Required attributes**

| Attribute | Description |
|:-------------------|:-----------------------------------------|
| `TYPE` | Vector type (`FLOAT16` or `FLOAT32`). Note: `COMPRESSION` is supported with both types. |
| `DIM` | The width, or number of dimensions, of the vector embeddings stored in this field. In other words, the number of floating point elements comprising the vector. `DIM` must be a positive integer. The vector used to query this field must have the exact same dimensions as the field itself. |
| `DISTANCE_METRIC` | Distance metric (`L2`, `IP`, `COSINE`). |

**Optional attributes**

`SVS-VAMANA` supports a number of additional parameters to tune the accuracy of the queries.

| Attribute | Description |
|:---------------------------|:-----------------------------------------|
| `COMPRESSION` | Compression algorithm (`LVQ8`, `LVQ8`, `LVQ4`, `LVQ4x4`, `LVQ4x8`, `LeanVec4x8`, or `LeanVec8x8`). Vectors will be compressed during indexing. Note: On non-Intel platforms, `SVS-VAMANA` with `COMPRESSION` will fall back to Intel’s basic quantization implementation. |
| `CONSTRUCTION_WINDOW_SIZE` | The search window size (the default is 200) to use during graph construction. A higher search window size will yield a higher quality graph since more overall vertexes are considered, but will increase construction time. |
| `GRAPH_MAX_DEGREE` | The maximum node degree in the graph (the default is 32). A higher max degree may yield a higher quality graph in terms of recall for performance, but the memory footprint of the graph is directly proportional to the maximum degree. |
| `SEARCH_WINDOW_SIZE` | The size of the search window (the default is 10). Increasing the search window size and capacity generally yields more accurate but slower search results. |
| `EPSILON` | The range search approximation factor (default is 0.01). |
| `TRAINING_THRESHOLD` | The number of vectors after which training is triggered. Applicable only when used with `COMPRESSION`. The default value is `10 * DEFAULT_BLOCK_SIZE` or, if provided, the value is limited to `100 * DEFAULT_BLOCK_SIZE`, where `DEFAULT_BLOCK_SIZE` is 1024. |
| `LEANVEC_DIM` | The dimension used when using `LeanVec4x8` or `LeanVec8x8` compression for dimensionality reduction. The default value is `DIM / 2`. If provided, the value should be less than `DIM`. |

**Example**

```
FT.CREATE documents
ON HASH
PREFIX 1 docs:
SCHEMA doc_embedding VECTOR SVS-VAMANA 12
TYPE FLOAT32
DIM 1536
DISTANCE_METRIC COSINE
GRAPH_MAX_DEGREE 40
CONSTRUCTION_WINDOW_SIZE 250
COMPRESSION LVQ8
```

In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `SVS-VAMANA` vector field named `doc_embedding` with six index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `GRAPH_MAX_DEGREE`, `CONSTRUCTION_WINDOW_SIZE` and `COMPRESSION`.

### Distance metrics

Redis supports three popular distance metrics to measure the degree of similarity between two vectors $u$, $v$ $\in \mathbb{R}^n$, where $n$ is the length of the vectors:
Expand Down Expand Up @@ -378,6 +427,17 @@ Optional runtime parameters for HNSW indexes are:
| `EF_RUNTIME` | The maximum number of top candidates to hold during the KNN search. Higher values lead to more accurate results at the expense of a longer query runtime. | The value passed during index creation. The default is 10. |
| `EPSILON` | The relative factor that sets the boundaries for a vector range query. Vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing a more extensive search and more accurate results at the expense of runtime. | The value passed during index creation. The default is 0.01. |

**SVS-VAMANA**

Optional runtime parameters for SVS-VAMANA indexes are:

| Parameter | Description | Default value |
|:----------|:------------|:--------------|
| `SEARCH_WINDOW_SIZE` | The size of the search window (applies only to KNN searches). | 10 or the value that was passed upon index creation. |
| `GRAPH_MAX_DEGREE` | The maximum node degree in the graph. | 32 or the value that was passed upon index creation. |
| `SEARCH_WINDOW_SIZE` | The size of the search window. | 10 or the value that was passed upon index creation. |
| `EPSILON` | The range search approximation factor. | 0.01 or the value that was passed upon index creation. |


### Important notes

Expand Down
4 changes: 2 additions & 2 deletions static/resources/redisuniversity.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,6 @@ links:
text: See the [Get started with Redis Software learning path](https://university.redis.io/learningpath/an0mgw5bjpjfbe) for courses.

# Redis Develop links
- page: /develop/interact/search-and-query/advanced-concepts/vectors
- page: /develop/ai/search-and-query/vectors
text: See the [Introduction to vector search course](https://university.redis.io/course/yz1lretjfpdlew?tab=details) to learn more.