Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions content/commands/ft.aggregate.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,8 @@ if set, overrides the timeout parameter of the module.
defines one or more value parameters. Each parameter has a name and a value.

You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set `DIALECT` to `2` or greater than `2`.

**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
</details>

<details open>
Expand Down
12 changes: 12 additions & 0 deletions content/commands/ft.search.md
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,8 @@ defines one or more value parameters. Each parameter has a name and a value.
You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set
[`DIALECT`]({{< relref "/develop/ai/search-and-query/advanced-concepts/dialects#dialect-2" >}})
to `2` or greater than `2` (this requires [RediSearch v2.4](https://github.com/RediSearch/RediSearch/releases/tag/v2.4.3) or above).

**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
</details>

<details open>
Expand Down Expand Up @@ -679,6 +681,16 @@ Search for books with semantically similar title to _Planet Earth_. Return top 1
{{< / highlight >}}
</details>

<details open>
<summary><b>Vector search with cluster optimization</b></summary>

Search for books with semantically similar title to _Planet Earth_ using cluster optimization. Each shard retrieves 60% of the requested results for improved performance in Redis cluster environments.

{{< highlight bash >}}
127.0.0.1:6379> FT.SEARCH books-idx "*=>[KNN 100 @title_embedding $query_vec]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: title_score}" PARAMS 2 query_vec <"Planet Earth" embedding BLOB> SORTBY title_score DIALECT 2
{{< / highlight >}}
</details>

<details open>
<summary><b>Search for a phrase using SLOP</b></summary>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,8 @@ The supported attributes are:

As of v2.6.1, the query attributes syntax supports these additional attributes:

* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).
* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).
* **$shard_k_ratio**: controls how many results each shard retrieves relative to the requested top_k in cluster setups. Value range: 0.1 - 1.0 (default: 1.0). Only applicable to vector KNN queries in Redis cluster environments. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for detailed information.
* **vector query params**: pass optional parameters for [vector queries]({{< relref "develop/ai/search-and-query/vectors#querying-vector-fields" >}}) in key-value format.

## A few query examples
Expand Down Expand Up @@ -458,6 +459,10 @@ As of v2.6.1, the query attributes syntax supports these additional attributes:

@age:[(18 +inf]

* Vector search with cluster optimization - retrieve 100 nearest neighbors with each shard providing 50% of results:

*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}

## Mapping common SQL predicates to Redis Query Engine

| SQL Condition | Redis Query Engine Equivalent | Comments |
Expand Down
26 changes: 25 additions & 1 deletion content/develop/ai/search-and-query/query/vector-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,28 @@ query = (
.return_fields('vector_dist', 'description')
.dialect(2)
)
</!-->
</!-->

## Cluster optimization

In Redis cluster environments, you can optimize vector search performance using the `$SHARD_K_RATIO` query attribute. This parameter controls how many results each shard retrieves relative to the requested `top_k`, creating a tunable trade-off between accuracy and performance.

### Basic cluster optimization

Retrieve 100 nearest neighbors with each shard providing 60% of the requested results:

{{< clients-example query_vector vector3 >}}
FT.SEARCH idx:bikes_vss "(*)=>[KNN 100 @vector $query_vector]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY vector_distance ASC RETURN 2 "vector_distance" "description" DIALECT 2
{{< /clients-example >}}

### Combined with filtering

You can combine `$SHARD_K_RATIO` with pre-filtering to optimize searches on specific subsets of data:

{{< clients-example query_vector vector4 >}}
FT.SEARCH idx:bikes_vss "(@brand:trek)=>[KNN 50 @vector $query_vector]=>{$SHARD_K_RATIO: 0.4; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY similarity ASC RETURN 2 "similarity" "description" DIALECT 2
{{< /clients-example >}}

{{% alert title="Note" color="warning" %}}
The `$SHARD_K_RATIO` parameter is only applicable in Redis cluster environments and has no effect in standalone Redis instances.
{{% /alert %}}
75 changes: 75 additions & 0 deletions content/develop/ai/search-and-query/vectors/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,24 @@ By default, Redis selects the best filter mode to optimize query execution. You
| `HYBRID_POLICY` | Specifies the filter mode to use during vector search with filters (hybrid). | `BATCHES` or `ADHOC_BF` |
| `BATCH_SIZE` | A fixed batch size to use in every iteration when the `BATCHES` policy is auto-selected or requested. | Positive integer. |

### Cluster-specific query parameters

**$SHARD_K_RATIO**

The `$SHARD_K_RATIO` parameter controls how many results each shard retrieves relative to the requested `top_k` in Redis cluster environments. This creates a tunable trade-off between accuracy and performance.

| Parameter | Description | Value Range | Default |
|:-----------------|:------------|:------------|:--------|
| `$SHARD_K_RATIO` | Ratio of `top_k` results to fetch per shard in cluster setups. Calculation: `shard_results = top_k × ratio`. | 0.1 - 1.0 (up to 2 decimal places) | 1.0 |

**Important considerations:**

- Only applicable to vector KNN queries in Redis cluster environments
- Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results
- If the user-defined ratio is lower than `top_k / #shards`, the server ignores it and uses the minimum required ratio
- Unbalanced shards returning fewer results than required may lead to fewer total results than requested in `top_k`
- Invalid ratios return descriptive error messages


### Index-specific query parameters

Expand Down Expand Up @@ -523,6 +541,63 @@ FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB]=>{$HYBRID_

To explore additional Python vector search examples, review recipes for the [`Redis Python`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) client library and the [`Redis Vector Library`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb).

### Cluster optimization examples

The following examples demonstrate using `$SHARD_K_RATIO` to optimize vector search performance in Redis cluster environments.

Return the top 100 nearest neighbors with each shard providing 50% of the requested results (50 results per shard):

```
FT.SEARCH documents "*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 2
```

Combine cluster optimization with filtering and other query attributes. Among movies with `action` category, return top 50 nearest neighbors with 30% shard ratio and custom EF_RUNTIME:

```
FT.SEARCH movies "(@category:{action})=>[KNN 50 @movie_embedding $BLOB]=>{$SHARD_K_RATIO: 0.3; $EF_RUNTIME: 150; $YIELD_DISTANCE_AS: movie_distance}" PARAMS 4 BLOB "\x12\xa9\xf5\x6c" EF 150 SORTBY movie_distance DIALECT 2
```

Use a higher ratio for better accuracy when precision is more important than performance:

```
FT.SEARCH products "*=>[KNN 20 @product_embedding $BLOB]=>{$SHARD_K_RATIO: 0.8; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY similarity DIALECT 2
```

### Cluster considerations and best practices

When using `$SHARD_K_RATIO` in Redis cluster environments, consider the following behavioral characteristics and best practices:

#### Minimum results guarantee

Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results to ensure the cluster can return the requested `top_k` results. If your specified ratio would result in fewer results per shard than this minimum, Redis automatically adjusts to the minimum required ratio.

**Example scenarios:**
- `top_k=100`, 4 shards, `ratio=0.5` → 50 results per shard (valid)
- `top_k=100`, 4 shards, `ratio=0.2` → 25 results per shard (minimum: 25, so valid)
- `top_k=100`, 8 shards, `ratio=0.1` → 10 results per shard (minimum: 13, so Redis uses 13)

#### Handling unbalanced shards

In cases where some shards contain fewer documents than required to fulfill the `top_k` results, using a lower ratio may lead to fewer total results than requested. To mitigate this:

1. **Monitor shard distribution**: Ensure your data is relatively balanced across shards
2. **Adjust ratio dynamically**: If you consistently get fewer results than expected, increase the ratio
3. **Use higher ratios for critical queries**: When result completeness is more important than performance

#### Performance vs accuracy trade-offs

| Ratio Range | Use Case | Performance | Accuracy |
|:------------|:---------|:------------|:---------|
| 0.1 - 0.3 | High-performance, approximate results | Excellent | Good |
| 0.4 - 0.6 | Balanced performance and accuracy | Good | Very Good |
| 0.7 - 1.0 | High-accuracy, precision-critical | Fair | Excellent |

#### Error handling

Redis returns descriptive error messages for invalid `$SHARD_K_RATIO` values:
- Values outside the 0.1 - 1.0 range
- Values with more than 2 decimal places
- Non-numeric values

### Range query examples

Expand Down