DOC-5800: search: document new vector search attrib.

Author: dwdougherty

content/commands/ft.aggregate.md

@@ -331,6 +331,8 @@ if set, overrides the timeout parameter of the module.
 defines one or more value parameters. Each parameter has a name and a value. 
 
 You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set `DIALECT` to `2` or greater than `2`.
+
+**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
 </details>
 
 <details open>

content/commands/ft.search.md

@@ -510,6 +510,8 @@ defines one or more value parameters. Each parameter has a name and a value.
 You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set
 [`DIALECT`]({{< relref "/develop/ai/search-and-query/advanced-concepts/dialects#dialect-2" >}})
 to `2` or greater than `2` (this requires [RediSearch v2.4](https://github.com/RediSearch/RediSearch/releases/tag/v2.4.3) or above).
+
+**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
 </details>
 
 <details open>
@@ -679,6 +681,16 @@ Search for books with semantically similar title to _Planet Earth_. Return top 1
 {{< / highlight >}}
 </details>
 
+<details open>
+<summary><b>Vector search with cluster optimization</b></summary>
+
+Search for books with semantically similar title to _Planet Earth_ using cluster optimization. Each shard retrieves 60% of the requested results for improved performance in Redis cluster environments.
+
+{{< highlight bash >}}
+127.0.0.1:6379> FT.SEARCH books-idx "*=>[KNN 100 @title_embedding $query_vec]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: title_score}" PARAMS 2 query_vec <"Planet Earth" embedding BLOB> SORTBY title_score DIALECT 2
+{{< / highlight >}}
+</details>
+
 <details open>
 <summary><b>Search for a phrase using SLOP</b></summary>
 

content/develop/ai/search-and-query/advanced-concepts/query_syntax.md

@@ -389,7 +389,8 @@ The supported attributes are:
 
 As of v2.6.1, the query attributes syntax supports these additional attributes:
 
-* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).   
+* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).
+* **$shard_k_ratio**: controls how many results each shard retrieves relative to the requested top_k in cluster setups. Value range: 0.1 - 1.0 (default: 1.0). Only applicable to vector KNN queries in Redis cluster environments. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for detailed information.
 * **vector query params**: pass optional parameters for [vector queries]({{< relref "develop/ai/search-and-query/vectors#querying-vector-fields" >}}) in key-value format.
 
 ## A few query examples
@@ -458,6 +459,10 @@ As of v2.6.1, the query attributes syntax supports these additional attributes:
 
         @age:[(18 +inf]
 
+* Vector search with cluster optimization - retrieve 100 nearest neighbors with each shard providing 50% of results:
+
+        *=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}
+
 ## Mapping common SQL predicates to Redis Query Engine
 
 | SQL Condition | Redis Query Engine Equivalent | Comments |

content/develop/ai/search-and-query/query/vector-search.md

@@ -106,4 +106,28 @@ query = (
      .return_fields('vector_dist', 'description')
      .dialect(2)
 )
-</!-->
+</!-->
+
+## Cluster optimization
+
+In Redis cluster environments, you can optimize vector search performance using the `$SHARD_K_RATIO` query attribute. This parameter controls how many results each shard retrieves relative to the requested `top_k`, creating a tunable trade-off between accuracy and performance.
+
+### Basic cluster optimization
+
+Retrieve 100 nearest neighbors with each shard providing 60% of the requested results:
+
+{{< clients-example query_vector vector3 >}}
+FT.SEARCH idx:bikes_vss "(*)=>[KNN 100 @vector $query_vector]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY vector_distance ASC RETURN 2 "vector_distance" "description" DIALECT 2
+{{< /clients-example >}}
+
+### Combined with filtering
+
+You can combine `$SHARD_K_RATIO` with pre-filtering to optimize searches on specific subsets of data:
+
+{{< clients-example query_vector vector4 >}}
+FT.SEARCH idx:bikes_vss "(@brand:trek)=>[KNN 50 @vector $query_vector]=>{$SHARD_K_RATIO: 0.4; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY similarity ASC RETURN 2 "similarity" "description" DIALECT 2
+{{< /clients-example >}}
+
+{{% alert title="Note" color="warning" %}}
+The `$SHARD_K_RATIO` parameter is only applicable in Redis cluster environments and has no effect in standalone Redis instances.
+{{% /alert  %}}

content/develop/ai/search-and-query/vectors/_index.md

@@ -414,6 +414,24 @@ By default, Redis selects the best filter mode to optimize query execution. You
 | `HYBRID_POLICY`  | Specifies the filter mode to use during vector search with filters (hybrid). | `BATCHES` or `ADHOC_BF` |
 | `BATCH_SIZE`     | A fixed batch size to use in every iteration when the `BATCHES` policy is auto-selected or requested. | Positive integer. |
 
+### Cluster-specific query parameters
+
+**$SHARD_K_RATIO**
+
+The `$SHARD_K_RATIO` parameter controls how many results each shard retrieves relative to the requested `top_k` in Redis cluster environments. This creates a tunable trade-off between accuracy and performance.
+
+| Parameter        | Description | Value Range | Default |
+|:-----------------|:------------|:------------|:--------|
+| `$SHARD_K_RATIO` | Ratio of `top_k` results to fetch per shard in cluster setups. Calculation: `shard_results = top_k × ratio`. | 0.1 - 1.0 (up to 2 decimal places) | 1.0 |
+
+**Important considerations:**
+
+- Only applicable to vector KNN queries in Redis cluster environments
+- Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results
+- If the user-defined ratio is lower than `top_k / #shards`, the server ignores it and uses the minimum required ratio
+- Unbalanced shards returning fewer results than required may lead to fewer total results than requested in `top_k`
+- Invalid ratios return descriptive error messages
+
 
 ### Index-specific query parameters
 
@@ -523,6 +541,63 @@ FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB]=>{$HYBRID_
 
 To explore additional Python vector search examples, review recipes for the [`Redis Python`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) client library and the [`Redis Vector Library`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb).
 
+### Cluster optimization examples
+
+The following examples demonstrate using `$SHARD_K_RATIO` to optimize vector search performance in Redis cluster environments.
+
+Return the top 100 nearest neighbors with each shard providing 50% of the requested results (50 results per shard):
+
+```
+FT.SEARCH documents "*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 2
+```
+
+Combine cluster optimization with filtering and other query attributes. Among movies with `action` category, return top 50 nearest neighbors with 30% shard ratio and custom EF_RUNTIME:
+
+```
+FT.SEARCH movies "(@category:{action})=>[KNN 50 @movie_embedding $BLOB]=>{$SHARD_K_RATIO: 0.3; $EF_RUNTIME: 150; $YIELD_DISTANCE_AS: movie_distance}" PARAMS 4 BLOB "\x12\xa9\xf5\x6c" EF 150 SORTBY movie_distance DIALECT 2
+```
+
+Use a higher ratio for better accuracy when precision is more important than performance:
+
+```
+FT.SEARCH products "*=>[KNN 20 @product_embedding $BLOB]=>{$SHARD_K_RATIO: 0.8; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY similarity DIALECT 2
+```
+
+### Cluster considerations and best practices
+
+When using `$SHARD_K_RATIO` in Redis cluster environments, consider the following behavioral characteristics and best practices:
+
+#### Minimum results guarantee
+
+Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results to ensure the cluster can return the requested `top_k` results. If your specified ratio would result in fewer results per shard than this minimum, Redis automatically adjusts to the minimum required ratio.
+
+**Example scenarios:**
+- `top_k=100`, 4 shards, `ratio=0.5` → 50 results per shard (valid)
+- `top_k=100`, 4 shards, `ratio=0.2` → 25 results per shard (minimum: 25, so valid)
+- `top_k=100`, 8 shards, `ratio=0.1` → 10 results per shard (minimum: 13, so Redis uses 13)
+
+#### Handling unbalanced shards
+
+In cases where some shards contain fewer documents than required to fulfill the `top_k` results, using a lower ratio may lead to fewer total results than requested. To mitigate this:
+
+1. **Monitor shard distribution**: Ensure your data is relatively balanced across shards
+2. **Adjust ratio dynamically**: If you consistently get fewer results than expected, increase the ratio
+3. **Use higher ratios for critical queries**: When result completeness is more important than performance
+
+#### Performance vs accuracy trade-offs
+
+| Ratio Range | Use Case | Performance | Accuracy |
+|:------------|:---------|:------------|:---------|
+| 0.1 - 0.3   | High-performance, approximate results | Excellent | Good |
+| 0.4 - 0.6   | Balanced performance and accuracy | Good | Very Good |
+| 0.7 - 1.0   | High-accuracy, precision-critical | Fair | Excellent |
+
+#### Error handling
+
+Redis returns descriptive error messages for invalid `$SHARD_K_RATIO` values:
+- Values outside the 0.1 - 1.0 range
+- Values with more than 2 decimal places
+- Non-numeric values
 
 ### Range query examples
 

shard.txt

@@ -0,0 +1,82 @@
+3.1 Core Functionality
+The feature introduces a shard window ratio  - suggested $shard_k_ratio - parameter that controls how many results each shard retrieves relative to the requested top_k. This creates a tunable trade-off between accuracy and performance.
+
+Calculation: shard_results = top_k × window_ratio
+
+3.2 API Integration
+Following Redis vector search patterns, the new parameter will be implemented as a Query Attribute using the existing =>{...} syntax.
+
+API Syntax
+
+
+FT.SEARCH <index_name> 
+"<filter>=>[KNN <top_k> @<vector_field> $<vector_param>]=>{$SHARD_K_RATIO: <ratio_value>; $YIELD_DISTANCE_AS: <distance_field>}"
+PARAMS <param_count> <vector_param> <vector_blob> [additional_params...]
+SORTBY <distance_field>
+DIALECT 2
+Example Usage
+
+
+# Basic usage: Retrieve 50 results per shard instead of 100
+FT.SEARCH documents 
+"*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}"
+PARAMS 2 BLOB "\x12\xa9\xf5\x6c"
+SORTBY vector_distance
+DIALECT 2
+# Combined with other query attributes
+FT.SEARCH documents 
+"*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.3; $EF_RUNTIME: 150; $YIELD_DISTANCE_AS: vector_distance}"
+PARAMS 4 BLOB "\x12\xa9\xf5\x6c" EF 150
+SORTBY vector_distance
+DIALECT 2
+# With filtering
+FT.SEARCH movies 
+"(@category:{action})=>[KNN 50 @movie_embedding $BLOB]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: movie_distance}"
+PARAMS 2 BLOB "\x12\xa9\xf5\x6c"
+SORTBY movie_distance
+DIALECT 2
+3.3 Parameter Specifications
+Attribute
+
+Value Range
+
+Default
+
+Behavior
+
+Attribute
+
+Value Range
+
+Default
+
+Behavior
+
+$SHARD_RATIO
+
+0.1 - 1.0
+
+1.0
+
+Ratio of top_k results to fetch per shard
+
+Validation Rules
+Range: Must be between higher than 0 and 1.0 (inclusive)
+
+Precision: Support up to 2 decimal places (0.01 minimum)
+
+Minimum results: Each shard must retrieve at least max(top_k/#shards, ceil(top_k × ratio)) results, not allowing to return less results than the requested K (NEED DOCUMENTATION)
+
+Error handling: Invalid ratios return descriptive error messages
+
+Behavioral Examples
+top_k=100, ratio=0.5 → 50 results per shard
+
+top_k=10, ratio=0.3 → 3 results per shard
+
+top_k=7, ratio=0.2 → 2 results per shard (ceiling applied)
+
+Documentation required for the following limitations:
+Minimum K results from the shards will always be top_k/shardsif the user-defined ratio is lower than that, the server will ignore it
+
+In case some unbalanced shard returns less than the amount required for fulfilling the top_k results, when using ratio, it could lead to fewer results than requested in the top_k. If that’s the case, it requires adjusting the ratio to a higher value