Skip to content

Commit 3ab5519

Browse files
Docs: Add weight parameter documentation for Weighted RRF retriever (#136698) (#137380)
* Added weighted rrf examples to the doc * Cleaned up doc * Modified information * Fixed discrepancies * made it more explicit * Update docs/reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md * Resolved PR comments * Resolved comments * Resolved comments * Update docs/reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md * addressed comments' * changes * Update rrf-retriever.md --------- Co-authored-by: Kathleen DeRusso <[email protected]>
1 parent 325398b commit 3ab5519

File tree

2 files changed

+204
-3
lines changed

2 files changed

+204
-3
lines changed

docs/reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md

Lines changed: 51 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,9 @@ First, let’s examine how to combine two different types of queries: a `kNN` qu
113113
While these queries may produce scores in different ranges, we can use Reciprocal Rank Fusion (`rrf`) to combine the results and generate a merged final result list.
114114

115115
To implement this in the retriever framework, we start with the top-level element: our `rrf` retriever.
116-
This retriever operates on top of two other retrievers: a `knn` retriever and a `standard` retriever. Our query structure would look like this:
116+
This retriever operates on top of two other retrievers: a `knn` retriever and a `standard` retriever.
117+
We can specify weights to adjust the influence of each retriever on the final ranking.
118+
In this example, we're giving the `standard` retriever twice the influence of the `knn` retriever:
117119

118120
```console
119121
GET /retrievers_example/_search
@@ -195,6 +197,54 @@ This returns the following response based on the final rrf score for each result
195197
::::
196198

197199

200+
### Using the expanded format with weights {applies_to}`stack: ga 9.2`
201+
202+
The same query can be written using the expanded format, which allows you to specify custom weights to adjust the influence of each retriever on the final ranking.
203+
In this example, we're giving the `standard` retriever twice the influence of the `knn` retriever:
204+
205+
```console
206+
GET /retrievers_example/_search
207+
{
208+
"retriever": {
209+
"rrf": {
210+
"retrievers": [
211+
{
212+
"retriever": {
213+
"standard": {
214+
"query": {
215+
"query_string": {
216+
"query": "(information retrieval) OR (artificial intelligence)",
217+
"default_field": "text"
218+
}
219+
}
220+
}
221+
},
222+
"weight": 2.0
223+
},
224+
{
225+
"retriever": {
226+
"knn": {
227+
"field": "vector",
228+
"query_vector": [
229+
0.23,
230+
0.67,
231+
0.89
232+
],
233+
"k": 3,
234+
"num_candidates": 5
235+
}
236+
},
237+
"weight": 1.0
238+
}
239+
],
240+
"rank_window_size": 10,
241+
"rank_constant": 1
242+
}
243+
},
244+
"_source": false
245+
}
246+
```
247+
198248

199249
## Example: Hybrid search with linear retriever [retrievers-examples-linear-retriever]
200250

docs/reference/elasticsearch/rest-apis/retrievers/rrf-retriever.md

Lines changed: 153 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ applies_to:
66

77
# RRF retriever [rrf-retriever]
88

9-
An [RRF](/reference/elasticsearch/rest-apis/reciprocal-rank-fusion.md) retriever returns top documents based on the RRF formula, equally weighting two or more child retrievers.
9+
An [RRF](/reference/elasticsearch/rest-apis/reciprocal-rank-fusion.md) retriever returns top documents based on the RRF formula, combining two or more child retrievers.
1010
Reciprocal rank fusion (RRF) is a method for combining multiple result sets with different relevance indicators into a single result set.
1111

1212

@@ -32,7 +32,13 @@ Combining `query` and `retrievers` is not supported.
3232
: (Optional, array of retriever objects)
3333

3434
A list of child retrievers to specify which sets of returned top documents will have the RRF formula applied to them.
35-
Each child retriever carries an equal weight as part of the RRF formula. Two or more child retrievers are required.
35+
Each retriever can optionally include a weight to adjust its influence on the final ranking. {applies_to}`stack: ga 9.2`
36+
37+
When weights are specified, the final RRF score is calculated as:
38+
```
39+
rrf_score = weight_1 × rrf_score_1 + weight_2 × rrf_score_2 + ... + weight_n × rrf_score_n
40+
```
41+
where `rrf_score_i` is the RRF score for document from retriever `i`, and `weight_i` is the weight for that retriever.
3642

3743
`rank_constant`
3844
: (Optional, integer)
@@ -53,6 +59,82 @@ Combining `query` and `retrievers` is not supported.
5359

5460
Applies the specified [boolean query filter](/reference/query-languages/query-dsl/query-dsl-bool-query.md) to all of the specified sub-retrievers, according to each retriever’s specifications.
5561

62+
Each entry in the `retrievers` array can be specified using the direct format or the wrapped format. {applies_to}`stack: ga 9.2`
63+
64+
**Direct format** (default weight of `1.0`):
65+
```json
66+
{
67+
"rrf": {
68+
"retrievers": [
69+
{
70+
"standard": {
71+
"query": {
72+
"multi_match": {
73+
"query": "search text",
74+
"fields": ["field1", "field2"]
75+
}
76+
}
77+
}
78+
},
79+
{
80+
"knn": {
81+
"field": "vector",
82+
"query_vector": [1, 2, 3],
83+
"k": 10,
84+
"num_candidates": 50
85+
}
86+
}
87+
]
88+
}
89+
}
90+
```
91+
92+
**Wrapped format with custom weights** {applies_to}`stack: ga 9.2`:
93+
```json
94+
{
95+
"rrf": {
96+
"retrievers": [
97+
{
98+
"retriever": {
99+
"standard": {
100+
"query": {
101+
"multi_match": {
102+
"query": "search text",
103+
"fields": ["field1", "field2"]
104+
}
105+
}
106+
}
107+
},
108+
"weight": 2.0
109+
},
110+
{
111+
"retriever": {
112+
"knn": {
113+
"field": "vector",
114+
"query_vector": [1, 2, 3],
115+
"k": 10,
116+
"num_candidates": 50
117+
}
118+
},
119+
"weight": 1.0
120+
}
121+
]
122+
}
123+
}
124+
```
125+
126+
In the wrapped format:
127+
128+
`retriever`
129+
: (Required, a retriever object)
130+
131+
Specifies a child retriever. Any valid retriever type can be used (e.g., `standard`, `knn`, `text_similarity_reranker`, etc.).
132+
133+
`weight` {applies_to}`stack: ga 9.2`
134+
: (Optional, float)
135+
136+
The weight that each score of this retriever's top docs will be multiplied in the RRF formula. Higher values increase this retriever's influence on the final ranking. Must be non-negative. Defaults to `1.0`.
137+
56138
## Example: Hybrid search [rrf-retriever-example-hybrid]
57139

58140
A simple hybrid search example (lexical search + dense vector search) combining a `standard` retriever with a `knn` retriever using RRF:
@@ -99,6 +181,75 @@ GET /restaurants/_search
99181
5. The rank constant for the RRF retriever.
100182
6. The rank window size for the RRF retriever.
101183

184+
## Example: Weighted hybrid search [rrf-retriever-example-weighted]
185+
186+
{applies_to}`stack: ga 9.2`
187+
188+
This example demonstrates how to use weights to adjust the influence of different retrievers in the RRF ranking.
189+
In this case, we're giving the `standard` retriever more importance (weight 2.0) compared to the `knn` retriever (weight 1.0):
190+
191+
```console
192+
GET /restaurants/_search
193+
{
194+
"retriever": {
195+
"rrf": {
196+
"retrievers": [
197+
{
198+
"retriever": { <1>
199+
"standard": {
200+
"query": {
201+
"multi_match": {
202+
"query": "Austria",
203+
"fields": ["city", "region"]
204+
}
205+
}
206+
}
207+
},
208+
"weight": 2.0 <2>
209+
},
210+
{
211+
"retriever": { <3>
212+
"knn": {
213+
"field": "vector",
214+
"query_vector": [10, 22, 77],
215+
"k": 10,
216+
"num_candidates": 10
217+
}
218+
},
219+
"weight": 1.0 <4>
220+
}
221+
],
222+
"rank_constant": 60,
223+
"rank_window_size": 50
224+
}
225+
}
226+
}
227+
```
228+
% TEST[continued]
229+
230+
1. The first retriever in weighted format.
231+
2. This retriever has a weight of 2.0, giving it twice the influence of the kNN retriever.
232+
3. The second retriever in weighted format.
233+
4. This retriever has a weight of 1.0 (default weight).
234+
235+
::::{note}
236+
You can mix weighted and non-weighted formats in the same query.
237+
The direct format (without explicit `retriever` wrapper) uses the default weight of `1.0`:
238+
239+
```json
240+
{
241+
"rrf": {
242+
"retrievers": [
243+
{ "standard": { "query": {...} } },
244+
{ "retriever": { "knn": {...} }, "weight": 2.0 }
245+
]
246+
}
247+
}
248+
```
249+
250+
In this example, the `standard` retriever uses weight `1.0` (default), while the `knn` retriever uses weight `2.0`.
251+
::::
252+
102253
## Example: Hybrid search with sparse vectors [rrf-retriever-example-hybrid-sparse]
103254

104255
A more complex hybrid search example (lexical search + ELSER sparse vector search + dense vector search) using RRF:

0 commit comments

Comments
 (0)