weighted reciprocal rank fusion #111623
Description
Description
The current implementation of retrievers follows closely to the original RRF (Reciprocal Rank Fusion) paper. I wonder if weighting could be introduced such that one result set has a higher/lower weight than the others. eg. 0.75 lexical and 0.25 semantic. This feature can also be found in other products (but not limited to):
- azure ai search: https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking#weighted-scores
- weaviate: https://weaviate.io/developers/weaviate/search/hybrid#balance-keyword-and-vector-search
Suggested API
Current Elasticsearch request:
GET example-index/_search
{
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"term": {
"text": "shoes"
}
}
}
},
{
"knn": {
"field": "vector",
"query_vector": [1.25, 2, 3.5],
"k": 50,
"num_candidates": 100
}
}
],
"window_size": 50,
"rank_constant": 20
}
}
}
With weights (could be named anything else):
GET example-index/_search
{
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"term": {
"text": "shoes"
}
}
},
"weight": 0.25
},
{
"knn": {
"field": "vector",
"query_vector": [1.25, 2, 3.5],
"k": 50,
"num_candidates": 100
},
"weight": 0.75
}
],
"window_size": 50,
"rank_constant": 20
}
}
}