Skip to content

Conversation

@ioanatia
Copy link
Contributor

Tracked in #123389

FUSE supports RRF and linear combination as the two method to assign new relevance scores.
FUSE cannot be pushed down to the data nodes, it needs to see the whole result set in order to assign new scores.

Because of this we need to:

  1. Ensure that FUSE can only be executed on the coordinator: we add the ExecutesOn.Coordinator and PipelineBreaker to FuseScoreEval.
  2. We want to make sure the input of FUSE is always limited - this makes sense in practice, since using RRF or linear combination is used with limited results sets. We don't use RRF or score normalization on results set of millions of docs.
    What this means practically is that the we are looking for another PipelineBreaker exists before FUSE.

@ioanatia ioanatia added >non-issue Team:Search - Relevance The Search organization Search Relevance team v9.2.0 :Search Relevance/ES|QL Search functionality in ES|QL labels Sep 26, 2025
@ioanatia ioanatia marked this pull request as ready for review September 26, 2025 12:39
@elasticsearchmachine elasticsearchmachine added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search - Relevance The Search organization Search Relevance team labels Sep 26, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice. 👍

TIL about PipelineBreaker and ExecutesOn.


ROW _id = ["a", "b", "c"], _score = 0.0, _index = "my_index", my_fork = "foo"
| MV_EXPAND _id
| LIMIT 10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - I understand why this is needed, but probably makes no sense in practice. Maybe we can check that there's a ROW at the top and avoid the need for LIMIT in that case, as input will be already limited?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the usage of ROW is limited in practice, especially in combination with FUSE, but I added a note in #123389 and will definitely look into this as a follow up.

@ioanatia ioanatia merged commit e95971d into elastic:main Sep 26, 2025
34 checks passed
@ioanatia ioanatia deleted the fuse_executes_on_coordinator branch September 26, 2025 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/ES|QL Search functionality in ES|QL Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants