Skip to content

Conversation

@iverase
Copy link
Contributor

@iverase iverase commented Aug 19, 2025

In #132722, we change the condition to stop the iteration of posting lists tot ake into account the number of documents visited instead of the number of posting lists. The condition was changed to take into account the number of scored docs.

This had a side effect because now in the case of a search with a filter we are visiting much more posting lists because we don't stop until we visit a non-filtered number of documents where before this was not taking into account. Ic hecked and I think the current check is wrong as we are visiting too many documents and the condition should be using expectedDocs so we bail out when we reach that number and we collected enough documents.

running a filtered search in main:

index_name       index_type  visit_percentage(%)  latency(ms)  net_cpu_time(ms)  avg_cpu_count     QPS  recall    visited  filter_selectivity
---------------  ----------  -------------------  -----------  ----------------  -------------  ------  ------  ---------  ------------------  
wiki1024en.docs         ivf                 0.00        12.99              0.00           0.00   76.98    0.99  180720.32                0.50

With this PR:

index_name       index_type  visit_percentage(%)  latency(ms)  net_cpu_time(ms)  avg_cpu_count      QPS  recall   visited  filter_selectivity
---------------  ----------  -------------------  -----------  ----------------  -------------  -------  ------  --------  ------------------  
wiki1024en.docs         ivf                 0.00         6.79              0.00           0.00   147.28    0.98  90911.02                0.50

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Aug 19, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense; lgtm

@iverase iverase merged commit 73334ad into elastic:main Aug 19, 2025
34 checks passed
@iverase iverase deleted the expectedDocs branch August 19, 2025 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants