Skip to content

Conversation

@iverase
Copy link
Contributor

@iverase iverase commented Jul 21, 2025

This commit implements the AVX-512 variant in MemorySegmentES91OSQVectorsScorer which seems to help bulk scoring. The current microbenchmarks looks like:

Benchmark                                                (dims)   Mode  Cnt   Score   Error   Units
OSQScorerBenchmark.scoreFromArray                          1024  thrpt    5   8.336 ± 0.992  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentAllBulk           1024  thrpt    5  15.222 ± 0.351  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentOnlyVector        1024  thrpt    5   8.724 ± 0.092  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentOnlyVectorBulk    1024  thrpt    5   9.444 ± 1.166  ops/ms

With this patch:

Benchmark                                                (dims)   Mode  Cnt   Score   Error   Units
OSQScorerBenchmark.scoreFromArray                          1024  thrpt    5   8.615 ± 0.330  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentAllBulk           1024  thrpt    5  18.649 ± 1.120  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentOnlyVector        1024  thrpt    5   8.633 ± 0.099  ops/ms
OSQScorerBenchmark.scoreFromMemorySegmentOnlyVectorBulk    1024  thrpt    5   9.909 ± 0.097  ops/ms

I have reorganised a bit the code so it is consistent for every method.

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 21, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Just to state out loud I think we have randomized tests that validate that this produces the same / correct result as before.

iverase added 2 commits July 31, 2025 10:46
# Conflicts:
#	libs/simdvec/src/main21/java/org/elasticsearch/simdvec/internal/vectorization/MemorySegmentES91OSQVectorsScorer.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants