Skip to content

Conversation

@benwtrent
Copy link
Member

We now can do multiple query time parameters, but this sort of broke running multiple indexing loads.

This corrects multiple indexing & search tests for the KnnIndexTester:

Example output.

index_name                           index_type  num_docs  index_time(ms)  force_merge_time(ms)  num_segments
-----------------------------------  ----------  --------  --------------  --------------------  ------------
corpus-quora-E5-small.fvec.flat             ivf    100000            4386                  4522             0
cohere-wikipedia-docs-768d.vec              ivf    100000            7784                  8331             0
corpus-dbpedia-entity-arctic-0.fvec         ivf    100000            7513                  8969             0

index_name                           index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count      QPS  recall   visited
-----------------------------------  ----------  -------  -----------  ----------------  -------------  -------  ------  --------
corpus-quora-E5-small.fvec.flat             ivf       10         0.87              0.00           0.00  1149.43    0.88   5511.60
corpus-quora-E5-small.fvec.flat             ivf       20         0.73              0.00           0.00  1369.86    0.94  10101.70
corpus-quora-E5-small.fvec.flat             ivf       30         0.76              0.00           0.00  1315.79    0.95  14382.15
corpus-quora-E5-small.fvec.flat             ivf       40         0.84              0.00           0.00  1190.48    0.96  18554.68
corpus-quora-E5-small.fvec.flat             ivf       50         0.99              0.00           0.00  1010.10    0.97  22609.69
cohere-wikipedia-docs-768d.vec              ivf       10         0.39              0.00           0.00  2564.10    0.61   2567.00
cohere-wikipedia-docs-768d.vec              ivf       20         0.53              0.00           0.00  1886.79    0.78   5017.25
cohere-wikipedia-docs-768d.vec              ivf       30         0.66              0.00           0.00  1515.15    0.84   7486.77
cohere-wikipedia-docs-768d.vec              ivf       40         0.80              0.00           0.00  1250.00    0.88  10177.80
cohere-wikipedia-docs-768d.vec              ivf       50         0.95              0.00           0.00  1052.63    0.90  12744.67
corpus-dbpedia-entity-arctic-0.fvec         ivf       10         0.48              0.00           0.00  2083.33    0.62   3302.07
corpus-dbpedia-entity-arctic-0.fvec         ivf       20         0.66              0.00           0.00  1515.15    0.74   6333.91
corpus-dbpedia-entity-arctic-0.fvec         ivf       30         0.78              0.00           0.00  1282.05    0.81   9545.16
corpus-dbpedia-entity-arctic-0.fvec         ivf       40         0.93              0.00           0.00  1075.27    0.85  12786.62
corpus-dbpedia-entity-arctic-0.fvec         ivf       50         1.08              0.00           0.00   925.93    0.87  15749.40

@benwtrent benwtrent requested review from iverase and john-wagster July 1, 2025 18:25
@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 1, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@benwtrent benwtrent added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Jul 1, 2025
@elasticsearchmachine elasticsearchmachine merged commit c1a4f8a into elastic:main Jul 1, 2025
32 checks passed
@benwtrent benwtrent deleted the qa-checkvec-fix-multi-runs branch July 1, 2025 19:58
Copy link
Contributor

@iverase iverase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

belated lgtm

I did not know we could do something like this, output looks cool.

mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Jul 2, 2025
We now can do multiple query time parameters, but this sort of broke
running multiple indexing loads. 

This corrects multiple indexing & search tests for the KnnIndexTester:

Example output.

```
index_name                           index_type  num_docs  index_time(ms)  force_merge_time(ms)  num_segments
-----------------------------------  ----------  --------  --------------  --------------------  ------------
corpus-quora-E5-small.fvec.flat             ivf    100000            4386                  4522             0
cohere-wikipedia-docs-768d.vec              ivf    100000            7784                  8331             0
corpus-dbpedia-entity-arctic-0.fvec         ivf    100000            7513                  8969             0

index_name                           index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count      QPS  recall   visited
-----------------------------------  ----------  -------  -----------  ----------------  -------------  -------  ------  --------
corpus-quora-E5-small.fvec.flat             ivf       10         0.87              0.00           0.00  1149.43    0.88   5511.60
corpus-quora-E5-small.fvec.flat             ivf       20         0.73              0.00           0.00  1369.86    0.94  10101.70
corpus-quora-E5-small.fvec.flat             ivf       30         0.76              0.00           0.00  1315.79    0.95  14382.15
corpus-quora-E5-small.fvec.flat             ivf       40         0.84              0.00           0.00  1190.48    0.96  18554.68
corpus-quora-E5-small.fvec.flat             ivf       50         0.99              0.00           0.00  1010.10    0.97  22609.69
cohere-wikipedia-docs-768d.vec              ivf       10         0.39              0.00           0.00  2564.10    0.61   2567.00
cohere-wikipedia-docs-768d.vec              ivf       20         0.53              0.00           0.00  1886.79    0.78   5017.25
cohere-wikipedia-docs-768d.vec              ivf       30         0.66              0.00           0.00  1515.15    0.84   7486.77
cohere-wikipedia-docs-768d.vec              ivf       40         0.80              0.00           0.00  1250.00    0.88  10177.80
cohere-wikipedia-docs-768d.vec              ivf       50         0.95              0.00           0.00  1052.63    0.90  12744.67
corpus-dbpedia-entity-arctic-0.fvec         ivf       10         0.48              0.00           0.00  2083.33    0.62   3302.07
corpus-dbpedia-entity-arctic-0.fvec         ivf       20         0.66              0.00           0.00  1515.15    0.74   6333.91
corpus-dbpedia-entity-arctic-0.fvec         ivf       30         0.78              0.00           0.00  1282.05    0.81   9545.16
corpus-dbpedia-entity-arctic-0.fvec         ivf       40         0.93              0.00           0.00  1075.27    0.85  12786.62
corpus-dbpedia-entity-arctic-0.fvec         ivf       50         1.08              0.00           0.00   925.93    0.87  15749.40
```
mridula-s109 pushed a commit to mridula-s109/elasticsearch that referenced this pull request Jul 3, 2025
We now can do multiple query time parameters, but this sort of broke
running multiple indexing loads. 

This corrects multiple indexing & search tests for the KnnIndexTester:

Example output.

```
index_name                           index_type  num_docs  index_time(ms)  force_merge_time(ms)  num_segments
-----------------------------------  ----------  --------  --------------  --------------------  ------------
corpus-quora-E5-small.fvec.flat             ivf    100000            4386                  4522             0
cohere-wikipedia-docs-768d.vec              ivf    100000            7784                  8331             0
corpus-dbpedia-entity-arctic-0.fvec         ivf    100000            7513                  8969             0

index_name                           index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count      QPS  recall   visited
-----------------------------------  ----------  -------  -----------  ----------------  -------------  -------  ------  --------
corpus-quora-E5-small.fvec.flat             ivf       10         0.87              0.00           0.00  1149.43    0.88   5511.60
corpus-quora-E5-small.fvec.flat             ivf       20         0.73              0.00           0.00  1369.86    0.94  10101.70
corpus-quora-E5-small.fvec.flat             ivf       30         0.76              0.00           0.00  1315.79    0.95  14382.15
corpus-quora-E5-small.fvec.flat             ivf       40         0.84              0.00           0.00  1190.48    0.96  18554.68
corpus-quora-E5-small.fvec.flat             ivf       50         0.99              0.00           0.00  1010.10    0.97  22609.69
cohere-wikipedia-docs-768d.vec              ivf       10         0.39              0.00           0.00  2564.10    0.61   2567.00
cohere-wikipedia-docs-768d.vec              ivf       20         0.53              0.00           0.00  1886.79    0.78   5017.25
cohere-wikipedia-docs-768d.vec              ivf       30         0.66              0.00           0.00  1515.15    0.84   7486.77
cohere-wikipedia-docs-768d.vec              ivf       40         0.80              0.00           0.00  1250.00    0.88  10177.80
cohere-wikipedia-docs-768d.vec              ivf       50         0.95              0.00           0.00  1052.63    0.90  12744.67
corpus-dbpedia-entity-arctic-0.fvec         ivf       10         0.48              0.00           0.00  2083.33    0.62   3302.07
corpus-dbpedia-entity-arctic-0.fvec         ivf       20         0.66              0.00           0.00  1515.15    0.74   6333.91
corpus-dbpedia-entity-arctic-0.fvec         ivf       30         0.78              0.00           0.00  1282.05    0.81   9545.16
corpus-dbpedia-entity-arctic-0.fvec         ivf       40         0.93              0.00           0.00  1075.27    0.85  12786.62
corpus-dbpedia-entity-arctic-0.fvec         ivf       50         1.08              0.00           0.00   925.93    0.87  15749.40
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants