[core] Support batch vector search - #7857

This batch query is not meaningful for performance, we need to ensure that the local file cache is enabled. See ce38b2c

Do you mean we should add a validation for local-cache.enabled=true?

@JingsongLi Sorry, I accidentally deleted your comment while trying to reply.

I want to confirm if I understood your point correctly.

Batch query vectors share one scan and one reader/native index loading per split within one invocation. Do you mean that local file cache for meta/global-index files is sufficient for repeated single-vector queries, so the reader/native-index reuse provided by this batch API is not necessary?

Also, how is the return value handled? How do users know which input corresponds to which one?

Also, how is the return value handled? How do users know which input corresponds to which one?

Results keep the same order as input vectors — results.get(i) corresponds to queryVectors[i]

This batch query is not meaningful for performance, we need to ensure that the local file cache is enabled. See ce38b2c

I ran an OSS benchmark with a 529MB Lumina index, 100 queries, dim=1024.

  Without Paimon local cache:
  - user-level loop: 188.6s
  - batch: 39.7s

  With warm local memory cache:
  - user-level loop: 41.4s
  - batch: 39.7s

So without cache, batch is effective. With warm cache, the benefit is very small, which matches your
point.

Since users need a batch API, for the Spark/Flink batch API, which direction do you prefer?

1. implement it by looping single-vector search internally and relying on local cache;
2. continue this PR and add a batch vector search/readBatch path.

@JingsongLi @yunfengzhou-hub Could you help take a look again when you are free

And we should also support Python api.

+1