Are there any guidelines for optimizing bulk Upserts?

[myyrakle]

1. In the case of Elasticsearch, if you don't do write processing in batches, there is a serious performance waste.
   What about qdrant?

2. I think that qdrant will also be more efficient if you do batch writes in batches.
   Then, how many should you use as the optimal write batch unit? Should you consider both the size and the number?

3. Isn't it inefficient to update or delete existing data? (Compared to Elasticsearch)

[timvisee]

In the case of Elasticsearch, if you don't do write processing in batches, there is a serious performance waste. What about Qdrant?

Batching is still the most efficient way. Though I believe not bulking is less bad vs ES.

In case of bulk updates, we do recommend to temporarily disable HNSW building.

Please see: https://qdrant.tech/documentation/database-tutorials/bulk-upload/

I think that qdrant will also be more efficient if you do batch writes in batches.
Then, how many should you use as the optimal write batch unit? Should you consider both the size and the number?

There's no optimal value that works for everyone. If each individual update is quite large already, pick a lower number. If each individual update is very cheap, pick a higher number.

You don't want your batch to be too large because that may 'block' the collection for a long time.

In many places we default to 10-100 items per batch. I'd recommend to pick something like that and see how it performs.

Isn't it inefficient to update or delete existing data? (Compared to Elasticsearch)

It definitely is not free, but that does not mean you should not use it.

If you want to change data, I'd recommend to update it directly and not to delete first and then reinsert.