CacheBackedEmbeddings should update the store periodically (by computing embeddings in batches)

[chrispy-snps]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Currently, CacheBackedEmbeddings computes vectors for all uncached documents before updating the store. I propose that the store should be periodically updated by computing embeddings in batches.

Motivation

I noticed this when I was trying this feature out on our 30k document set and the cache directory hadn't appeared on disk after 30 minutes.

The motivation is to minimize compute/data loss when problems occur:

• If there is a transient embedding failure (e.g. a network outage at the embedding endpoint triggers an exception), at least the completed vectors are written to the store instead of being discarded.
• If there is an issue with the store (e.g. no write permissions), the condition is detected early without computing (and discarding!) all the vectors.

Proposal (If applicable)

Compute and store the embedding vectors in batches. Provide a new batch_size configuration parameter to CacheBackedEmbeddings instances.

[chrispy-snps]

I submitted #18070 to implement this.