[DiskBBQ] Fix our silly imbalanced cluster issue

[benwtrent]

Description

Cluster imbalance is a known issue and yeah, we are hitting it. I consistently see oversized clusters (multiples in size of our configured limit). This can lead to inconsistent search latency behavior for different queries (what if the query is nearer only tiny clusters or nearer only the huge ones?) and can harm our logic for fetching postings lists.

We should strive for cluster size consistency, even with SOAR overspilling.

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)