[8.19] New threadpool-based merge scheduler which is disk space aware

[albertzaharovits]

This is the backport of a few PRs related to the implementation of the new threadpool-based merge scheduler.
The new merge scheduler uses a node-level threadpool (sized the number of CPU cores) to execute all the merges across all the shards on the node, limiting the amount of concurrently executing merges, irrespective of the number of shards that the node hosts. Smaller merges continue to have priority over larger ones.
In addition, the new merge scheduler implementation also monitors the available disk space on the node, so that it won't start executing any new merges when the available disk space becomes scarce (the used disk space gets above the indices.merge.disk.watermark.high (95%) limit (same as the the allocation flood stage (the limit that flips shards on the node to read-only))).
The new merge scheduler is now enabled by default (indices.merge.scheduler.use_thread_pool is true).

Here is the complete list of backported PRs:

• Threadpool merge scheduler by albertzaharovits · Pull Request #120869 · elastic/elasticsearch - this is the initial implementation of the new merge scheduler
• Start indexing throttling only after disk IO unthrottling does not keep up with the merge load by albertzaharovits · Pull Request #125654 · elastic/elasticsearch - this fixes a problem (where the merge scheduler throttles indexing too early) detected by nightly benchmarks in https://github.com/elastic/elasticsearch-benchmarks/issues/2437
• Slack merge throttling params for fewer merge tasks by albertzaharovits · Pull Request #126016 · elastic/elasticsearch - this slackens the disk IO throttling algorithm to generally do less disk IO throttling of merges
• Fix failure in ScalingThreadPoolTests after addition of merge scheduler by mark-vieira · Pull Request #125245 · elastic/elasticsearch, Fix ThreadPoolMergeSchedulerStressTestIT testMergingFallsBehindAndThenCatchesUp by albertzaharovits · Pull Request #125956 · elastic/elasticsearch, Fix ThreadPoolMergeSchedulerTests testMergeSourceWithFollowUpMergesRunSequentially by albertzaharovits · Pull Request #126050 · elastic/elasticsearch, Fix ThreadPoolMergeSchedulerTests testSchedulerCloseWaitsForRunningMerge by albertzaharovits · Pull Request #126110 · elastic/elasticsearch, and Fix IndexStatsIT by albertzaharovits · Pull Request #126113 · elastic/elasticsearch - these are fixes for test failures
• Threadpool merge executor is aware of available disk space by albertzaharovits · Pull Request #127613 · elastic/elasticsearch - this adds the feature that monitors for the available disk space, and blocks before starting any merges when the available disk space gets low
• Move MergeMemoryEstimator by pxsalehi · Pull Request #125686 · elastic/elasticsearch, and Expose merge events and their memory usage estimate by pxsalehi · Pull Request #126667 · elastic/elasticsearch - optional change that exposes heap memory estimations for merge tasks
• ES-10264: Bring over merge scheduler features from stateless by BrianRothermich · Pull Request #128155 · elastic/elasticsearch optional refactoring that provides a few extension hook methods (that are mostly used for metrics)

See also: #129134
Relates: ES-11701 ES-10046