libutil: Use ThreadPool in computeClosure to parallelize promise resolution

[pkpbynum]

Motivation

Note: I'm not much of a C dev (yet), and don't know enough about the codebase to know if this is "safe."

I found this while looking at slower-than-expected HTTP Binary Cache store query times. This line immediately blocks on the queryPathInfo promise when it is passed from computeFSClosure here. In the context of an HTTP binary cache store, this prevents HTTP requests from happening concurrently (HTTP2 or HTTP1).

computeFSClosure and computeClosure are used in quite a few places--including path-info, which I used to benchmark just by querying the Linux stdenv build-time closure & adjusting narinfo-cache-positive-ttl:

Before:

$ time nix path-info --store https://cache.nixos.org --recursive --option narinfo-cache-positive-ttl 10 /nix/store/7zz3zmv2a0ssmgqlfhy4rsb6ii6z475a-stdenv-linux.drv
... paths output
real    0m6.904s
user    0m0.314s
sys     0m0.209s

After:

$ time ./src/nix/nix path-info --store https://cache.nixos.org --recursive --option narinfo-cache-positive-ttl 10 /nix/store/7zz3zmv2a0ssmgqlfhy4rsb6ii6z475a-stdenv-linux.drv
... paths output
real    0m0.395s
user    0m0.097s
sys     0m0.072s

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.