Reaping connections that are no longer useful

[ioquatix]

socketry/falcon#101

Potential options:

1. Timer based loop, removing connections which are not being actively used.
2. On acquire, close unused connections.
3. On release, close unused connections.

Regarding connection management, there may be some minimum allowable number of persistent connections. If we add the time the connection was last used, we can also implement a metric for determining utilization. For example, if there are lots of small requests, which tend to overlap, we don't want to close those connections even if the usage is 0... because they will probably be used again in the future. You'd also want to return hot resources first... so that if your request volume drops, you will eventually close older connections.

The main concerns are the additional book keeping required (probably better to use Async::Clock.now for monotonic elapsed time), and additionally the cost of additional book keeping (maybe a priority queue of resources from hot to cold).

cc @tleish

[ioquatix]

I've added a gardener to the pool who will clean up. It's not periodical but we could consider adding that.

[ioquatix]

Ideas:

• https://www.schneems.com/2020/07/08/a-fast-car-needs-good-brakes-how-we-added-client-rate-throttling-to-the-platform-api-gem/
• https://making.pusher.com/production-ready-connection-pooling-in-go/