Resize block buffer after allocation if significant wasted space

[TheSafo]

What does this PR do?

When constructing a block cache, the cache pre-allocates a buffer of max block size for each block. Payload generators then serialize bytes into that buffer. The buffer previously was never resized and thus held onto its excess memory. This made caches with small blocks hold (# blocks * max block size) in memory instead of the specified cache size.

This change resizes blocks that use under half of the pre-allocated space. This should enforce that the buffers of a cache are now no more than 2x the maximum configured cache size.

Motivation

Significant memory usage from my payload generator that has lots of smaller blocks.

Additional notes

We could probably resize every block since this is only done on lading creation anyways - wdyt? 2x was an arbitrary decision here.

[TheSafo]

• Resize block buffer after allocation if significant wasted space #1674 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[GeorgeHahn]

This looks good to me. Thanks for investigating - this was an interesting finding.

[GeorgeHahn]

We could probably resize every block since this is only done on lading creation anyways - wdyt? 2x was an arbitrary decision here.

I went back and forth on this, but I do like the idea of resizing all blocks. Some extra work up front to reduce memory usage at runtime is probably the right tradeoff for lading as it runs in SMP jobs. That said, I'm not too sad about the 50% worst case inefficiency either. That's about the same as we could expect from allocating the BytesMut on the fly. Your call.