Adjust size of in-memory block cache #47
Description
bifrost-gateway runs with in-memory 2Q cache with size set to 1024 blocks.
2Q is an enhancement over the standard LRU cache in that it tracks both frequently and recently used entries separately. This avoids a burst in access to new entries from evicting frequently used entries.
Current cache performance: ~50% cache HIT
Cache metrics from bifrost-stage1-ny after one day (~48%):
ipfs_http_blockstore_cache_hit 7.273594e+06
ipfs_http_blockstore_cache_requests 1.515003e+07
And second sample from other day (~50%):
ipfs_http_blockstore_cache_hit 2.7508843e+07
ipfs_http_blockstore_cache_requests 5.4966088e+07
iiuc the above means that in-memory "frecency" cache of 1024 blocks produces cache HIT ~50% of time.
This is not that surprising, every website will cause the same parent blocks to be read multiple times for every subresource on a page.
We run on machines that have 64GiB of memory and bifrost-gateway only utilizes ~5GiB.
Proposal: increase cache size
Improving cache hit here won't improve things like video seeking or fetching big files, but will have impact for how fast popular websites and directory enumerations load, avoiding trashing of the most popular content.
Tasks
- refactor cache size configuration: remove CLI parameter, and use ENV variable instead (to match plan from Update Saturn logger configuration #43 and new configuration convention agreed with George)
- with ability to tweak cache size with env variable, run experiments on
bifrost-stage1-nyand increase block cache size, let's say initiallly x5 (to 5120 blocks) and see if it improves cache hit, or if it produces diminishing returns. - Once we find the optimal cache size on staging, update implicit default