Making the cluster cache global can result in highly increased memory usage for mono-ZIM-file use cases

[veloman-yunkan]

This ticket grows from kiwix/libkiwix#1265

Making the cluster cache global (#960) had a fallout in the form of increased memory usage in mono-ZIM-file scenarios (when a single ZIM file is open at any given time). Global cache was considered mainly in the context of kiwix-serve where multiple ZIM files are simultaneously active, and each having its own independent cache could lead to indefinite growth of memory usage. Once the cache was made global (shared across all ZIM archives) it was natural to set a high default max limit so that kiwix-serve's performance didn't suffer. However, for mono-ZIM-file use cases this effectively translated to a significantly increased limit for the cluster cache - going from 16 to about half a thousand in terms of cluster count (assuming 2MiB clusters).

Though affected client applications can explicitly lower the cluster cache limit to the previous level, @kelson42 considers the change of the default behavior a regression. An easy way to fix that regression is to lower the default limit of the (global) cluster cache. But that will mean a regression for kiwix-serve and other multi-ZIM applications!

One (more complex) solution that will satisfy both sides, is to bind the current cache size limit to the number of open ZIM files while capping it at the hard limit, i.e.

cache_memory_limit = min(512MiB, 16MiB * (number of open ZIM files))

[benoit74]

I don't think this is a libzim issue, we have the ability to tune the cache size with zim::setClusterCacheMaxSize

And from my understanding, the pretty high default max limit was meant to not have users complains about bad performances (due to too small cache size) and let users having memory issues set more conservative values.

It is impossible (or way too complex / risky at this stage) to have a solution to satisfy all sizes. Usage is too diverse. Users have to understand the libs they are using, the constraints their device is having, and configure the first with the second.

[kelson42]

@veloman-yunkan Thank you for opening the issue. So here are the things to do:

• Set cluster cache default values to numbers which are compatible with a usage on low-end devices. I recommend to set them in a way that the memory usage is similar to previous the revamping (with one or only a few ZIM)

• Implement Expose new libzim settings to handle memory consumption kiwix/kiwix-tools#745 so we can improve performance on kiwix-serve when needed or possible.

• I plan to open an issue to gather statistics about cluster cache hit statistic. My intuition is that this uses a lot of memory but the cache is actually hit rarely.

• I clearly don't want to bind the (max) cache size to the number of ZIMs.

[veloman-yunkan]

• Set cluster cache default values to numbers which are compatible with a usage on low-end devices. I recommend to set them in a way that the memory usage is similar to previous the revamping (with one or only a few ZIM)

Done in #1038