Clustermedia cluster list watch multi cluster. Is the memory usage of the informer queue large on the host

[xumujian]

How much memory does the host need for the cluster running the clustermedia service

[Iceber]

clusterpedia replaces the informer's cache store with a store that only stores the resource version of the resource, reducing memory usage.

Of course, resources are cached in memory before they are processed by the storage layer.

[xumujian]

Therefore, for centralized multi cluster resource synchronization, the host of clustermpedia needs a lot of memory

[Iceber]

For kube pods, consider setting a high limit and a low request, since a normally running clusterpedia doesn't pile up a lot of kube resources and the memory footprint is very low at this time

[xumujian]

Our business scenario is mainly to watch the pod resources of multiple clusters, and report the changed pod data in real time when the container status in the pod changes. Now, when there are many clusters, memory may be a bottleneck

[Iceber]

Even though clusterpedia has a temporary memory usage during synchronization due to poor performance of the user-provided storage component(mysql/postgres), it still saves memory resources compared to using the native informer directly.

In addition, I'd like to add some concurrent cluster collection limits at startup to avoid a lot of temporary memory usage, but this would result in a longer time for all clusters to fully synchronize, and I'm not sure if this is necessary.

[xumujian]

If all events of resource changes in any cluster are written to the database, will database access be too frequent