Only cache objects managed by operator

Configure controller-runtime cache to only cache objects that have the commonly used labels set by the operator.
With that the operator will build up a cache for only the resources it manages.

Normally controller-runtime would build up a cache for all resources the client interacts with.
This is especially a problem for large cluster as then the operator would cache all Deployments and DaemonSets, which would then drastically increase memory usage.

With this change we can keep memory usage relatively small and also stable regardless the size of the cluster

Signed-off-by: Benjamin Isinger <[email protected]>

Author: isibeni

main.go

@@ -23,18 +23,21 @@ import (
 
 	// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
 	// to ensure that exec-entrypoint and run can make use of them.
+	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/apimachinery/pkg/runtime"
 	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
 	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 	_ "k8s.io/client-go/plugin/pkg/client/auth"
 	"k8s.io/client-go/rest"
 	ctrl "sigs.k8s.io/controller-runtime"
+	"sigs.k8s.io/controller-runtime/pkg/cache"
 	"sigs.k8s.io/controller-runtime/pkg/healthz"
 	"sigs.k8s.io/controller-runtime/pkg/log/zap"
 	metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
 
 	objectdriverv1alpha1 "github.com/IBM/ibm-object-csi-driver-operator/api/v1alpha1"
 	"github.com/IBM/ibm-object-csi-driver-operator/controllers"
+	"github.com/IBM/ibm-object-csi-driver-operator/controllers/constants"
 	"github.com/IBM/ibm-object-csi-driver-operator/controllers/util/common"
 	//+kubebuilder:scaffold:imports
 )
@@ -75,6 +78,9 @@ func main() {
 		HealthProbeBindAddress: probeAddr,
 		LeaderElection:         enableLeaderElection,
 		LeaderElectionID:       "af88e983.csi.ibm.com",
+		Cache: cache.Options{
+			DefaultLabelSelector: labels.SelectorFromSet(constants.CommonCSIResourceLabels),
+		},
 
 		// LeaderElectionReleaseOnCancel defines if the leader should step down voluntarily
 		// when the Manager ends. This requires the binary to immediately end when the