Linkerd Crashed

[husnerabbi19]

Hi,
We have deployed Linkerd on a Kubernetes Cluster.

Node Details

• OS (CentOS Linux release 7.9.2009)
• Kernel (3.10.0-1160.59.1.el7.x86_64)
• RAM 64GB, CPU 6

Kubernetes Cluster Details

• Self managed created by kubeadm
• version (1.17.5)
• single node cluster

Linkerd Details

• version (stable-2.11.1)
• Deployed by using the Getting Started Guide

We have injected Linkerd into our Deployment and Statefulset.
Initially it works fine, however after 2 days Linkerd components crash.

Here are the Logs of Linkerd POD.

Warning Failed pod/orchestration-controller-5b6d475f94-bxjz9 Error: failed to start container "linkerd-init": Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:275: applying cgroup configuration for process caused \"mkdir /sys/fs/cgroup/memory/kubepods/burstable/pod87f10c9a-b458-4337-a670-1945702f4577/linkerd-init: cannot allocate memory\"": unknown

Let me know, if any other info/logs are required.

[olix0r]

Note that the error is not coming from Linkerd, but from your container runtime. cannot allocate memory might suggest that the node has no available memory?

[husnerabbi19]

Thank you @olix0r for your reply.

Please find below the 'RAM Usage' details snapshot,

Another thing to note is that if we deploy our application without Linkerd on the same node and with the same software version, our application works fine without any crash loopback issue.

[kleimkuhler]

This is not exactly a solution you may be looking for, but it looks like this could be related to the Node's OS version based off conversations in docker/for-linux#841. CentOS 7.9.2009 seems to be problematic. If you can try testing with a different OS on the node it may be worth seeing if you still run in to any issues.