Using Linkerd Init Container with AWS CNI Customer Networking

[suharshs]

I am attempting to increase the number of IP addresses in my AWS EKS cluster by associating a new subnet using an ENIConfig which all Pods will use.

So my EKS cluster assigned nodes to 10.0.0.0/20 and my ENIConfig uses 100.64.0.0/18.l for all pods

I have installed linkerd in non-CNI mode using the helm script, and it has worked great for us so far.

When I add the ENIConfig and restart all nodes, pods are restarted in the new subnet, and the linkerd-proxy-injection pod fails with and all sidecar containers fail:

Warning  Unhealthy  1s                   kubelet  Liveness probe failed: Get "http://100.64.113.21:9995/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  1s                   kubelet  Readiness probe failed: Get "http://100.64.113.21:9995/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

I have made sure to associate the cluster's security groups with the ENIConfig, so launched pods have the same security group.

Are there any considerations I am missing regarding custom networking using ENiConfig in EKS and its interaction with linkerd's setup?

Please let me know if there is more information I can provide that would be helpful.

[stevej]

Is the control plane unhealthy? Can you share the output of linkerd check or kubectl describe

[suharshs]

Hi sorry for the delay, today I was able to diagnose the issue to not have the NaCL setup correctly for the new secondary cidr block we created. Once that block has the same network ingress/egress rules as the primary block things worked.

Thanks!