Why NetworkUnavailable condition is not reported on nodes?

[nicolas-goudry]

Hi, I’m using the VPC CNI on several EKS managed clusters and on each of them I can’t see the NetworkUnavailable condition being reported on nodes, is that a missing feature or a misconfiguration of the CNI? I went along configuration variables and nothing is mentioning node conditions.

Here is an example of NetworkUnavailable condition reported by Weave CNI on an on-premise cluster:

$ kubectl describe node NODE_NAME
Conditions:                                                                                                                                                                                                                                  
  Type                          Status  LastHeartbeatTime                 LastTransitionTime                Reason                          Message                                                                                          
  ----                          ------  -----------------                 ------------------                ------                          -------                                                                                          
...
  NetworkUnavailable   False   Fri, 08 Jul 2022 13:16:01 +0000   Fri, 08 Jul 2022 13:16:01 +0000   WeaveIsUp                    Weave pod has set this
...

I can confirm this condition is also reported on other cloud provider’s managed clusters.

Here is the output on an EKS cluster (I tested on several, it’s always missing):

Conditions:                                                                                                                                              
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message                      
  ----             ------  -----------------                 ------------------                ------                       -------                      
  MemoryPressure   False   Thu, 22 Sep 2022 22:40:53 +0000   Fri, 09 Sep 2022 02:05:24 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Thu, 22 Sep 2022 22:40:53 +0000   Tue, 26 Apr 2022 01:02:40 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure                                                                                     
  PIDPressure      False   Thu, 22 Sep 2022 22:40:53 +0000   Tue, 26 Apr 2022 01:02:40 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Thu, 22 Sep 2022 22:40:53 +0000   Tue, 26 Apr 2022 01:03:10 +0000   KubeletReady                 kubelet is posting ready status

Any idea?

Thanks ✌️

[jayanthvn]

Yes the feature is missing, AWS VPC CNI doesn't update the node status once CNI comes up. It would be a good feature to add. Can you please open a feature request?

[nicolas-goudry]

Thanks for your confirmation @jayanthvn 🙂

I just opened a feature request about that.

[nicolas-goudry]

Since the feature request has been closed, how should we proceed to track the availability state of the CNI?

cc @jchen6585 @jdn5126

[jdn5126]

The mechanism for tracking availability is the health of the aws-node pod, so that is what will need to be relied on. We are working to optimize the initialization time for the aws-node container to minimize the time pods would be waiting for an IP address during the event described in this discussion.

The challenge here is that any solution to this problem either drastically increases the security footprint of the aws-node pod or adds a lot of complexity and API server overhead to the controller. So those drawbacks outweigh the gain of solving this problem, which is narrow and should happen very infrequently. Upgrades of the aws-node daemonset should already be performed during maintenance windows.

[jdn5126]

Closing as outdated. Support for "Network Unavailable" condition is not something we have plans to implement, so please file a feature request at https://github.com/aws/containers-roadmap/issues if you would to get more visibility for this.