CFCR Upgrade to 0.44 failed. Worker node not able to join the cluster back.

[rajendragosavi]

What happened:
Upgrade from 1.16 to 1.17 failed. During bosh upgrade, one of the worker node is not able to join the cluster.

What you expected to happen:
We expected the cluster upgrade should work properly.

How to reproduce it (as minimally and precisely as possible):
We updated the deploy.yaml to upgrade our existing cluster. We changed the kube-release as mentioned in release 0.44 and ran bosh deploy <deploy.yaml>.

Anything else we need to know?:

kube_proxy.stderr.log

E0212 16:34:28.487109 9841 node.go:124] Failed to retrieve node info: nodes "ip-10-72-72-253.us-west-2.compute.internal" not found
E0212 16:34:29.644935 9841 node.go:124] Failed to retrieve node info: nodes "ip-10-72-72-253.us-west-2.compute.internal" not found
E0212 16:34:32.045740 9841 node.go:124] Failed to retrieve node info: nodes "ip-10-72-72-253.us-west-2.compute.internal" not found
E0212 16:34:36.421604 9841 node.go:124] Failed to retrieve node info: nodes "ip-10-72-72-253.us-west-2.compute.internal" not found
E0212 16:34:45.592443 9841 node.go:124] Failed to retrieve node info: nodes "ip-10-72-72-253.us-west-2.compute.internal" not found
I0212 16:34:45.592464 9841 server_others.go:145] can't determine this node's IP, assuming 127.0.0.1; if this is incorrect, please set the --bind-address flag
I0212 16:34:45.592473 9841 server_others.go:150] Using iptables Proxier.
I0212 16:34:45.593595 9841 server.go:529] Version: v1.16.8
I0212 16:34:45.594005 9841 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
I0212 16:34:45.594038 9841 conntrack.go:52] Setting nf_conntrack_max to 131072
I0212 16:34:45.594100 9841 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
I0212 16:34:45.594145 9841 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
I0212 16:34:45.594325 9841 config.go:313] Starting service config controller
I0212 16:34:45.594342 9841 shared_informer.go:197] Waiting for caches to sync for service config
I0212 16:34:45.594412 9841 config.go:131] Starting endpoints config controller
I0212 16:34:45.594427 9841 shared_informer.go:197] Waiting for caches to sync for endpoints config
I0212 16:34:45.694495 9841 shared_informer.go:204] Caches are synced for service config
I0212 16:34:45.694507 9841 shared_informer.go:204] Caches are synced for endpoints config
W0212 17:14:58.740087 9841 reflector.go:299] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: too old resource version: 188 (472)
W0212 17:29:32.049761 9841 reflector.go:299] k8s.io/client-go/informers/factory.go:134: watch of *v1.Service ended with: too old resource version: 472 (7461)

Environment:

• Deployment Info (bosh -d <deployment> deployment):
• Environment Info (bosh -e <environment> environment):
• Kubernetes version (kubectl version): 1.16
• Cloud provider (e.g. aws, gcp, vsphere): aws