Application pod fails to connect to Kubernetes API server

[jbmassicotte]

We use the Apache Ignite grid in an Azure Kubernetes cluster. The Ignite nodes are configured to form a Kubernetes service. At startup each Ignite node connects to the Kubernetes API server, retrieves the IP addresses of the nodes in the Ignite service, and subsequently connects to these nodes to form a grid.

The connection to the Kubernetes API server fails whenever linkerd is enabled in the application namespace, causing the Ignite pods to terminate; without linkerd no such error occurs and all nodes join to form a grid. What is curious though is that with linkerd enabled, the ignite pods will be restarted (by kubertenetes, as expected), and seem to be able to get the IPs from the API server on this second attempt, and eventually able to join the grid.

Here is the error message returned by Ignite:

class org.apache.ignite.IgniteException: Unable to establish secure connection. Was remote cluster configured with SSL? [rmtAddr=/10.244.6.100:47500, errMsg="Remote host terminated the handshake"]

My simplistic understanding of linkerd is that traffic leaving the Ignite pods do so via the linkerd proxy, but since the destination is not linkerd enabled, packets should leave the pod unaltered.

Any explanation why the calls to the API server are failing?
And suggestions on how to resolve the above problem?

[ihcsim]

This sounds similar to #4980. Try skip the outbound 443 port per comment at #4980 (comment).

[jbmassicotte]

@ihcsim Oh thank you so very much! And to @Pothulapati too, for his #4980 answer.
Using config.linkerd.io/outbound-ports: "8080" did the trick for us.

[cpretzer]

@jbmassicotte thanks for the update, glad to hear that you got it resolved