Service failed error=channel closed

[viktorvoltaire]

Hi, i recently started testing linkerd for grpc load balancing in GKE. Im running version:

Client version: stable-2.10.1
Server version: stable-2.10.1

Previously we have been using istio and that has pretty much worked out of the box. So i belive our setup to be nothing special really.

So what I am seeing a lot of in my sidecar logs are, like every 10 seconds, and they are towards different orig_dst
[ 67737.411248s] WARN ThreadId(01) server{orig_dst=10.3.2.213:80}: linkerd_reconnect::service: Service failed error=channel closed

This doesnt always affect our application, but some times together with the above we get
[ 28425.112681s] WARN ThreadId(01) server{orig_dst=10.3.2.213:80}: linkerd_app_core::errors: Failed to proxy request: connection error: broken pipe client.addr=10.2.7.69:40698

Which in turn will affect our application, that will throw a:

io.grpc.StatusRuntimeException: UNAVAILABLE: connection closed

Is there something we need to tweak for regular grpc loadbalancing? Any input here is appriciated, i have heard a lot of good about linkerd so I dont want to give up just yet

[olix0r]

Hi @viktorvoltaire,

Is the target server (e.g. at 10.3.2.213) meshed as well? are there any logged messages that on the server's proxy that correlate with these errors?

No configuration should be required for gRPC load balancing, but we'll probably need some more information to understand why connections are being torn down.