"Encountered error in routeChat" error emitted for every call in Java ServiceCallout

[DinoChiesa]

I have a Java authorization callout working - modeled after the Redirect or JwtAuth samples in this repo.

For every call at the end of the call, I see this in the logs;

io.grpc.StatusRuntimeException: CANCELLED: client cancelled
	at io.grpc.Status.asRuntimeException(Status.java:524)
	at io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onCancel(ServerCalls.java:291)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.closedInternal(ServerCallImpl.java:375)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.closed(ServerCallImpl.java:364)
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1Closed.runInContext(ServerImpl.java:910)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Why?

This is being emitted by the ServiceCallout class that is part of this repo.

Details

I modified it slightly to emit the stack trace.

  /**
   * gRPC service implementation that handles the processing of requests and sending of responses.
   */
  private class ExternalProcessorImpl extends ExternalProcessorGrpc.ExternalProcessorImplBase {

    @Override
    public StreamObserver<ProcessingRequest> process(
        final StreamObserver<ProcessingResponse> responseObserver) {
      return new StreamObserver<ProcessingRequest>() {
        @Override
        public void onNext(ProcessingRequest request) {
          responseObserver.onNext(processRequest(request));
        }

        @Override
        public void onError(Throwable t) {
          logger.log(
              Level.WARNING, String.format("Encountered error in routeChat: %s", t.toString()));
          t.printStackTrace();
        }

        @Override
        public void onCompleted() {
          responseObserver.onCompleted();
        }
      };
    }
  }

I generally understand what "Cancelled" means. It seems in this instance the LB is cancelling the connection to my Java callout. Why? I'm not doing anything exotic in onRequestHeaders(). It either returns, or uses buildImmediateResponse with a statusCode. In either case I see the exception, always.