Server-streaming call isn’t cancelled when client task is cancelled

[Obbut]

Describe the bug

Cancelling the client Task during a server-streaming RPC does not cancel the server-side handler: the server keeps successfully writing messages indefinitely. Neither Task.isCancelled nor context.cancellation.isCancelled becomes true during the call. Cancellation is only observed much later (when the graceful shutdown of the server starts).

Versions:

• grpc-swift: grpc-swift-2 v2.1.0 (GRPCCore, GRPCInProcessTransport)
• protoc-gen-grpc-swift: v2.1.1 (grpc-swift-protobuf)
• swift-protobuf: v1.31.0
• Swift: Swift version 6.1 (swift-6.1-RELEASE), Target: aarch64-unknown-linux-gnu
• Platform: Ubuntu 24.04.2 LTS (Dev Container); also reproduces on macOS
• Build: swift run (debug)
• Transport: In-process (GRPCInProcessTransport)

We are experiencing this issue when using the Swift NIO based HTTP2 transport as well. In our production code, we have a set-up with an iOS app, that connects over gRPC to a backend ("app API"), which in turn connects to another backend over gRPC ("core API"). The entire codebase is built with GRPC Swift 2 (server and client). We're noticing that streams that are closed by the app aren't properly cancelled on the App API or Core API.

To reproduce

git clone https://github.com/Obbut/GRPC-Swift-2-cancellation.git
cd GRPC-Swift-2-cancellation
swift run

The repro project makes 3 requests to a server streaming RPC in series. Each request is cancelled (by cancelling the Task) before starting the next.

As should be evident by observing the logs, the cancellation is not correctly propagated to the server. The writer keeps succeeding.

Expected behaviour

• The server‑side handler observes cancellation promptly.
• writer.write(...) throws or the handler exits quickly.
• context.cancellation.isCancelled and/or Task.isCancelled becomes true for that call.
• Per‑call resources are released (e.g., the activeWriters in the repro project returns to zero between calls).

Additional information

Repro project, main file: https://github.com/Obbut/GRPC-Swift-2-cancellation/blob/c597dc35930193a45cafb0addaba547037118158/Sources/GRPCSwift2Cancellation/main.swift
Repro project, service implementation: https://github.com/Obbut/GRPC-Swift-2-cancellation/blob/main/Sources/GRPCSwift2Cancellation/HelloWorldService.swift

[glbrntt]

Thanks @Obbut – I can reproduce this with the in-process transport. That's a bug in the in-process transport that we'll need to fix.

We are experiencing this issue when using the Swift NIO based HTTP2 transport as well.

I swapped out the transport in your repro to the NIO one and couldn't reproduce this issue.

Expected behaviour

...

• context.cancellation.isCancelled and/or Task.isCancelled becomes true for that call.

To clarify: context.cancellation.isCancelled should become true. Task.isCancelled should not become true. Propagating Task cancellation would make it very difficult to write a correct service (e.g. imagine the handler was cancelled mid-way through a DB update).

[Obbut]

Thanks for your response!

That makes sense. We initially observed this behavior with the HTTP/2 transport when testing end to end. I then wrote a unit test (in our backend’s test suite, which uses the in-process transport) to confirm the behavior, which led me to file this issue.

I haven’t tried my repro project with the HTTP/2 transport, so your observation suggests that the initial problem may actually lie elsewhere in our codebase (and not in gRPC Swift).

[Obbut]

@glbrntt I did some more testing and it appears that even with the HTTP2 Posix and TransportServices transports, context.cancellation.isCancelled never becomes true.

The reason the repro still showed the expected behavior in this case, is that the calls to writer.write throwed an error (I/O on closed channel). But context.cancellation.isCancelled never becomes true.

In my production set-up, we have an open subscription to Redis. The implication is that the resources for the gRPC streams (and Redis subscriptions) are not cleaned up until we try to write a new event, which could take a very long time.

I pushed a new branch, http2, to my repro project's git repo. This uses the HTTP2 transport, and I'm intentionally catching all errors from writer.write to demonstrate that isCancelled never becomes true.

[glbrntt]

Oh interesting, thanks for looking into that.

I see the bug, the state machine receives the RST_STREAM frame but thinks it doesn't need to propagate the cancellation because the client has sent its end of stream, so it just ignores it. See here: https://github.com/grpc/grpc-swift-nio-transport/blob/e59b4cfc25dde151aea8feaf1af9f1af65f549aa/Sources/GRPCNIOTransportCore/GRPCStreamStateMachine.swift#L1759

I'll try to get a fix up later today.

[Obbut]

Thanks so much! Let me know if there’s anything I can do to help.

[glbrntt]

FYI the PR is up: grpc/grpc-swift-nio-transport#124

[glbrntt]

The fix is available in https://github.com/grpc/grpc-swift-nio-transport/releases/tag/2.1.1

I'll leave this issue open to track the in-process transport bug.