Memory leak when returning PipeReader

[AArnott]

Discussed in #1145

^{Originally posted by crazyjncsu February 7, 2025}
First, sorry, but I have limited familiarity with the inner workings here. But we have some code about like the following:

      using (var multiplexingStream = await MultiplexingStream.CreateAsync(monitoredWebsocket, new MultiplexingStream.Options() { ProtocolMajorVersion = 3 }))
      using (var rpcChannel = await multiplexingStream.AcceptChannelAsync(MultiplexedChannelNames.JSON_RPC))
      using (var formatter = new JsonMessageFormatter() { MultiplexingStream = multiplexingStream })
      using (var headerDelimitedMessageHandler = new HeaderDelimitedMessageHandler(rpcChannel, formatter))
      using (var jsonRpc = new JsonRpc(headerDelimitedMessageHandler, new BackendRPC(this)))
      {
        jsonRpc.CancelLocallyInvokedMethodsWhenConnectionIsClosed = false;
        _agentAPI = jsonRpc.Attach<IOurRpc>();
        jsonRpc.StartListening();

        var finalTask = await Task.WhenAny(multiplexingStream.Completion, jsonRpc.Completion).WithCancellation(_wsConnectionCts.Token);

We yield at Task.WhenAny and you can see the _agentAPI is saved off for others to use.

With IOurRpc looking about like this:

public interface IOurRpc { Task<PipeReader> DoThingAsync(); }

We are calling DoThingAsync, all is working, and we call .Complete() on the resulting PipeReaders (confirmed).

The issue is what we perceive as a memory leak in MessageFormatterDuplexPipeTracker

Upon returning a pipe for consumption, it adds the channel to a list of channels internally:

But the only time these are removed are in CleanupInboundResources:

... which is not called upon Complete'ing the PipeReader. And is only called upon the ResponseSent from JsonRpc, which does not get called in this case.

So we get gigs and gigs of memory leaked.

My guess is the channel should never be added to inboundRequestChannelMap? I don't know, but I figured an expert here could maybe give it a look?

[crazyjncsu]

Any chance you’ve had a chance to look into this? Do you need any help with the diagnosis?