GC and lifetime of objects captured by closure

[acaly]

I noticed this question recently: How does lambda affects the exact lifetime of its captured objects?

Yes it should keep it alive in order to have access to it. But consider the code below:

using System;
using System.Runtime.CompilerServices;

public class C {
    public int Value;
    public Func<C, int> M(C x, C y) {
        M2(z => z.Value == y.Value);
        return z => z.Value + x.Value;
    }
    [MethodImpl(MethodImplOptions.NoInlining)]
    public static void M2(Func<C, bool> x) {
    }
}

Here C y is only temporarily used in M, but its reference is carried by the closure that returns, as confirmed here. If the caller to M holds a reference to the returned delegate, both x and y would be alive, although no one should be able to access y any more.

Isn't this considered to be a kind of memory leak? I imagine the problem will deteriorate for larger methods that use many lambdas (for example, for Linq).

Note that the spec says:

However, the lifetime of a captured outer variable is extended at least until the delegate or expression tree created from the anonymous function becomes eligible for garbage collection.

So the compiler seems to be obeying the spec.

[qrli]

As a default implementation, the closure is allocated on heap, so it is up to GC to clean them up, just as all reference objects. If you count that as a leak, then all reference objects also leak, before GC kicks in.

Surely the compiler can do better (to not allocate on heap) in theory, but it is hard if the M2 is in another scope, and impossible if M2 is in another assembly, because the compiler need to prove that M2 does not store the reference to the lambda. That's one reason why local method feature was introduced, which makes it easier to do that optimization.

[acaly]

@bernd5 I mentioned the spec in my original post.

https://github.com/dotnet/csharplang/blob/main/spec/expressions.md#captured-outer-variables

However, the lifetime of a captured outer variable is extended at least until the delegate or expression tree created from the anonymous function becomes eligible for garbage collection.

HaloFour's link to the other issue actually contains a real-world example: dotnet/roslyn#117 (comment). Unfortunately, even that did not change LDM's dicision.

I guess the point is, normally one would just write lambdas in simple form, so unless the user actually realizing the problem, the new syntax does not help either. But after the user realizes it, without the new syntax, there are still something we can do to avoid it (for example as mentioned in the other issue, make a new block scope for that object).

[bernd5]

But a implementation which checks for non-overlapping capturing is not prohibited by that spec. It says only that the lifetime of captured object(s) are not bound to the current stackframe - their lifetime is extended to the lifetime of the capturing delegate / anonymous function / expressions tree / local function.
Roslyn does not go this way to reduce allocations and maybe because of additional reasons.

[TonyValenti]

Is there a way for me to identify functions that have multiple closures in them which could be causing this scenario? I'm pretty sure I have a memory leak in my code because of a small variable that is inadvertently holding onto a large one.

[acaly]

Although it sounds like an analyzer's work, I think it's even easier to directly search for compiler generated closure types with more than one methods. This can be done with System.Reflection or Mono.Cecil.

[qrli]

@TonyValenti If you have VisualStudio Enterprise, it comes with nice memory debugger/analyzer, which can trace and diagnose the object reference chain easily.
If not, you can use WinDbg. There are a few low level SOS commands to learn, but as long as you get past that, things are pretty easy too. It typically takes me 15 minutes to search and relearn the commands to find the root cause each time.