Proposal: Persisted Method-Scoped Locals

[Hawkynt]

Summary:

This proposal introduces a new language feature in C# that allows developers to declare locals within a method that retain their values across method invocations. The scope of these locals can be finely controlled, enabling access from method overloads and overrides based on the developer's requirements. The feature is designed to enhance method encapsulation, facilitating more stateful method implementations while maintaining clear and concise method signatures.

Motivation:

In certain scenarios, developers might require the ability to retain state within a method across invocations without exposing this state at the class level or resorting to static fields, which are shared across all instances. Such capability would be particularly useful in recursive methods, stateful iterators, or methods with a significant computational state that should persist between calls. Current workarounds involve using class-level fields or static variables, which can lead to less intuitive code and potential state management issues, especially in multi-threaded contexts.

Syntactic Examples:

public class MyClass
{
    public void MyMethod()
    {
        persist var methodLocal = 0;
        methodLocal++;
        Console.WriteLine(methodLocal);
    }

    public static void MyMethod2()
    {
        static persist var methodLocal = 0;
        methodLocal++;
        Console.WriteLine(methodLocal);
    }

    public void MyMethod3()
    {
        persist var methodLocal = 0;
        static persist var staticMethodLocal = 0;
        methodLocal++;
        staticMethodLocal--;
        Console.WriteLine(methodLocal);
    }

    // memoization support
    public static void Calculate(int value)
    {
        static persist var resultCache = new Dictionary<int, double>();
        return resultCache.GetOrAdd(value, v => Math.Sqrt(value));
    }
}

Design Specification:

1. Syntax: Introduce a new keyword persist to declare a method-scoped persisted local. This local is only visible within the method and/or its specified overloads/overrides. We also allow static persist for static persistance.

2. Visibility Control: The visibility of persisted locals can be precisely managed to define their accessibility scope. For instance, declaring persist internal int x enables the variable x to be accessed within overloads or overrides contained within the same assembly. Using persist private int x restricts the visibility to the declaring method itself, excluding any of its overloads or overrides. Conversely, persist protected int x allows the variable to be accessible in overriding methods, fostering inheritance compatibility. By default, overloads can access persisted locals unless the private modifier is explicitly applied, ensuring a flexible yet secure approach to method-specific state management.

3. Implementation Mechanics and Initialization: The C# compiler will internally generate a private nested class for each method requiring persisted locals, encapsulating the persisted state within fields.

• Instance Methods: For instance methods, the nested class is instantiated once per containing class instance, maintaining a unique state per instance.
• Static Methods: For static methods, a single static nested class instance is maintained, ensuring a consistent state across all class instances.
• Initialization: The initialization of persisted locals depends on their access modifier:
  • Private Modifier: When persist private is used, the initial value is hardcoded into the field initializer of the nested class, ensuring that the value is tightly bound to the method.
  • Other Modifiers: For other access levels (internal, protected, etc.), a parameterized constructor in the nested class is utilized. The first method invocation dictates the initial value, which is then passed to the nested class constructor.

Code Example Demonstrating Initialization Mechanisms:

// examples for static methods
public class StaticClassExample
{
    // Nested class for MyStaticMethod (demonstrating private initialization)
    [CompilerGenerated]
    private class MyStaticMethodState
    {
        public int methodLocal = 10; // Hardcoded initial value for private
    }

    // Static field for the state of MyStaticMethod
    private static MyStaticMethodState _myStaticMethodState;

    public static void MyStaticMethod()
    {
        // static persist private var methodLocal = 10;
        _myStaticMethodState ??= new MyStaticMethodState();

        _myStaticMethodState.methodLocal++;
        Console.WriteLine(_myStaticMethodState.methodLocal);
    }

    // Nested class for MyOtherStaticMethod (demonstrating parameterized initialization)
    [CompilerGenerated]
    private class MyOtherStaticMethodState
    {
        public int methodLocal;

        public MyOtherStaticMethodState(int initialValue) => methodLocal = initialValue;
    }

    // Static field for the state of MyOtherStaticMethod
    private static MyOtherStaticMethodState _myOtherStaticMethodState;

    public static void MyOtherStaticMethod()
    {
        // static persist var methodLocal = 20;
        _myOtherStaticMethodState ??= new MyOtherStaticMethodState(20);

        _myOtherStaticMethodState.methodLocal++;
        Console.WriteLine(_myOtherStaticMethodState.methodLocal);
    }

    public static void MyOtherStaticMethod(int initialMethodLocal)
    {
        // static persist var methodLocal = 30;
        _myOtherStaticMethodState ??= new MyOtherStaticMethodState(30);

        _myOtherStaticMethodState.methodLocal++;
        Console.WriteLine(_myOtherStaticMethodState.methodLocal);
    }
}

// examples for instance methods
public class MyClass
{
    // Nested class for class-wide (static) state
    [CompilerGenerated]
    private class MyMethodStaticState
    {
        public int methodStaticLocal = 5; // Hardcoded initial value for private
    }

    // Nested class for MyMethod (demonstrating private initialization)
    [CompilerGenerated]
    private class MyMethodState
    {
        public int methodLocal = 10; // Hardcoded initial value for private
    }

    private static MyMethodStaticState _myMethodStaticState;
    private MyMethodState _myMethodState;

    public void MyMethod()
    {
       // static persist private var methodStaticLocal = 5;
        _myMethodStaticState??= new MyMethodStaticState();

       // persist private var methodLocal = 10;
        _myMethodState??= new MyMethodState();

        _myMethodState.methodLocal++;
        _myMethodStaticState.methodStaticLocal--;
        Console.WriteLine(_myMethodState.methodLocal);
    }

    // Nested class for MyOtherMethod (demonstrating parameterized initialization)
    [CompilerGenerated]
    private class MyOtherMethodState
    {
        public int methodLocal;

        public MyOtherMethodState(int initialValue) => methodLocal = initialValue;
    }

    private MyOtherMethodState _myOtherMethodState;

    public void MyOtherMethod()
    {
        // persist var methodLocal = 20;
        _myOtherMethodState ??= new MyOtherMethodState(20);

        _myOtherMethodState.methodLocal++;
        Console.WriteLine(_myOtherMethodState.methodLocal);
    }

    public void MyOtherMethod(int parameter)
    {
        // persist var methodLocal = 30;
        _myOtherMethodState ??= new MyOtherMethodState(30);

        _myOtherMethodState.methodLocal++;
        Console.WriteLine(_myOtherMethodState.methodLocal + parameter);
    }
}

In this example:

• MyMethod uses a persist private local, initializing it directly within the nested class's field declaration (methodLocal = 10;).
• MyOtherMethod demonstrates a non-private persistent local. The initial value is not hardcoded but passed to the nested class's constructor, allowing dynamic initialization (new MyOtherMethodState(20);).

4. State Initialization: The first call to the method initializes the persisted local. Subsequent calls use the retained state. This way memory is only allocated for the state if a method is called at all. When initial state is not constant, the parametrized constructor is always used. However code needed to initialize is only called when the state is not yet initialized upon call.

public class ArrayInitializationExample
{
    // Nested class generated by the compiler for maintaining state.
    [CompilerGenerated]
    private class MyMethodState
    {
        public int[] methodLocal = new int[] {1, 2, 3, 4};
    }

    private MyMethodState _myMethodState;

    public void MyMethod()
    {
        // persist var methodLocal = new [] {1, 2, 3, 4};
        _myMethodState ??= new MyMethodState()
        // Use _myMethodState.methodLocal here...
    }
}

public class AnotherFuncInitializationExample
{
    [CompilerGenerated]
    private class MyOtherMethodState
    {
        public SomeType otherMethodLocal;

        public MyOtherMethodState(Func<SomeType> initializer) => otherMethodLocal = initializer();
    }

    private MyOtherMethodState _myOtherMethodState;

    // Assume this method is defined elsewhere and returns SomeType
    private SomeType CallAnotherFunction() { /* Implementation */ }

    public void MyOtherMethod()
    {
        // persist var otherMethod = CallAnotherFunction();
        _myOtherMethodState ??= new MyOtherMethodState(CallAnotherFunction);
        // Use _myOtherMethodState.otherMethodLocal here...
    }
}

[Richiban]

I know it's something of a cliche at this point, but you could do this today with a Roslyn analyzer. Ultimately, what's the difference between a "persisted method local" and a class instance field? The only difference is that the latter has limited scope, and any time you want a change to the language that is nothing more than a restriction on code that is currently legal, an analyzer is the way to do it.

With such an analyzer you could write the following:

public class MyClass
{   
    [ScopedTo(nameof(MyMethod))]
    int methodLocal = 0;
    
    public void MyMethod()
    {
        methodLocal++;
        Console.WriteLine(methodLocal);
    }
    
    public void MyMethod2()
    {
        methodLocal++; // Compiler error: `methodLocal` is scoped to `MyMethod` only
        Console.WriteLine(methodLocal);
    }
}

[Hawkynt]

not sure that the roslyn analyzer can do all that overload/overwrite checking, though?
Also the memory usage patterns would be different, when for example the method is never called (e.g in a library)

[AlgorithmsAreCool]

You could do this with an analyzer i think, but then you're created your own language dialect...

Also the memory usage patterns would be different, when for example the method is never called (e.g in a library)
I don't understand this, isn't the persisted variable basically always a member of the nested class, doesn't that always use memory?

[Hawkynt]

When the nested class isn't initialized it's only the null reference that's present.

[HaloFour]

See: #832

[HaloFour]

The first call to the method initializes the persisted local.

I'll note that one point that is mentioned in the other discussion is what happens when multiple threads are making that first method call at the same time. There are three potential threading policies that could be in play. Languages like VB and C++ provide guarantees that the initialization expression can only be executed exactly once and that other threads will block, but that comes with overhead on every call to that method.

[MgSam]

Presumably compiler would add a lock but only if one of these static locals was used.

[Hawkynt]

Double-checked-locking would solve this:

If(instance==null)
    Lock()
      If(instance==null)  // in case another thread got the init lock first
        Init();

This way the overhead would only occur during initialization

[HaloFour]

A lock reference (additional heap allocation) and an initialized flag (or a field on that allocated lock reference), per local. Plus the field itself would need to be volatile to ensure that the initialization is seen by all threads. That's still overhead incurred on every method call.

I'd probably agree that this is the closest to "correct" behavior, especially given both VB and C++ employ these techniques. But that's one of the reasons that the language team has steered clear of lazy evaluation as a language feature in general, let alone for static locals.

[Hawkynt]

I see. Maybe instead of trying to solve this ( and other potential races) by compiler sugar, we could,leave this to the user. If he needs to make sure to not double initialize, he can implement locking in the init method himself?

[AlgorithmsAreCool]

I'll throw in that I like the feature in principle. I think that it marries with the idea of local functions which are scoped to the method also.

I encounter this pattern usually in two situations

1. Variables that must be updated as a group
2. Variables that require locks/synchronization so they are dangerous to touch

That being said, my usual workaround is placing the dangerous field in its own class with safe accessors.

My workaround:

1. Works fine and isn't too hard.
2. Is more flexible than the proposed feature because I can share the field/variable between as many methods as I need
3. Allows me to control the synchronization mechanism (locks, interlocked, sheer willpower...)

Because of that I am not sure this feature will improve the situation enough to justify it. But i said the same thing about Local Functions also so 🤷‍♀️