decouple IDisposable and using operator, define clear semantic

[dmitriyse]

I(Async)Disposable designed (as stated in the official documentation) to free-up unmanaged resorces when they are no longer needed without deferring this to GC.
Despite the official purpose, many frameworks adopts I(Async)Disposable for various other purposes, just to reuse using C# language construct - unofficial I(Async)Disposable use cases.

The most popular unofficial use case is implicit lifetime management. For example ILogger.BeginScope returns IDisposable token that closes scope when Dispose invoked, frequently used withing using block.

The aim of this proposal is to segregate from IDisposable such "unofficial use cases" to corresponding abstractions.

Firstly we needs to allow reuse using operator for arbitrary purposes

        // using operator simplified microcode, along with existing for IDisposable
        { // using (ICodeScopeExtension obj = ....)
            try
            {
                // scope code
            }
            catch (Exception ex)
            {
                obj?.OnLoseCodeScope(ex);
                throw;
            }

            obj?.OnLoseCodeScope(null);
        }

    /// Contracts that using operator should start to support

    /// <summary>
    ///     Scoped language construct extension, primarily <see langword="using" /> operator.
    /// </summary>
    public interface ICodeScopeExtension // using operator should be extended to support also this interface.
    {
        /// <summary>
        ///     Executed by a language construct when the language construct loses its scope.
        /// </summary>
        /// <param name="exception">
        ///     Exception that was caused the scope loosing, or <see langword="null" /> for a normal execution
        ///     flow.
        /// </param>
        void OnLoseCodeScope(Exception? exception);
    }

    /// <summary>
    ///     Scoped language construct asynchronous extension, primarily <see langword="using" /> statement.
    /// </summary>
    public interface IAsyncCodeScopeExtension // await using statement
    {
        /// <summary>
        ///     Asynchronously executed by a language construct when the language construct loses its scope.
        /// </summary>
        /// <param name="exception">
        ///     Exception that was caused the scope loosing, or <see langword="null" /> for a normal execution
        ///     flow.
        /// </param>
        /// <returns>Async operation task.</returns>
        ValueTask OnLoseCodeScopeAsync(Exception? exception);
    }

Now we assume that using supports ICodeScopeExtension, and we can define our new abstraction, with totally different usage scenario not related to IDisposable.

Example: Explicit lifetime management abstraction

    /// ILifetimeExplicit proposed to be added to .Net Standard, as it very very common usage scenario. But it could be just in a popular nuget library (like Newtonsof.Json).

    /// <summary>
    ///     The object with the explicitly controlled lifetime.
    /// </summary>
    public interface ILifetimeExplicit: ICodeScopeExtension
    {
        /// <summary>
        ///     Gracefully finalizes lifetime of the object. Supports one and only one invocation.
        /// </summary>
        void Dispose(); 

        // C# 8.0 default interface methods behavior override, already supported.
        void ICodeScopeExtension.OnLoseCodeScope(Exception? ex)
        {
            Dispose();
        }
    }

    /// <summary>
    ///     The object with the explicitly controlled lifetime, suitable for asynchronous execution.
    /// </summary>
    public interface IAsyncLifetimeExplicit: IAsyncCodeScopeExtension
    {
        /// <summary>
        ///     Gracefully finalizes lifetime of the object. Supports one and only one invocation.
        /// </summary>
        ValueTask DisposeAsync();

        // C# 8.0 default interface methods behavior override, already supported.
        ValueTask IAsyncCodeScopeExtension.OnLoseCodeScopeAsync(Exception? exception)
        {
            return DisposeAsync();
        }
    }

    /// <summary>
    ///     The object with explicit lifetime management which also distinguish lifetime termination.
    /// </summary>
    public interface ILifetimeTerminable : ILifetimeExplicit
    {
        /// <summary>
        ///     Terminates lifetime. Supports one and only one invocation mutually exclusive with Dispose.
        /// </summary>
        /// <param name="reasonException">The exception caused lifetime termination.</param>
        void Terminate(Exception reasonException);

        // C# 8.0 default interface methods behavior override, already supported.
        void ICodeScopeExtension.OnLoseCodeScope(Exception? exception)
        {
            if (exception == null)
                Dispose();
            else
                Terminate(exception);
        }
    }

    /// <summary>
    ///     The object with explicit lifetime management which also distinguish lifetime termination and suitable for asynchronous execution.
    /// </summary>
    public interface IAsyncLifetimeTerminable : IAsyncLifetimeExplicit
    {
        /// <summary>
        ///     Terminates lifetime. Supports one and only one invocation mutually exclusive with Dispose.
        /// </summary>
        /// <param name="reasonException">The exception caused lifetime termination.</param>
        ValueTask TerminateAsync(Exception reasonException);

        // C# 8.0 default interface methods behavior override, already supported.
        ValueTask IAsyncCodeScopeExtension.OnLoseCodeScopeAsync(Exception? exception)
        {
            return exception == null ? DisposeAsync() : TerminateAsync(exception);
        }

    }

Explicit lifetime management abstraction usage demo

    public class XmlFileEditContext : ILifetimeTerminable
    {
        private readonly string _xmlFileName;
        public XmlDocument Xml { get; }
        
        public XmlFileEditContext(string xmlFileName)
        {
            _xmlFileName = xmlFileName;
            
            Xml = new XmlDocument();
            Xml.Load(xmlFileName);
        }

        public void Terminate(Exception reasonException)
        {
            // Some error occurred.
            // We needs only write trace message
            Trace.WriteLine($"Xml editing failed, error = {reasonException.Message}");
        }

        public void Dispose()
        {
            // This method is not about Unmanaged resources anymore.
            // IExplicitLifetime.Dispose especially designed to perform any actions at the end of lifetime.
            
            Xml.Save(_xmlFileName);
        }
    }

    public class Program
    {
        public static void Main()
        {
            using (var ctx = new XmlFileEditContext("my.xml"))
            {
                ctx.Xml.DocumentElement.InnerText = "Successful op";
                ctx.Xml.DocumentElement["UnknownNode"].InnerText = ""; // Raises exception, which passed to ctx.Terminate method implicitly by using operator.
            }
        }
    }

Should IDisposable implements ICodeScopeExtension ?

    /// Opened question: does IDisposable should be as a special case of ICodeScopeExtension + backward compatibility exceptional behavior????
    public interface IDisposable: ICodeScopeExtension
    {
        void Dispose();

        // C# 8.0 default interface methods behavior override, already supported.
        void ICodeScopeExtension.OnLoseCodeScope(Exception? ex)
        {
            Dispose();
        }
    }

If IDisposable will not inherited from ICodeScopeExtension and an object implement both

In this case ICodeScopeExtension have preference. Still big verification work required.

[dmitriyse]

You can play with the proposed abstractions and using statement Polyfill by downloading package: https://www.nuget.org/packages/ClrPro.DotNetLanguages.Proposals

[HaloFour]

Support Dispose, AsyncDispose pattern-matching on all classes/structs.

This was already discussed going into C# 8.0 and the team rejected it because it would silently change the behavior of the compiler when iterating via foreach as that statement automatically attempts to dispose of the iterator. Currently that requires that the iterator implement IDisposable, but with this change it would also pick up any accessible Dispose method. Personally I hope it's a conversation that can be reconsidered when the team digs further into shapes.

https://github.com/dotnet/csharplang/blob/master/meetings/2018/LDM-2018-12-12.md#when-we-start-recognizing-pattern-dispose-do-we-call-it-in-foreach

I think this proposal also has to work out the issues as to what happens when a type implements more than one of these interfaces and/or IDisposable. And why is ILifetimeExplicit necessary given that it's a copy of IDisposable? There has been argument that IDisposable does not represent an arbitrary unit of work, but we're 19 years into people using it for such and frankly it's not a situation that I think needs to be resolved, particularly through a combination of BCL and language changes.

[dmitriyse]

@HaloFour, thank you for your considerations and references. I have improved this proposal. Pattern-matching is no more mandatory thing for this proposal (but still possible). Problems that you specified has been worked out.

There has been argument that IDisposable does not represent an arbitrary unit of work, but we're 19 years into people using it for such and frankly it's not a situation that I think needs to be resolved, particularly through a combination of BCL and language changes.

Even if we stay IDisposable/using as is, documentation for it should be significantly improved and give support/best practices for more use cases.

Number of years saying nothing. 19 Years C# Language was without implicit scope using:

using var test = new MyDisposable();
var str = "asdfasdf";
....

16 Years BCL was without Task-like async methods, 19 years without DIM, etc.

Currently IDisposable/using solution lacks the ability to monitor scope lose exception, it's most limiting thing which is require BCL/language change.
C++, Ruby, Python have equivalent for IDisposable/usings with support of detect exception on exit and this feature is very natural.

The current version of proposal is very cheap in terms of implementation efforts by LDM Team, very close to implicit scope using.

Current proposal can help to write far more clear code bases and get rid from IDisposable semantic mess. Which every time eats your brain if you use IDisposable/using for something other than unmanaged resource disposal.

[dmitriyse]

There are already number of attempt to improve IDisposable/using #216 #213 #2948 #85 #1457, But now with C# DIM 8.0 we becomes much closer to resolve problems discussed in the issues above.

[HaloFour]

@dmitriyse

Even if we stay IDisposable/using as is, documentation for it should be significantly improved and give support/best practices for more use cases.

I think that modifying the documentation would be a lot easier than trying to change the patterns people use. I agree that it's probably worth expanding on the definition of "disposable". There have been a couple conversations about this already and seemingly the C# team themselves often can't agree on when and how to use IDisposable: #93 (comment)

My point about the number of years isn't about how long we've lived without a particular feature, it's about how much of an existing ecosystem there is. Lots of projects depend on IDisposable to represent a unit of work, including libraries from Microsoft such as Rx. And yes, many other features had workarounds prior to the feature being implemented, but those features also brought additional benefits. Like ValueTask<T> made it possible to avoid a memory allocation in common paths. Adding a new interface supported by using that doesn't also bring improved functionality only splits the ecosystem. It's unlikely that many projects would migrate over, particularly libraries where doing so would break consumers. And for those that do it represents a period of negative productivity as existing perfectly working code is updated to satisfy a stylistic preference. In my opinion I'd estimate that most projects would never migrate at all.

Note that I'm only arguing against adding an interface that duplicates IDisposable. I can see the utility in adding a flavor that supports accepting an exception.

[CyrusNajmabadi]

"lose" not "loose"

[CyrusNajmabadi]

This feature didn't seem to substantively do more than what is already possible today with IDisposable and using

[theunrepentantgeek]

I'm pretty sure the best possible outcome would be to update the documentation for IDisposable to cover the additional use case that's already in common use throughout the C# ecosystem.

Adding a new interface is effectively a non-starter because adoption would be low - most libraries wouldn't use it because of backward compatibility issues, most developers wouldn't use it because they wouldn't know about it, so it would be nothing more than a curiosity.

Extending the existing IDisposable interface with a OnException(Exception ex) method would be possible now that we have default interface methods - but there's an additional complication.

Since C#8, the using keyword has been pattern based, so a type only needs to implement void Dispose(), it doesn't have to implement IDisposable.

Having the compiler suddenly start automatically calling OnException(Exception ex) just because it existed on the type already would be a breaking change, regardless of the actual name or signature.

[yaakov-h]

@theunrepentantgeek are you sure about pattern-based using? I thought that was rejected except for ref struct and I still get error CS1674 if I attempt it.

[theunrepentantgeek]

Interesting.

My playing around with SharpLab only got as far as getting this error message:

error CS1674: 'C': type used in a using statement must be implicitly convertible 
to 'System.IDisposable' or implement a suitable 'Dispose' method.

which I thought proved it had been released.

However, switching the branch to master (from default) changes the message to

error CS1674: 'C': type used in a using statement must be implicitly convertible
to 'System.IDisposable'.

So something has definitely changed.

If pattern based disposal is truly dead, then we might be able to extend IDisposable using DIM.

[333fred]

Pattern-based is only used for ref structs. It was too much of a breaking change for us to consider for other types.

[HaloFour]

@333fred

Pattern-based is only used for ref structs. It was too much of a breaking change for us to consider for other types.

What other scenarios would "break" other than foreach and pattern-based matching on the enumerator?

[sharwell]

I(Async)Disposable designed (as stated in the official documentation) to free-up unmanaged resorces when they are no longer needed without deferring this to GC. Despite the official purpose, many frameworks adopts I(Async)Disposable for various other purposes, just to reuse using C# language construct - unofficial I(Async)Disposable use cases.

The design for these interfaces is based on a motivating scenario. Now that these types and the corresponding language features are defined with specific semantics, there is nothing stopping a user from writing code that needs the same semantics. In other words, unmanaged resources are one example use for these interfaces, but any number of other valid examples can exist.

The most popular unofficial use case is implicit lifetime management. For example ILogger.BeginScope returns IDisposable token that closes scope when Dispose invoked, frequently used withing using block.

This is already a valid use of IDisposable. It's not clear to me how/why a language change would improve clarity in this space.

[ajdepersio]

@sharwell

I believe that's why many are saying that the documentation around IDisposable should be updated to make clear that usages outside of releasing unmanaged resources are acceptable and not a misuse of the interface.

As things stand now, the following line in the IDisposable documentation causes purists to believe that the only usage of IDisposable should be for releasing unmanaged resources when in practice this is simply not the case.

The primary use of this interface is to release unmanaged resources.

[franchyd]

@ajdepersio Its actually this line here:

You should implement IDisposable only if your type uses unmanaged resources directly

[333fred]

@HaloFour those scenarios were enough.

[dmitriyse]

This is already a valid use of IDisposable. It's not clear to me how/why a language change would improve clarity in this space.

1. If we have multiple valid usages than we have N alternative classic IDisposable usage scenarios which best practices implementation should be defined somewhere. For some usages Dispose double call should rise an exception (which?) and Dispose/Finalizer should detect missing call to Dispose and give debug time assertion failure. So more conventions required, not only Unmanaged resources disposal. This topic requires full Article, the problem is more important than it might seem at first glance.
2. Missing the way to read exception on losing scope of the using operator. Again if we have other case than unmanaged resource disposal it's becomes very important to distinguish normal scope lose and exceptional scope lose. Some sort of OnException is impossible to add into compiled code, that's why extension to IDisposable is not a good idea until CLR-level interception support of IDisposable.Dispose call (still bad idea).
3. If we finally need some sort of OnException, than we needs IDisposable2 (COM nostalgia name), but if we need new interface why we need to bind its name to Unmanaged resources? We are free to re-bind its name to using-like operators (ICodeScopeExtension) and make it officially semantic free.
4. This proposal should solve many problems at once ( Add standard way to detect/handle exceptions raised from using(){} block. #216 Proposal: make ```using``` call ```Complete()``` on completing without an exception #213 Discussion: Using Statement And IDisposable #2948 Proposal: A clear and standard syntax for instrumentation #85 Language proposal: DSL-like method invokes and custom code blocks wrappers #1457)
5. Put like if you at least once asked google or stackoverflow "how to read exception at the Dispose method?" - big fraction of developers migrated to C# soon or later asking this question.

P.S.
Hypothetical IDisposable2 should not be inherited from IDisposable, otherwise it's totally the same case as adding new method to IDisposable.

[HaloFour]

@333fred

those scenarios were enough.

Seems like a scenario solved easily enough by requiring that pattern-based enumerators implement IDisposable to be disposed. Given that the type is specifically designed for the purposes of enumeration that doesn't seem remotely onerous, and the language already has a precedent for a language feature driven by the combination of pattern and interface: awaiters.

The ask is for pattern-based using, not for pattern-based disposal through foreach.

[TahirAhmadov]

I'd like to bump this discussion. It's often better to do:

void /*async Task*/ Foo()
{
  using var transactionScope = BeginScope();
  // do stuff
}

Than:

void /*async Task*/ Foo()
{
  using var transactionScope = BeginScope();
  // do stuff
  transactionScope.Commit();
}

Because the method may have more than return, which necessitates sprinkling the Commit call, which is error prone.

All we need here is:

try
{
  // main logic
}
finally(Exception? ex)
{
  // we know whether an exception is bubbling
}

Then:

interface IExceptionDisposable { void Dispose(Exception? ex); }
interface IAsyncExceptionDisposable { ValueTask DisposeAsync(Exception? ex); }

[quetzalcoatl]

However, catch is also needed, in case you actually want to really silence the exception. That happens.
With your proposal, assuming finally retains its meaning and just gets a view into the passing-by exception, there would be no way of doing this.

[TahirAhmadov]

Oh yea, you are free to also have as many catch blocks as you need.
If you want to handle the exception and not throw a new exception or re-throw the existing one, I imagine the finally block will get null for ex.

try
{
  // main logic
}
catch(IOException)
{
  throw;
}
catch(InvalidOperationException ex)
{
  throw new ApplicationException("Invalid operation: " + ex.Message);
}
catch(ArgumentException)
{
}
finally(Exception? ex)
{
  // we know whether an exception is bubbling
}

In the above example, if:

• "main logic" throws IOException, then finally gets IOException;
• "main logic" throws InvalidOperationException, then finally gets ApplicationException;
• "main logic" throws ArgumentException, then finally gets null.

[TahirAhmadov]

On the other hand, we can say that finally(Exception? ex) { ... } form is only allowed if there are no catch blocks, to not have to answer the question of re-throwing/ignoring exceptions.

[Ai-N3rd]

I would like to make a vote that this feature goes into C# 13