Proposal: Virtual Extension Methods

[dmitriyse]

This proposal adds to C# an ability to define virtual extension methods and override them.
Primary benefit of this proposal is ability to build advanced extensible Linq-like frameworks.
And even binary-compatible upgrade of existing Linq-like frameworks with extensibility features.

New syntax usage example:

// We even can upgrade linq with binary compatiblity
public static class Enumerable
{
    // ...

    public static virtual int Count<T>(this IEnumerable<T> items)
    {
        // for binary compatibility here should be original Linq code with some sort of type checks.

        int count = 0;
        foreach (var i in items)
        {
            count++;
        }
        return count;     
    }

    // ...
}

public class SmartList<T>: IEnumerable<T>
{
    private int m_count;
    
    // ...

    // Custom implementation of "int Enumerable.Count<T>(this IEnumerable<T>)"
    override int Enumerable.Count<T>()
    {
        return m_count;
    }

    // ...
}

C# 8.0+ will translate above example to this code (assuming that #52 already in the language):
(That's what CLR will see)

public static class Enumerable
{
     public static int Count<T>(IEnumerable<T> items)
     {
         // -----  Compiler generated -----
         if (items is Enumerable.__IExtension__Count<T> itemsEx)
         {
              return itemsEx.Count();
         }
         // -----------------------------------

         int count = 0;
         foreach (var i in items)
         {
             count++;
         }
         return count;     
     }
     
     public interface __IExtension__Count<T>: IEnumerable<T>
     {
         int Count()
         {
             // Body from Enumerable.Count but with small transformation.
             int count = 0;
             foreach (var i in this) // items variable is replaced by !this!
             {
                 count++;
             }
             return count;     
         }
     }
}

public class SmartList<T>: IEnumerable<T>, Enumerable.__IExtension__Count<T>
{
    private int m_count;

    // ...

    int Enumerable.__IExtension__Count<T>()
    {
        return m_count;
    }

   // ...

}

[orthoxerox]

How will this work across assemblies?

[dmitriyse]

Target Type assembly <---ref--- Extension method Assembly <---- ref ---- Target Type inheritor with extension method override assembly.
This is most distributed combination of code accross assemblies.

According to topic example:
assembly with IEnumerable <---ref--- assembly with Enumerable <---ref--- assembly with SmartList

[orthoxerox]

This is likely a dupe of my old issue #310, and as far as I know, you cannot reliably declare virtual extension methods across multiple assemblies. The whole group of methods must be defined in a single assembly or you have to use dynamic features to implement them.

[dmitriyse]

@orthoxerox I don't see "whole group of methods must be defined in a single assembly" limitation for this proposal. Please provide the case what exactly is impossible.

Both #1061 #310 have the same goals but with different implementation details.

[orthoxerox]

Okay, let me try to come up with an example:

1. Assembly A:

public class A
{  
}

2. Assembly B, references A:

public static class B
{
    public virtual void PrintA(this A a)
    {
        Console.WriteLine("This is A");
    }
}

3. Assembly C, references A:

public class C : A
{
}

4. Assembly D, references B,C

public static class D
{
    public override void PrintA(this C c)
    {
        Console.WriteLine("This is C");
    }
}

A a = new C();
a.PrintA(); //what will happen here?

[dmitriyse]

@orthoxerox, this proposal does not allows to override extension methods globally in a static class.
This code is not allowed:

public static class D
{
    public override void PrintA(this C c)
    {
        Console.WriteLine("This is C");
    }
}

The only place to override is in the C assembly.

public class C : A
{
    override void B.PrintA()
    {
         Console.WriteLine("This is C !!!");
    }
}

So the extension method cannot override another extension method.
The class to which extension method is applicable can override extension method.

[orthoxerox]

@dmitriyse Hm, if you are not allowed to declare overrides in an assembly other than the pseudo-receiver's one, then I guess it could work. C would implement the hidden interface from B etc. This doesn't solve the same use case as #310, though. And doesn't solve the use case of adding optimized LINQ methods to a third-party collection type.

[dmitriyse]

And doesn't solve the use case of adding optimized LINQ methods to a third-party collection type.

Third-party collections is exactly best place to provide optimized LINQ method implementation.
Why it should be defined somewhere else ?

[orthoxerox]

What if that library is no longer supported or you cannot upgrade to the version in which the developer implemented optimized methods due to other unrelated breaking changes?

[dmitriyse]

@orthoxerox

What if that library is no longer supported or you cannot upgrade to the version in which the developer implemented optimized methods due to other unrelated breaking changes?

Probably you wants too much from the CLR/Compiler. It's the same that you want's hack up any third party library method.

May be you need CLR level iterception. Or LINQ-like library should provide some hooks or extensibility points (the same approach as in ASP.Net Core.)

[jnm2]

Emits Enumerable.__IExtension__Count

Oh, my poor intellisense.

[yaakov-h]

@jnm2 [EditorBrowsable(EditorBrowsableState.Never)] - though the IDE would have to be updated to actually respect that within the current solution.

[jnm2]

Also: Oh, my poor generated API contract file.

[TonyValenti]

Hi All,
There are a few different ways to accomplish something like this. Here is an example showing how to create an "overridable" extension method. More info in next post.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApp4 {
    class Program {
        static void Main(string[] args) {

            Console.WriteLine("Before:");
            Console.WriteLine("Test".ToStringEx());
            Console.WriteLine(555.ToStringEx());
            Console.WriteLine();

            ExtensionMethods.ToStringExHandler = OverrideToStringEx;


            Console.WriteLine("After:");
            Console.WriteLine("Test".ToStringEx());
            Console.WriteLine(555.ToStringEx());
            Console.WriteLine();
        }


        private static string OverrideToStringEx(object This, ExtensionMethods.ToStringExInvoker Base) {
            if (This is long || This is int) {
                return $"[{This}]";
            } else {
                return Base(This);
            }
        }

    }


    public static class ExtensionMethods {

        public static string ToStringEx(this Object This) {
            return _ToStringEx_Invoker(This);
        }

        static ExtensionMethods() {
            ToStringExHandler = ToStringEx_Default;
        }

        private static ToStringExInvoker _ToStringEx_Invoker;

        public static ToStringExDelegate ToStringExHandler {
            set {
                if(value != null) {
                    var OldInvoker = _ToStringEx_Invoker;
                    _ToStringEx_Invoker = This => value(This, OldInvoker);
                }
            }
        }

        private static string ToStringEx_Default(this Object This, ToStringExInvoker Base) {
            return This?.ToString();
        }

        public delegate string ToStringExInvoker(Object This);
        public delegate string ToStringExDelegate(Object This, ToStringExInvoker Base);

        

    }

}

[TonyValenti]

Personally, I'm not really a fan of overriding extension methods that way. The way that I handle it is to essentially have a static event that my extension method calls and my event has a return value and a "handled" value that indicates whether the next item in the chain should be invoked. Here is a fairly large example showing an example (sorry about all the plumbling code). Also, this code was designed so that I could essentially do async event overriding.
In my code I have two main extension methods:

• InvokeEvent which invokes handlers in order from First to Last
• InvokeFunction which invokes handlers in order for Last to First

//Usage Example:
public static event AsyncEventHandler<Object, ConvertFilePathToDavPathEventArgs> ConvertFilePathToDavPathEvent;
        public static Task<string> ConvertFilePathToDavPath(string Path) {
            var Args = new ConvertFilePathToDavPathEventArgs {
                Path = Path,
                Result = Path,
            };

            return ConvertFilePathToDavPathEvent.InvokeFunction(this, Args, x=>x.Result);
        }

        public class ConvertFilePathToDavPathEventArgs : FunctionEventArgs<String> {
            public string Path { get; set; }
        }

//PLUMBING:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace System {
    public static class AsyncEvents {
        public const OnErrorAction OnErrorDefault = OnErrorAction.Throw;
        public const OnErrorAction OnErrorTryDefault = OnErrorAction.SilentlyContinue;
        public const OnHandledAction OnHandledDefault = OnHandledAction.Break;
        public const InvokeDirection InvokeDirectionDefault = InvokeDirection.FirstToLast;

        public static Task TryInvokeEvent<TSender, TEventArgs>(this AsyncEventHandler<TSender, TEventArgs> Delegate, TSender sender, TEventArgs e,
            OnErrorAction OnError = OnErrorTryDefault,
            OnHandledAction OnHandled = OnHandledDefault,
            InvokeDirection InvokeDirection = InvokeDirectionDefault    
            ) {
            return InvokeEvent(Delegate, sender, e,
                OnError,
                OnHandled,
                InvokeDirection
                );
        }

        public static Task<T> TryInvokeFunction<TSender, TEventArgs, T>(this AsyncEventHandler<TSender, TEventArgs> Delegate, TSender sender, TEventArgs e, Func<TEventArgs, T> Result,
            OnErrorAction OnError = OnErrorTryDefault,
            OnHandledAction OnHandled = OnHandledDefault,
            InvokeDirection InvokeDirection = InvokeDirectionDefault) where TEventArgs : FunctionEventArgs<T> {

            return InvokeFunction(Delegate, sender, e, Result,
                OnError, 
                OnHandled, 
                InvokeDirection
                );

        }

        public static async Task<T> InvokeFunction<TSender, TEventArgs, T>(this AsyncEventHandler<TSender, TEventArgs> Delegate, TSender sender, TEventArgs e, Func<TEventArgs, T> Result,
            OnErrorAction OnError = OnErrorDefault,
            OnHandledAction OnHandled = OnHandledDefault,
            InvokeDirection InvokeDirection = InvokeDirectionDefault) where TEventArgs : FunctionEventArgs<T> {
            await InvokeEvent(Delegate, sender, e,
                OnError: OnError,
                OnHandled: OnHandled,
                InvokeDirection: InvokeDirection
                ).DefaultAwait();

            return Result(e);
        }


        public static async Task InvokeEvent<TSender, TEventArgs>(this AsyncEventHandler<TSender, TEventArgs> Delegate, TSender sender, TEventArgs e,
            OnErrorAction OnError = OnErrorDefault,
            OnHandledAction OnHandled = OnHandledDefault,
            InvokeDirection InvokeDirection = InvokeDirectionDefault) {

            var Methods = (Delegate?.GetInvocationList()).EmptyIfNull();


            if (InvokeDirection == InvokeDirection.LastToFirst) {
                Methods = Methods.Reverse();
            }

            foreach (var item in Methods) {

                if (e is IHandledEventArgs IHandled && IHandled.Handled && OnHandled == OnHandledAction.Break) {
                    return;
                }

                if (item is AsyncEventHandler<TSender, TEventArgs> Method) {

                    try {
                        await Method.Invoke(sender, e)
                        .DefaultAwait()
                        ;
                    } catch (Exception ex) {
                        ex.Ignore();
                        if (OnError == OnErrorAction.Throw) {
                            throw;
                        }
                    }


                }
            }

        }

    }

    public delegate Task AsyncEventHandler<TSender, TEventArgs>(TSender sender, TEventArgs e);

    public enum OnErrorAction {
        SilentlyContinue,
        Throw,
    }

    public enum OnHandledAction {
        Break,
        Continue,
    }

    public enum InvokeDirection {
        FirstToLast,
        LastToFirst,
    }

}

    public class FunctionEventArgs<T> : HandledEventArgs, IFunctionEventArgs<T> {
        public T Result { get; set; }
    }

    public interface IFunctionEventArgs<T> {
        T Result { get; set; }
    }

    public class HandledEventArgs : IHandledEventArgs {
        public bool Handled { get; set; }
    }

    public interface IHandledEventArgs {
        bool Handled { get; set; }
    }