[Proposal]: Improve binding of spread elements in Collection Expressions

[BlinD-HuNTeR]

Collection expressions have the ability to spread the elements of any object that implements IEnumerable or IEnumerable<T> into the collection. However, many old collection types from BCL do not implement the generic IEnumerable<T>, and because of that, they're recognized as object collections, and can't be spread into collections of their specific type.

The idea is that, when encountering a spread element, the compiler could check for the presence of an indexer, in addition to the Length/Count property that it already checks, and if both are present, then it iterates the collection using the indexer, instead of using GetEnumerator/MoveNext. Also it would recognize the type of the spread element as the return type of the indexer, instead of object.

[CyrusNajmabadi]

Just do .Cast<T> on the collection. Supporting untyped collections (which are now 20 years old) is a non-goal (esp when the solution is easy to make it typed). Thanks!

[CyrusNajmabadi]

Put another way. We didn't do any support for this when we did linq (and that was much closer in time to the non-generic collections). The answer even then was .Cast. So we're def not changing things now, when the cat majority of the ecosystem is on modern
.Net

[BlinD-HuNTeR]

@CyrusNajmabadi This isn't only about the old collections, but it would also be good for any type that exposes an indexer, as iterating with indexer is usually better and doesn't allocate an IEnumerator.

Just do .Cast<T> on the collection. Supporting untyped collections (which are now 20 years old) is a non-goal (esp when the solution is easy to make it typed). Thanks!

When you do that, you lose the "known length" of the collection expression.

[CyrusNajmabadi]

but it would also be good for any type that exposes an indexer, as iterating with indexer is usually better

I don't see why that would be the case. An enumerator has private access to the contents of the collection.

When you do that, you lose the "known length" of the collection expression.

You shouldn't. .Cast should preserve Count information.

[jnm2]

You shouldn't. .Cast should preserve Count information.

This really surprised me, so I checked and it does not:

using System.Collections;

// Prints: False
Console.WriteLine(new ArrayList { 1, 2, 3 }.Cast<int>().TryGetNonEnumeratedCount(out _));

https://github.com/dotnet/runtime/blob/v8.0.0/src/libraries/System.Linq/src/System/Linq/Cast.cs#L32-L57:

        public static IEnumerable<
#nullable disable // there's no way to annotate the connection of the nullability of TResult to that of the source
                TResult
#nullable restore
                > Cast<TResult>(this IEnumerable source)
        {
            if (source is IEnumerable<TResult> typedSource)
            {
                return typedSource;
            }


            if (source == null)
            {
                ThrowHelper.ThrowArgumentNullException(ExceptionArgument.source);
            }


            return CastIterator<TResult>(source);
        }


        private static IEnumerable<TResult> CastIterator<TResult>(IEnumerable source)
        {
            foreach (object obj in source)
            {
                yield return (TResult)obj;
            }
        }

Maybe it would be possible if Cast<T>() returned an object implementing both IEnumerable<T> and IIListProvider<T>, but we'd need runtime input on that.

[CyrusNajmabadi]

is usually better

As mentioned, it may or it may not be. We're not going to invest in heuristics that can just as easily make things worse. As Linq itself didn't invest here, and that's been fine for literally 20 years, we're not going to start now :-)

[jnm2]

This suggestion seems good to me. I remember this coming up a while ago. It would bring more consistency to the language. Pattern matching already has this type of recognition of the element type of a nongeneric collection, and it works via the indexer property type exactly the way you're describing:

using System.Collections;

if (new C() is [var x])
{
    x.ThisIsStronglyTypedViaTheIndexer();
}

class ElementType
{
    public void ThisIsStronglyTypedViaTheIndexer() { }
}

class C : IEnumerable
{
    public int Count => throw new NotImplementedException();

    public ElementType this[int index] => throw new NotImplementedException();

    public IEnumerator GetEnumerator() => throw new NotImplementedException();
}

What I'm not sure about is the priority of supporting nongeneric collections, but this type of feedback is exactly what we need to get a sense of the need.

[CyrusNajmabadi]

I thought it would be good to flesh out some of my previous off-the-cuff comments. I can definitely see where this idea is coming from, and the sorts of scenarios it wants to enable. That said, this runs into a common design question of how to break problems up, and how to best align those constituent parts with the language as is, and where we want the language to be.

So, when it comes to collection expressions, spreads, and untyped IEnumerables, there is a lot of overlap with many parts of the language, and many of the design choices we've aligned with over the years.

First (and to me, foremost), we took a fairly strong stance on the design group that spreads should behave very similar to foreachs. Specifically, if you were spreading a collection, it should be intuited as equivalent to foreaching that collection and adding the individual elements one at a time to the final collection. We felt this so strongly that much of the design and thinking around this feature space revolved around the idea of the 'iteration type' (the type you get when you foreach over a collection).

So, in a very real sense, adding such a 'bridging' feature would be taking us away from that intuition and design alignment. We would now be saying "you should think of this as akin to foreach except for untyped collections, where it now follows an entirely different rule altogether). This would also then cause these IEnumerables to behave differently from many other lang features. For example, as mentioned above, they would work for spreading in a collection expression, but not for use in a linq-query.

So, to me, my view here is that it's not desirable to approach this from a direction of trying to special case these collections solely for collection expressions. Instead, my personal view is that if we were to tackle this space, we'd want to do so in a way that allowed them to gain better functionality in a consistent way across the language. So, for this sort of feature, instead of saying that a collection-expr could spread these in a certain way, it would be more desirable to just say "all collections like this are iterable in this fashion". If we did that, then something like a collection-expression would more 'fall out'/'light up'.

Now, that said, i still land on the position that i think the existing solutions to this space are sufficient. There are two that i can think of right off the bat:

1. Use .Cast<> on the untyped enumerable to allow spreading to now work on the typed IEnumerable<T>. This is the general guidance we've continually given for this space with Linq (which is closely related to collection-exprs).
2. Add an extension GetEnumerator for these types. That would allow alloc-free, no-overhead, iterability of these untyped collections. This would work well, and would align above with the design goals of spread (which itself was to align with foreach). The extension GetEnumerator would light up these collections for both scenarios efficiently.

Thanks for the feedback! We'll definitely roll this into our future thinking in this space. But, of course, we can't promise anything will necessarily be done here specifically.

[jnm2]

Extension GetEnumerator doesn't win over the nongeneric collection type's own public instance GetEnumerator which returns a nongeneric enumerator. (SharpLab)

An alternative might be a wrapper class implementing IEnumerable<T> with a Count property so that the collection being created can be presized.
[.. nongenericCollection.AsTyped<T>()]

[jnm2]

Here's a working example, showing that the collection expression reads the count to presize the collection: SharpLab

[BlinD-HuNTeR]

I understand that design decisions were made to have spread behave the same as foreach. I even tested it on types that don't implement IEnumerable at all, and it works with duck-typing as long as there is a GetEnumerator method in scope.

However I also remember from some discussions before C# 12 was released, there was an alignment that collection expressions' features should be in parity with those of pattern matching. And since pattern matching already works with indexers that way, I think it would be nice to have that on spread too.

Plus, there really isn't any good alternative to this matter. Using .Cast<> loses the known length, and extension GetEnumerator doesn't win over an existing GetEnumerator method on the type.

[BlinD-HuNTeR]

An alternative might be a wrapper class implementing IEnumerable<T> with a Count property so that the collection being created can be presized. [.. nongenericCollection.AsTyped<T>()]

You don't need to use a different name, you can name the method Cast<T> and it will be preferred over the built-in Cast method when the receiver is ICollection.

[CyrusNajmabadi]

there was an alignment that collection expressions' features should be in parity with those of pattern matching

That's fair. Though that's more about the collection itself, and not spreads. For spreads we took a broader stance of aligning with foreach (which has much looser restrictions, and which also brings the untyped view of non-generic IEnumerable)

Plus, there really isn't any good alternative to this matter. Using .Cast<> loses the known length

So, when examining this situation we'd have to ask ourselves: how bad is that?

Given that this is all for net1.0 collections, we're already in an area where investment in squeezing out perf is not really a goal for any team or group.

Sometimes you take the "good enough" solution that exists that and accept that that may be the extent of the investment in the area.

Add I mentioned above though, we'll add this to the list to look at. However, based on the convos so far about legacy types, I would not get hopes up on getting support here. As another example, we considered, and explicitly rejected, the ability to construct an IList (non-generic) collection from a collection expression.

Thanks!