[Proposal]: Collection Expressions Next (C#13 and beyond)

[CyrusNajmabadi]

Collection Expressions Next

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: Not Started

Summary

This issue is intended to be the umbrella tracking item for all collection expression designs and work following the core design (#5354) that shipped in C#12.

As this is likely to be a large item with many constituent parts, it will link out to respective discussions and designs as they occur.

Roughly, here are the items we would like to consider, as well as early notes on the topic: https://github.com/dotnet/csharplang/blob/main/meetings/working-groups/collection-literals/CL-2024-01-23.md

1. Dictionary expressions. [Proposal]: Dictionary expressions #7822
2. Natural type
3. Immediately enumerated collections. e.g. ["a", .. b ? ["c"] : []] and foreach (bool? b in [true, false, null]). Collection expressions: inline collections in spreads and foreach #7864
4. Extension methods on collections
5. Missing collection-creatable types (e.g. Memory<T>, ArraySegment<T> etc.)
6. Supporting non-generic core collections (IEnumerable, etc.)
7. Relaxing restrictions. Open issue: relax requirement that type be enumerable to participate in collection expressions #7744

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-01-08.md - Iteration types of collections
https://github.com/dotnet/csharplang/blob/main/meetings/working-groups/collection-literals/CL-2024-01-23.md - WG meetup
https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-01-10.md - Conversions vs construction
https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-02-05.md#collection-expressions-inline-collections
https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-09-06.md#collection-expressions-next-c13-and-beyond

[KennethHoff]

I'd like to see some way of calling the constructor of a type and/or setting some property during initialization. The first thing that came to mind is the comparer of a dictionary, but I'm sure there are other use-cases.

Either way, collection expressions are amazing, and support for natural types and inline expressions would make them that little bit better!

[CyrusNajmabadi]

@KennethHoff that's in the list, as part of hte dictionary-expression exploration work. Thanks :)

[BlinD-HuNTeR]

I'm not sure if the following proposal is too crazy, so I will describe it here quickly, as it's related to this topic:

Imagine that I have a int[] ints = [1, 2, 3], and I would like to have a string[] with the representations of those ints.
One way to do that would be string[] strings = [ints[0].ToString(), ints[1].ToString(), ints[2].ToString()], but that only works because I know the length.

Ideally I would like to spread the array of ints into the array of strings while also calling ToString on each element.

If int had an user-defined implicit conversion operator to string, then I could write string[] strings = [.. ints], and it would be valid C# code, invoking the op_Implicit method on each int.

But since there is no such conversion, I would like to write something like string[] strings = [ (..ints).ToString()], and it would do exactly that. Hence, the proposal:

Spread operator as a first-class citizen.

So basically the spread operator .. becomes a regular unary-operator of the C# language, which at first may only be used inside collection expressions, but possibly could have its use expanded to everywhere.

The spread unary operator may be applied to any other expression, resulting in a spread_expression. When that happens, an implicit foreach loop is inserted for expression following the usual rules. It is, therefore, a compile-error if the operator is applied to something that isn't enumerable.

The spread_expression then behaves, for the purposes of member-lookup and overload-resolution, as an expression whose type is the element-type of the original enumerable. Because of that, one may invoke any members the element type might have, as well use the spread_expression on a method that takes element-type as argument. These invocations will then be inserted inside the invisible foreach loop, and spread_expression will be substituted for the loop variable.

The result of a member invocation performed on, or taking the spread_expression as an argument, is also itself a spread_expression, whose element-type is the return type of the invoked member, if not void. Method invocations can be chained on a spread_expression , and they will all be moved to the inside of the foreach loop.

Finally, one will want to capture the result of all these method invocations on each element of the original collection. Therefore, any spread_expression can be used regularly as a spread_element in a collection expression, with the existing rules.

Problems:

• There is a known ambiguity between spread operator and range operator. Originally this was addressed by forcing range operator to be inside parentheses, but with this proposal, it will also be necessary to put the spread inside parentheses when invoking members of the element-type. This is not an impeditive per se, because range operator may only be applied to int or System.Index, which are not (usually) enumerable, but the complexity of parsing would increase.

[HaloFour]

@BlinD-HuNTeR

C# already has a query comprehension syntax, LINQ.

[BlinD-HuNTeR]

C# already has a query comprehension syntax, LINQ.

True, but, that's not really an argument. If you would argue against all proposals saying "it can already be done in some way", then none would ever be accepted. Lists could already be created with LINQ or with initializers, yet collection expressions were introduced. And they have the added benefits of duck typing/nice syntax/good performance.

LINQ, on the other hand, is interface-based and makes use of delegates and anonymous objects. If one could perform some simple transformations through the use of spread operator, everything would be inserted directly in the caller method, with no delegates or closures. There isn't any optimization better than that, LINQ would almost be obsolete.

[HaloFour]

@BlinD-HuNTeR

See: #7634

[CyrusNajmabadi]

@BlinD-HuNTeR

Imagine that I have a int[] ints = [1, 2, 3], and I would like to have a string[] with the representations of those ints.

That's where supporting extension methods would help. As you could write:

[1, 2, 3].Select(i => i.ToString()).ToArray()

[KennethHoff]

[1, 2, 3].Select(i => i.ToString()).ToArray()

I assume you could also do this?

[ ..[1, 2, 3].Select(i => i.ToString()) ]

[CyrusNajmabadi]

@KennethHoff yes.

[jnm2]

@KennethHoff Yes, or [.. from i in [1, 2, 3] select i.ToString()]

[En3Tho]

I suggest widening support for collection builders to accept other buildable collections.

For example I can choose to wrap a HashSet or a List or an Array or whatever. It feels so weird to accept a Span for such types.

[CyrusNajmabadi]

@En3Tho Can you give an example?

[En3Tho]

@CyrusNajmabadi

public class ArrayWrapperBuilder
{
    // This works
    public static ArrayWrapper<T> Create<T>(ReadOnlySpan<T> values)
    {
        return new(values.ToArray());
    }

    // I want this to work instead. Array is already a creatable collection, so just let compiler create it and use directly here
    // Imagine if it was a List<T> or HashSet<T> wrapper. Even more allocations while compiler is perfectly able to create this kind of collection directly.
    public static ArrayWrapper<T> Create<T>(T[] values)
    {
        return new(values);
    }
}

[CollectionBuilder(typeof(ArrayWrapperBuilder), nameof(ArrayWrapperBuilder.Create))]
public readonly struct ArrayWrapper<T>(T[] array) : IEnumerable<T>
{
    public T[] Array => array;
    public IEnumerator<T> GetEnumerator()
    {
        throw new NotImplementedException();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

public static class ArrayWrapperCreator
{
    public static ArrayWrapper<T> GetWrapper<T>() => [default, default, default];
}

[koszeggy]

I'm glad the lack of foreach support is covered in the plans.

Something like

foreach (int i in [1, 2, 3])

was literally the first thing I tried with collection literals and I was surprised why it was failing to compile with CS9176 saying there was no target type. It basically provides exactly the same amount of information as

foreach (var i in (IEnumerable<int>)[1, 2, 3])

which happily compiles today (though I would rather use some old-school array initialization instead of the bulky cast).

I think this could be added even independently from natural types.

[eiriktsarpalis]

One addition that we would like to see is support for multi-dimensional collection literals. These are important when working with tensor libraries, e.g. here's how simple tensors are defined using the pytorch library:

data = [[1, 2],[3, 4]]
x_data = torch.tensor(data)

The above form isn't feasible using today's C# collection literals, so supporting it without some kind of language support seems unlikely.

One possibility is that we could reuse nested collection literal syntax to construct multi-dimensional collections. TL;DR it should be possible to extend the collection builder pattern to recognize factory methods such as

public static T[,] Create<T>(ReadOnlySpan<T> values, int nDim, int mDim);

and then be able to define 2-D arrays like so

T[,] values = [[1, 0], [0, 1]];

In principle, it should be possible for the compiler to infer the rank and dimensions and detect shape mismatches (e.g. something like [[1], [2, 3]]).

What's more interesting though is that by reusing nested collection syntax there is no inherent limit on the supported number of dimensions, and the number of dimensions doesn't need to be fixed for a given type. We could for instance support builder methods of shape

public static Tensor<T> Create<T>(ReadOnlySpan<T> values, params ReadOnlySpan<int> dimensions);

Which should let you specify the following 2x2x2 tensor:

Tensor<int> tensor = [[[0, 0], [0, 0]], [[1, 0], [0, 1]]];