Rethinking shape syntax.

[YairHalberstadt]

This proposal tries to take a fresh look at the syntax suggested for shapes (#164), and suggests that an alternative syntax might allow us to express what we're trying to express more directly.

First we'll look at three examples to see why I feel the current syntax isn't quite right.

Specimen 1. SGroup

This is how the current syntax for groups (monoids) looks:

public shape SGroup<T> 
{ 
    static T operator +(T t1, T t2); 
    static T Zero { get; } 
}

What's that T meant to represent? Well it's meant to represent whatever type implements the group.

Unfortunately it doesn't actually work, since I can use any type for T. I can partially remedy this, by changing the signature to:

public shape SGroup<T> where T : SGroup<T>
{ 
    static T operator +(T t1, T t2); 
    static T Zero { get; } 
}

However, you've now made the signature more complicated, and it's still possible for T to be any type that implements SGroup, not necessarily the type that you're currently implementing SGroup with. For example this is perfectly fine:

public class Foo : SGroup<Foo> {...}
public class Bar : SGroup<Foo> {...}

Consuming SGroup also is a bit uglier than it needs to be:

public T Add<T>(IEnumerable<T> source) where T : SGroup<T>

Why should I have to specify where T : SGroup<T>?

Why isn't it enough to just specify where T : SGroup?

All of these are not problems, in the sense that they don't make life significantly more difficult for the programmer. What they do do however, is point to the fact that we're not actually expressing what we want to express, and are forced to use a workaround.

Specimen 2. Constructors and casts

How would we indicate that a shape requires a constructor to be implemented?

Currently there's no obvious way. We would have to make up some new syntax to indicate this, like using the name of the group, or the keyword new.

public shape SConstructable
{
    // public SConstructable(); ?
    // public new(); ?
}

How would we express that whatever implements a shape is explicitly/implicitly castable to int? Again there's no obvious syntax

specimen 3. HKTs

What if C# wanted to add the concept of higher kinder types at some point?

For example, what if we wanted to express that a type that implements SFunctor, has to be generic over all T?

Currently there's no obvious way syntax wise to this. Some entirely new syntax would have to be made up, like a <> constraint. Any such syntax clearly looks like a hack. I've also played around writing code with many ideas, and none of them tend to work well in practice.

Eg.

public shape SFunctor<TFunctor, T> : TFunctor<T> where TFunctor : <>
{
    TFunctor<S> Map<S>(Func<T, S> func);
}

Then we would need to make up extra syntax if it it only needed to be generic over certain types, or could have other generic constraints. Or how would we say that for any type parameter T, the generic type must implement ICollection<T>? where TFunctor : <>, ICollection<>?

The Problem

IMO the problem here is that we're modelling the syntax of shapes very closely on interfaces, but interfaces express a very different idea to shapes. This leads to increasing complexity when we try to use shapes to do anything very different to how interfaces are currently used.

interfaces simply express that any type that implements an interface has to have this bundle of methods. It tells you nothing about what the type that implements the interface looks like.

On the contrary, shapes are precisely about what the class that implements the shape looks like. They're trying to express that any type which implements a specific shape, has the same 'shape' as the shape.

As such, I believe we should express that directly, by including in the definition of a shape, an extra identifier that represents the type that implements the shape. For example, this is how all the above specimens would look:

Specimen 1. Revisited

public shape SGroup T
{
    static T operator +(T t1, T t2); 
    static T Zero { get; } 
}

Here T is a placeholder for the type that implements the shape. It's saying that to match SGroup, you would have to be able to replace T with your type, and have all those methods implemented. We've expressed what we wanted directly.

implementing SGroup is non-generic:

public Foo : SGroup
{
    static Foo operator +(Foo t1, Foo t2); 
    static Foo Zero { get; } 
}

Consuming SGroup is now:

public T Add<T>(IEnumerable<T> source) where T : SGroup

As desired. We now no longer make SGroup needlessly generic

Specimen 2. Revisited.

It's now obvious how to implement constructors and casts:

public shape SConstructable T
{
    public T();
}

public shape SCastable T
{
    public implicit operator int(T t);
}

Specimen 3. Revisited

HKT's are where this syntax really comes into it's own. Specifying that a type that implements a shape must be generic is now trivial:

public shape SFunctor TFunctor<T>
{
    TFunctor<S> Map<S>(Func<T, S> func);
}

note that public shape SFunctor<T> TFunctor and public shape SFunctor : TFunctor<T> mean very different things. The first means it is possible to implement SFunctor<T> for any T. The second means, Anything that implements SFunctor must be generic over T.

It's also obvious how you would express that it only needs to be generic over a subset of types. For example lets say SConvertable represents a type which contains numbers (like a matrix), and it has to be possible to convert the contained numbers from one numeric representation (like doubles) to another (like ints):

public shape SConvertable TConvertable<T> where T : SNumeric
{
    TConvertable<S> Convert<S>() where S : SNumeric
}

Pretty much everything you would want to say becomes straightforward. For example, to say that for any T, a type implements ICollection is simply:

public shape SCollection TCollection<T> : ICollection<T>
{
}

Conclusion

The current syntax for shapes feels like it hasn't broken free from it's interface roots. As such, it means that it's not powerful enough to express a wide range of abstractions, without increasing complexity, and constant additions of new syntax.

By rethinking the syntax slightly, we are able to directly express concepts which are impossible with the current syntax.

Notes

In some cases, we don't need a type placeholder expressing the current type. In such cases, it would be optional:

public shape SEnumerable<T>
{
    public SEnumerator<T> GetEnumerator();
}

[YairHalberstadt]

@MadsTorgersen
I know you've been working on shapes / roles and extensions. I wonder what you think of this?
Thanks

[orthoxerox]

I sense 🚲shedding! :)

According to the latest Build, shapes might even be an extension to interfaces:

interface IMonoid<T>
{
    static T operator +(T t1, T t2); 
    static T Zero { get; } 
    bool IsZero();
}

For HKTs, I think the question of semantics and runtime support is much more important than the syntax. If shapes will be interfaces, I think either a This type or something like IFunctor<F, T> where F : <>, IFunctor<F, _> will work. CRTPs are confusing, but I doubt many people will write HKTs for a living.

[YairHalberstadt]

@orthoxerox

I don't think this is bike shedding.

Syntax fundamentally changes how you interact with a feature, and it's worth exploring different options fully.

One of the purposes of this issue, is to highlight why I don't think shapes should be extensions to interfaces. I think that will actually make C# more complex in the long run, rather than less complex.

[Joe4evr]

For a new feature, I think creating a new syntax like this should definitely be something to consider. Shapes are (likely) already going to get the compiler-machinery to generate methods with a "hidden" type parameter, it wouldn't even need to be that much more trouble to utilize the same stuff to hide the type parameter of the Shape type at the places shapes are used.

[FiniteReality]

I personally don't mind SGroup<T> syntax, as long as consuming it doesn't require we fill in the generic type params during consumption. Maybe something generic type inference can solve?

Otherwise, I would prefer shape SGroup of T instead of Shape SGroup T, as it (IMO) flows better.

[YairHalberstadt]

I would prefer shape SGroup of T instead of Shape SGroup T, as it (IMO) flows better.

I'm not sure I understand this syntactically. How is this an SGroup of T? What's the 'of' relationship between SGroup and a T?

[FiniteReality]

@YairHalberstadt A group applies to any type exposing a particular shape, correct? The train of thought was somewhat similar to a for...of syntax, where 'T' would be the set of types meeting the requirement to "implement" the shape.

[YairHalberstadt]

So "for any T of shape SGroup, T has these properties"?

What worries me about that is that of quite a different meaning in VB. List(Of T) in VB means List<T> in C#. I'm worried that might lead to confusion.

Either way, deciding on SGroup of T or SGroup T at this stage is definitely bikeshedding. I'm sure the LDC would consider the exact keyword to use when they come round to it.

[FiniteReality]

Yes, that was kind of the idea, and I suppose you are correct that it may cause confusion. I personally just thought it was weird that we were all of a sudden going to have two types in a row in source code - I can't really think of any case in C# where that's ever happened.

Yes, you are definitely correct in that it's probably bikeshedding.

[yaakov-h]

Why do we still need T? Why can't we eliminate it entirely in favour of the shape itself, i.e.:

public shape SConstructable
{
    public SConstructable();
}

public shape SCastable
{
    public implicit operator int(SCastable t);
}

[theunrepentantgeek]

This entire thread has me substantially confused - I thought I understood the Shapes proposal, both syntax and possible implementation. Now I'm not so sure.

@YairHalberstadt writes:

What's that T meant to represent? Well it's meant to represent whatever type implements the group.

😦 I thought the T was simply (and only) a generic type parameter, allowing the type of the shape to vary in a similar exactly the same way that generic type parameters allow existing types to vary.

So, I think we could define a shape representing a factory like this:

public shape SFactory<T>
{
    T Create();
}

The type T is just the type of the items that can be created by the factory.

The actual type implementing conforming to the shape isn't declared here, not anywhere.

@YairHalberstadt wrote:

IMO the problem here is that we're modelling the syntax of shapes very closely on interfaces, but interfaces express a very different idea to shapes.

I disagree - I think shapes are a very similar idea to interfaces:

An interface declares a contract that can be explicitly implemented by any type.

A shape declares a contract that can be implicitly implemented by any type.

To implement an interface, the type itself needs to explicitly provide an implementation, conforming to the specifications of the interface, both syntactically and semantically.

To implement a shape, anything the type itself doesn't already provide can be provided by other code in the project, allowing it to confirm to the specifications of the shape, both syntactially and semantically.

@FiniteReality writes:

I personally don't mind SGroup syntax, as long as consuming it doesn't require we fill in the generic type params during consumption.

Based on other discussions about Shapes, I'd would expect this method declaration to work:

public void Demonstratify<T>(SFactory<T> factory)
{
    var item = factory.Create();
    // ...
}

When calling this method, you'd only need to provide the generic type parameters explicitly if the compiler was unable to work them out for itself - mirroring the situation with existing generic types.

At least, that's what I'd expect. 😀

[TonyValenti]

I really like some of the thoughts listed here and it got me thinking about shapes and HKTs. I think the proposed syntax is a bit confusing. I would prefer to see something like this:

//This is a regular generic shape.
public shape MyShape<T> {
   T Property {get; set;}
}

//This is a generic shape that references the implementing type.
public shape<TImplementor> MyShape<T> {
  public T Property {get; set;} 

  public TImplementor(); //The shape must also have a default constructor
  public TImplementor(T Parameter); //And a constructor that takes a T.
}

[YairHalberstadt]

Why do we still need T? Why can't we eliminate it entirely in favour of the shape itself, i.e.:

Because the name of the shape itself has meaning. I need to distinguish between

public shape SShape T
{
    SShape { get; } // returns any SShape
    T { get; } // returns the type that implements the shape
}

In the case of a constructor, you probably could replace T with the name of the shape, but I feel it's confusing that SShape would be given these two meanings. In a constructor it's standing in for the name of the implementing type, elsewhere it means any implementation of that shape.

Similarly in casts you want to be clear that this is a cast from the implementing type, and not suggest that this is a cast from any instance of the shape.

[YairHalberstadt]

@theunrepentantgeek

So, I think we could define a shape representing a factory like this:

public shape SFactory<T>
{
    T Create();
}

The type T is just the type of the items that can be created by the factory.

The actual type conforming to the shape isn't declared here, not anywhere.

That's exactly my point. Both SFactory<T> and SGroup<T> have the same syntax, but means very different things. SFactory<T> means this is a generic shape, which can be implemented for any T.

SGroup is not generic. You are never required to use it generically, other than as a result of a limitation of the type system. Instead the T there is a trick used in order to allo the shape to refer to the type that implements it.

This proposal is about allowing you to distinguish between

shape SFactory<T> // can be implemented for any T
shape SGroup T // any Type that implements SGroup must look like T

which you currently cannot do.

[YairHalberstadt]

I disagree - I think shapes are a very similar idea to interfaces:

I think that shapes have many purposes, as pointed out by @gafter here: #164 (comment)

The reason why I'm looking at shapes this way is because shapes allow you to define things like static operators which are about the type that implements the shape, rather than a bunch of methods.

TBH I'm just realizing that I would be ok with interfaces being allowed static operators, if they were to also introduce the syntax I've suggested here. I would be fine with interface IGroup T, just not with interface IGroup<T>

[Richiban]

@YairHalberstadt, @TonyValenti

I'm pleased to see others asking for a syntax for referring to the implementing type within a shape, similar to how Haskell typeclasses work and Rust's traits.

It might finally allow us some alternative to repeating instances of the massive hack that was interface IEquatable<T> and friends.

This syntax has stuck in my mind for some time now:

public shape SEquatableList<TItem> as TSelf
{
    IEnumerator<TItem> GetEnumerator();
    bool Equals(TSelf other);
}

But, since the as TSelf syntax allows one to introduce a type variable that always refers to the implementing type, is there any benefit to being able to give it a name? Why not, simply:

public shape SEquatableList<TItem>
{
    IEnumerator<TItem> GetEnumerator();
    bool Equals(this other);
}

[YairHalberstadt]

@Richiban
I think the advantage of an explicit name rather than an implicit keyword is that it helps for HKTs, but it could be that it would be best to cross that hurdle when we come to it.

[FiniteReality]

At the cost of sounding like more bikeshedding, I think I prefer the explicit TSelf over this as, to me, this could refer to either the shape (SEquatableList<TItem> for example) or the implementor.

Of course, as Yair mentioned, using an explicit name also helps for HKTs, which I think would be a huge use-case for shapes.

[TheUnlocked]

Expanding on @TonyValenti, you could possibly also do things like this:

public shape<T> SAddable where T : Thingy { // Only "Thingy"s can fit in this shape 
    static T operator +(T t1, T t2);
}

[TonyValenti]

Yes! Shapes should have generics at two levels.

Zero or one generics right after shape. If one is provided, then you can reference the implementing type.
Ie:

public shape foo { } //cannot reference implementing type
public shape<T> foo { } //T is the implementing type
public shape<T,U> foo { } //error

Zero or more after the name of the shape which makes the shape generic.

public shape foo { } // basically the same as public interface foo { }
public shape foo<T,U,V> { } // basically the same as public interface foo<T,U,V> { }

//It must have a constructor that accepts T, U and V and must implement enumerable<t>.
public shape<TSelf> Foo<T,U,V> : IEnumerable<T> {
  public TSelf(T, U, V);
}

Perhaps shapes really should just be Interfaces V2? It would simply expand on the interface syntax and I don't think there would be any problems with it.

[HaloFour]

@TonyValenti

Perhaps shapes really should just be Interfaces V2?

I get the sense that might be the direction that the team is interested in going. Having a new type called "shapes" means that you have to build an entirely new ecosystem and API around them. But if shapes are interfaces, and types can implement these interfaces with interfaces via witnesses, then all of the ecosystem is already shape-enabled. But I think that runtime changes might be needed to accomplish that with zero cost.

[yaakov-h]

@HaloFour They seem to be a bit more open to runtime changes now, though, particularly with the roadmap for .NET 5 and the announcement that .NET Framework will not be getting new features.

It could happen.

[canton7]

Adding to the bikeshedding...

Personally I'm not a fan of shape<TSelf> SGroup. Here, TSelf is something important - it refers to the type that implements the shape. However by letting the user control the name of TSelf, it means that you're never going to get any sort of consistency. When reading a shape, you're always going to have to keep referring to the first line of the shape to see what name this particular person has chosen instead of TSelf, and what are actual generic type parameters.

I'd propose taking a leaf out of Rust's book, and using the special keyword Self to refer to the type that implements the shape.

1. You don't have the difference between shape SGroup and shape<TSelf> SGroup -- it's always shape SGroup, and Self may or may not be used in the shape.
2. The user doesn't get to choose the name of TSelf -- it's always Self.

For HKTs, you could use something similar to previous proposals:

public shape<T> SFunctor
{
    Self<S> Map<S>(Func<Self<T>, Self<S>> func);
}

Here, shape<T> means that "the type implementing the shape must be generic with arity 1, and we'll call its single generic type parameter T". If you squint at shape<T>, you can mentally replace shape with the name of whatever type implements the shape, i.e. Foo<T> can implement shape<T>; Foo<T1, T2> can implement shape<T1, T2>, etc. It also means that Self is considered generic, and must always be given generic type parameters (i.e. in shape<T> SFunctor it's legal to say Self<T> or Self<S>, but not Self).

So, shape SFoo means "the type implementing this can be non-generic, or can be generic with any arity" while shape<T> SFoo means "the type implementing this must be generic with arity 1".

This is more wordy than @YairHalberstadt's proposal, but to me it reads a bit better. It says "func is a delegate which takes (the type which implements this shape), an returns (the same type, but with a different generic type parameter). Map is a method which takes func, and returns (the type which implements this shape, but with a different generic type parameter)".

This is quite similar to @Richiban's suggestion, but changes the keyword and lets the shape itself (and therefore Self) be generic.

Note that it is subtly different to @TonyValenti's proposal of allowing shape to have generic type parameters. There, a shape which applied to types with two generic type parameters would be written shape<TSelf<T1, T2>> SShape, whereas here it would be shape<T1, T2> SShape. With @TonyValenti's proposal, it's not clear whether shape<TSelf> means "the type implementing the shape can have any arity", or "the type implementing the shape must not be generic".

---

Addressing some of the other original examples:

// Before
public shape SGroup T
{
    static T operator +(T t1, T t2); 
    static T Zero { get; } 
}

// After
public shape SGroup
{
    static Self operator +(Self t1, Self t2);
    static Self Zero { get; }
}

// Before
public shape SConstructable
{
    // public SConstructable(); ?
    // public new(); ?
}

// After
public shape SConstructable
{
    public Self();
}

// Before
public shape SConvertable TConvertable<T> where T : SNumeric
{
    TConvertable<S> Convert<S>() where S : SNumeric
}

// After
public shape<T> SConvertable where T : SNumeric
{
    Self<S> Convert<S>() where S : SNumeric
}

// Before
public shape SCollection TCollection<T> : ICollection<T>
{
}

// After
public shape<T> SCollection where Self<T> : ICollection<T>
{
}

Of course:

public shape SEnumerable<T>
{
    public SEnumerator<T> GetEnumerator();
}

stays the same.

---

I think it's also worth thinking about HKTs alongside #339. If C# ends up with syntax like void Foo<T, A> where T : <A>, then it's natural that shapes end up with something like shape SFoo where Self : <A> rather than shape<A> SFoo.

[FiniteReality]

@canton7 If they were going to add a keyword, I'd personally prefer it be lowercase to be consistent (i.e. self) - What you have suggested is also very similar to what @Richiban suggested, using this instead of self.

[canton7]

@FiniteReality The reason I capitalised it is that it refers to a type, and not an instance. Since types normally start with an upper-case, I thought that made sense.

Agreed that it makes it the odd keyword out though. If that's the biggest complaint with the proposal, I'm happy!

What you have suggested is also very similar to what @Richiban suggested, using this instead of self.

Indeed:

This is quite similar to @Richiban's suggestion, but changes the keyword and lets the shape itself (and therefore Self) be generic.

[FiniteReality]

Indeed:

This is quite similar to @Richiban's suggestion, but changes the keyword and lets the shape itself (and therefore Self) be generic.

Whoops, I missed that bit while reading your comment. Sorry 😅