Remaining design work in and around records

[MadsTorgersen]

Remaining design work in and around records

C# 9.0 introduces init-only properties, as well as record declarations. Both represent an independently useful but somewhat narrow slice of the full scope we had in mind. Here is an attempt at a breakdown of leftover ideas that we could pick back up in the next version of C#.

More initialization functionality

Init fields

Like readonly fields, but can be initialized in an object initializer. We're currently holding off on this one.

Init members

Allow other kinds of members (and accessors) to be marked with the init modifier. They would be callable only during initialization, often as helpers to constructors, init-only properties and final initializers. In return they would be able to modify readonly fields and call other init-only members.

A possible member kind to include is init-only fields, which would not only be indirectly assignable by init-only members, but also directly in an object initializer.

Final initializers

A new kind of member declaration, probably of the form init { ... } designating a block of code that will always be executed at the very end of the initialization phase, after the constructor and any init-only members. This allows for checking or bringing the object into a valid state.

Final initializers will require participation by the client to call the final initializer after any object initializer is complete. Therefore they will probably need to "poison" the constructors of the class so that they can't be called by languages or language versions that don't know to call final initializers.

Required members

Annotations (yet to be designed) would combine to express a set of members that must be initialized when a given constructor or factory is called. It would be an error to call the constructor without also using an object initializer to initialize the required members.

Factories

Methods (and other members?) could be designated as factories. They are restricted to return a fresh object. In return, their clients can apply object initializers to their result.

What about collection initializers?

For init-only we have focused on object initializers. It's likely to be worth thinking through what can and should be done with collection initializers - after all they also function via mutation, and might equally benefit from something morally similar to init-only properties. Init-only Add methods?

Generalizing away record magic

Records have a few ways that make them inherently special in ways that ordinary classes cannot participate in. Generalizing those out to independent features would help with the expressiveness of records and non-records alike, and make it easier to move types from one to the other.

Allowing users to define cloning

Currently a special unnameable method is generated to implement the virtual cloning functionality of records that with expressions rely on. Once we support factories (see above), we could switch over to generating and recognizing a Clone factory method. That way, non-records could support with expressions, and records could define custom Clone functionality.

Other approaches might also work for this; e.g. an operator-like declaration.

Cross-inheritance between records and non-records

Currently hierarchies of record and non-record types are seperated - they cannot inherit from one another (with the exception of object). We should generalize the base class requirements of records in such a way that records and non-records can meaningfully inherit each other where it makes sense.

Primary constructors

Positional members in records were designed with an eye to allowing generalized primary constructors on other classes. Essentially at this point, primary constructors are fully included in positional members, and offering the feature on its own would involve removing aspects such as generation of properties and deconstructors.

Bodies and attributes for primary constructors

Final initializers may somewhat alleviate the need for primary constructors to have a body. However, we could still allow one to be declared - not just for statements in the body itself (things you want to get done before object initializers run), but as a syntactic location to put attributes, accessibility etc. to apply to the underlying constructor.

Automatic with-ing on all structs?

By virtue of their value semantics, structs could always support with-expressions. They would be implemented simply using the copying mechanism built-in to structs. Should we?

More record functionality

Certain aspects of records themselves weren't quite finished in C# 9.0.

Struct records

Even though much of record semantics is inspired by structs, not all record features are available on structs. We should allow structs to explicitly be records, so that they can e.g. have positional members and strongly typed Equals/IEquatable<T> implementations.

Data properties

data string Name; didn't make it into C# 9.0 as a shorthand for public string Name { get; init; }, but is on its way out in preview. We should finish it up based on feedback on that preview.

Discriminated unions

We've said that we'd look at discriminated union scenarios once we got records under control. That time has come. The design space is wide open at this point.

[HaloFour]

I think it would be interesting to explore if "factory" and "final initializer" could be combined in a way to enable builder patterns in the language. I don't know if that's remotely compelling, but it would seemingly involve a "final initializer" being able to return a value that is not necessarily the same instance or even the same type as the instance being initialized.

I'm very curious to see the take on discriminated unions, especially if the team is going to tackle both reference- and value-type flavors. I think that records play well into the former, but the latter would be very helpful in defining zero/low-cost abstractions like Maybe<T>.

[orthoxerox]

I am really excited about DUs. I would love them to cover both:

• closed set of values, so I don't get badgered about including _ => throw new Exception("You made this enum value yourself, didn't you") in a switch
• closed set of subtypes, so I do get badgered about not including one of them in a switch.

Maybe<T> is a very interesting type to keep in mind while designing DUs, since you want None to be a type (or even a value!) that is a valid subtype of Maybe<T> for all T.

[IS4Code]

Which interfaces can be implemented by records by default? I think IEquatable<T>, IComparable, IComparable<T>, IStructuralEquatable, IStructuralComparable, IConvertible, IEnumerable, IEnumerable<S>, IList, IList<S>, IReadOnlyList<S>, IDisposable, and ICloneable (T being the record type and S the homogenous member type) would all make sense to be provided automatically, conditionally whenever sensible and applicable. Tuples already implement some of these, so there is a good reasoning for those.

I am also thinking about sematically different "typedefs" here, i.e. something like struct record UserName(string Value); which should be a lightweight wrapper around a string but should still be maximally usable like a string. Additionally struct record ValueArray3<T>(T Item1, T Item2, T Item3); (hope records can be generic) makes sense to implement collection interfaces.

[Unknown6656]

I absolutely love DUs, that would get my rid of my (ugly as hell) auto-generated DUs (e.g. DiscriminatedUnions.tt which generates DiscriminatedUnions.cs)

[HaloFour]

Are there any plans for records to generate equality operators? It seems really strange to me that despite value semantics that they still compare by reference. Languages like F# and Scala automatically let you compare records (or their equivalent) with equality operators.

[SingleAccretion]

It seems really strange to me that despite value semantics that they still compare by reference

record.Equals(anotherRecord) != (record.Equals == anotherRecord) (unless == is forbidden for records). Yes, there are problems around operators not being polymorphic and performance, but there is no way this is the right default.

[333fred]

Are there any plans for records to generate equality operators? It seems really strange to me that despite value semantics that they still compare by reference. Languages like F# and Scala automatically let you compare records (or their equivalent) with equality operators.

@HaloFour this is now on the agenda for next Monday.

[HaraGabi]

Required members
Annotations (yet to be designed) would combine to express a set of members that must be initialized when a given constructor or factory is called. It would be an error to call the constructor without also using an object initializer to initialize the required members.

Sorry, I don't quite get this. Why not just initialize it from the constructor then?

[MgSam]

I hope you guys fix up the == design before you ship records. I'd go so far as to say I'd rather the feature be pushed to C# 10 then shipped with (record.Equals(record)) != (record == record), which will take us squarely into the pit of failure.

[SingleAccretion]

Sorry, I don't quite get this. Why not just initialize it from the constructor then?

This would, among other things, solve the issues around init (and properties/fields that are expected to be initialized by the caller in general) and NRTs:

class POCO
{
    //Right now requires a workaround in the form of null!
    public Enitity Value { get; init; }
}

Constructors are not a very good solution for the above because:

• Some serializers/deserializers do not work well (if at all) with them.
• They must be primary in order to avoid boilerplate around Property = property.
• Aren't as nice as object initializers syntactically.

[Joe4evr]

Sorry, I don't quite get this. Why not just initialize it from the constructor then?

This has already been argued in the past and answered by #2328 (comment)

[YairHalberstadt]

Cross-inheritance between records and non-records

I would love to use records for attributes, which mostly have no logic, only data, but I can't because they can't inherit from System.Attribute.

[ufcpp]

Can factories or final initializer be exception free?

Rational r = new Rational { Numerator = 1, Denominator  = 2 };
if (r with { Denominator = 0 } is { } valid) { ... }
else { ... }

instead of

Rational r = new Rational { Numerator = 1, Denominator  = 2 };

try
{
    var valid = r with { Denominator = 0 }
}
catch
{
    ...
}

[HaloFour]

@ufcpp

Can factories or final initializer be exception free?

How can a final initializer enforce property validation unless it can throw? I'm pretty sure that's the primary motivation for the feature.

[ufcpp]

How about combination of DU, closed type, and never type?

from

record Rational(int Numerator, int Denominator)
{
    init
    {
        if (Denominator == 0) return false; // or some new syntax
    }
}

var r1 = r with { Denominator = 0 };

to

record Rational(int Numerator, int Denominator)
{
    public bool TryInit()
    {
        if (Denominator == 0) return false;
        return true;
    }
}

var temp_r1 = r.Clone();
temp_r1.Denominator = 0;
if (temp_r1.TryInit())
{
    var r1 = temp_r1;
    ...
}

// or

var temp_r1 = r.Clone();
temp_r1.Denominator = 0;
Rational | never r1 = temp_r1.TryInit() ? temp_r1 : never;
// never must be type checked
...

[HaloFour]

@ufcpp

That provides no indication as to why the type failed the validation, though. I'd much rather exceptions which are the idiomatic way to handle validation errors in situations like this. If the validation was via constructor you'd expect an exception to be thrown.

[ufcpp]

There are some scenarios.

// stack trace is important and performance is not.
T Factory()
{
    if (invalid) throw new SomeException("fail reason");
    ....
}

// stack trace is not needed but fail reason is important and performance is preferred .
T | Exception Factory()
{
    if (invalid) return new SomeException("fail reason");
    ....
}

// performance is the most important.
T? Factory() // or T | never Factory(), or bool TryFactory(out T)
{
    if (invalid) return null; // or return never, or return false
    ....
}

I know it's not a good idea to create a dialect in C#. If I have to choose one of them, I think the existing exception mechanism is the best. But there are some scenarios where performance is the highest priority, and in that case, exceptions are not an option.

[dersia]

I still don't understand why init only properties are not required. This breaks so many thing especially with nullability. What happens to non-null properties that are not initialized by the caller? What state are they then in?

IMO init olny properties should be required and enforced. This at least makes sure, that they are at least initialized with a default value. I don't want to end up in a state where a nullpointer is again possible, even if I opted out of it by enabling nullability. @MadsTorgersen are there any ideas how to avoid this?

This would also make it unnecessary for another keyword to define required properties.
If one still needs optional init only properties, you can still just set the default value through expressions.

This is how it would look like with enabled nullability.

public partial class B { } 
public partial class A
{
    public string StringA { get; init; } //required
    public B Bprop { get; init; } //required
    public int IntA { get; init; } = 0; //optional
} 

Since StringA and B are not nullable and not assigned a value they are required during initialisation.

[theunrepentantgeek]

This would also make it unnecessary for another keyword to define required properties.

There are important scenarios the LDM wants to support where init properties are permitted to be initialized, but not required.

Deferring support for those while the whole initialization debt feature is designed (and debated) would be a case of letting the perfect be the enemy of the good.

If you have a property that must be initialized, there's already a solution available that you can continue to use until the new feature is available; no one is talking about deprecating constructors.

[dersia]

All of that is fine, but still there is no answer to what the state of an reference-type object would be, if nullability is enabled. If you have B being init only and it is not initialized it won't ever have a value, it will be default(B), if I am correct. And this is why I am saying this breaks all nullability features we got with c#8. Now the caller has to always check against null before calling/using a init-only member, because you can't rely on the nullability feature.

And I do understand, that there are scenarios that the LDM wants to support for optional init-only fields, this is why I included an example how this could look like.

I guess what I am trying to say is, that init-only needs to be required by default to not break nullability and if we are in need of optional init-only members then we should come up with a proposal for intentionally defining a init-only member as optional. So instead of

required string A { get; init; }

It should rather be

optional string A { get; init; }

This can then be rewritten by the compiler as

string? A { get; init; } = null;

With nullability

string A { get; init; } = null;

Without nullability

//
This would also allow for the compiler to create an auto-constructor for all init-only fields that are not optional. As far as I can remember there are some issues asking for this behavior already.

[SamPruden]

What about collection initializers?

For init-only we have focused on object initializers. It's likely to be worth thinking through what can and should be done with collection initializers - after all they also function via mutation, and might equally benefit from something morally similar to init-only properties. Init-only Add methods?

One thing that collection initializers don't handle well at the moment is fixed size collections. If we want to create an instance of a fixed size collection, we have to do something like new MyCustomArray<int>(3) { 1, 2, 3 }. There are a few problems with this.

• Repeating the length in two places - manually and implicitly from the items in the initializer - is inefficient
• We're responsible for keeping the lengths in sync, and it's only a runtime error if that goes wrong
• It requires an Add method on the collection, but that's not appropriate on a fixed size collection - it implies a List like type
• It's not compatible with immutable collections

An improvement would be to allow the init to access the full initialization list. This would allow the initializer to implicitly infer the length, and would remove the need for an Add method or other mutator.

A member along the lines of public init(Span<T> items) might be a reasonable approach to this. It would allow new MyCustomArray<int> {1, 2, 3} where items is populated by an implicit stackalloc. This init would have to be responsible for allocating the memory based on items.Length.

---

There are some tricky things with this approach. If the init is allocating memory, then that needs to be guaranteed to happen exactly once. That means that the collection initializer has to be callable no more than once, and that there needs to be a default init that happens in the case where the collection initializer hasn't run. This means that factories are a problem because if you can run the collection initializer on a factory result then it could happen multiple times.

A usage may look something like this.

unsafe struct MyCustomArray<T>
{
    public int Length { get; init set; }
    private T* unsafeData;
    
    // Optional because structs implicitly have parameterless constructors
    public MyCustomArray(int length) => this.Length = length;

    // Collection initializer - only runs if used
    public init(Span<T> items)
    {
        if (items.Length > this.Length)
            throw new ArgumentException("Collection initializer has more items than manually specified length.");

        if (this.Length == 0)
           this.Length = items.Length;

        this.unsafeData = // Allocation goes here
    }

    // Main initializer - runs always, and after collection initializer if it runs
    public init()
    {
        if (this.Length == 0)
            throw new Exception("Length must either be specified or a collection initializer must be used.");

        if (this.unsafeData == null)
            this.unsafeData = // Allocation goes here
    }
}

---

There's a real use case for this. The Unity game engine is pushing their experimental new DOTS (Data Oriented Technology Stack) tech at the moment, and they have a restriction on having no managed types. This means they have custom "native collection" types which are structs with manually managed memory, like NativeArray<T>. Documentation. They don't support collection initializers at the moment, but could if this change were made. This is becoming a core API in a fairly major C# product, so many people would benefit from this change.

What I've presented here is only a very rough sketch of an idea after a sleepless night, and has many problems I'm sure, but I hope that the goal - supporting collection initializers for fixed size collections - may be adopted.

[dersia]

There's a real use case for this. The Unity game engine is pushing their experimental new DOTS (Data Oriented Technology Stack) tech at the moment, and they have a restriction on having no managed types. This means they have custom "native collection" types which are structs with manually managed memory, like NativeArray. Documentation. They don't support collection initializers at the moment, but could if this change were made. This is becoming a core API in a fairly major C# product, so many people would benefit from this change.

What I've presented here is only a very rough sketch of an idea after a sleepless night, and has many problems I'm sure, but I hope that the goal - supporting collection initializers for fixed size collections - may be adopted.

@SamPruden I think the best way is to create a separate proposal for this, so it can be discussed and designed in a good proposal

[SamPruden]

@SamPruden I think the best way is to create a separate proposal for this, so it can be discussed and designed in a good proposal

Agreed. I meant to only make a throwaway comment but it was late at night and got away from me. I'll try to put a proposal together soon, thanks for reminding me.

[wastaz]

Discriminated unions

We've said that we'd look at discriminated union scenarios once we got records under control. That time has come. The design space is wide open at this point.

If the design space for discriminated unions is, as it says, wide open. Could this please be a feature where the implementation is compatible with the F# implementation discriminated unions out of the box? Interopability between languages on the .NET platform is and should be a very valuable thing. And for a feature that is so powerful and yet basic as discriminated unions it would really be great if interopability with the existing implementation on the .NET platform was taken into account.

[HaloFour]

@wastaz

Unfortunately I doubt that will be possible (or desirable). Most F# features like that are encoded using a ton of custom F# attributes, and F# also relies on an opaque blob of metadata embedded as a resource in the assembly to describe more metadata around the types. F# generally emits what makes sense for F# without considering the greater ecosystem. Any C# design will have to be more careful and deliberate.

[mcintyre321]

How about a Union<T0, ..., TN> type like https://github.com/mcintyre321/OneOf all coated in lovely syntactic sugar? Using one-shot records types under the hood for tagged unions.

[wastaz]

@HaloFour
Sure, that is an issue. But that doesnt mean that its impossible. F# does what F# has to because of the ecosystem it lives in (and Id argue that a lot of its language design is much more "careful and deliberate" than C#s design, but that is a completely different discussion which is best had with plenty of 🍻).

But now that discussion is being had about pulling DUs into C#, the "main language" of the CLR, instead of creating a parallell concept and making the .NET ecosystem even more split how about sitting down and see if some of the "custom F# attributes" can instead become part of the BCL, or even if (now that both languages are supposed to support DUs) some of this metadata can be removed completely. Instead of trying to solve this in two silos further cementing the divide, try to find a middle way where perhaps F# needs to make some changes as well.

That way C# could immediately gain access to all of the good DU stuff that the F# community has been doing all over the years, and easier consumption of F# libraries. And F# could maybe remove some of this extra blob of stuff and also benefit from for example performance fixes that is probably going to be more highly prioritized for C#.

[wastaz]

Honestly, I understand that it might not be possible in the end. But to outright disregard the chance to make something interopable without exploring the options available seems like such a waste. It really cements the "C#LR"-meme that already exists.

[HaloFour]

@wastaz

F# does what F# does because it can move fast and break things, cross-language interop is not a priority for the language. Any F# design here would already have to be pretty drastically altered. Just for starters all of the metadata would have to be moved out of the F# assemblies into the BCL and all reliance on the binary metadata blob would have to be reimplemented in some other way. What gets emitted would also have to be changed as it relies on other primitives in F# for stuff like comparison. It's probably easier to design from scratch than it would be to try to break down F#'s implementation to make them similar, and in the end they wouldn't be compatible with each other anyway.

It'd be easier (and much more likely) that F# will change to understand (if not emit) C# DUs. It's happened a couple of times that F# changed from a baked-in type to using something offered by the BCL, like with System.Tuple (and later System.ValueTuple after C# based its implementation around that). Fortunately it's a lot easier for F# to make changes like that.

I can't speak for the C# LDG but my understanding is that they work closely with the F# team and do discuss these kinds of language changes. Covid aside I think they physically work down the hall from one another. I assume that they'd be invited to discuss DUs. But I'll let someone from the LDG chime in on that.

[markm77]

I want to make a comment about generalising primary constructors.

With the addition of required members (and hopefully final initialisers) I believe object initialisers will become the superior way to initialise public (accessible) fields and properties. They are much more readable (esp. with nesting) and avoid constructor parameter pass-through boilerplate in inheritance chains. On the other hand, I believe constructors will remain the best way to initialise non-public (inaccessible) fields and properties due to flexibility: you can have multiple constructors and they can take any parameters including those corresponding to inaccessible things like private fields.

Given this, I think nominal records will in future become much more important than positional records and note one can always add a Deconstruct method to a nominal record if desired.

My concern is what happens when we generalise primary constructors to non-record classes. In order to be useful for likely future best practice I think it is important that when generalised they (a) drop the auto-deconstructor and (b) provide useful syntax for synthesising private/protected fields. This will make them a massive time-saver e.g. when creating service classes that use dependency injection etc.