Proposal: "Closed" enum types

[gafter]

Champion issue: #9011

Related to #485, it would be helpful to be able to declare an enumerated type that is known to the language to be closed. The syntax is an open question, but as a first proposal I'll suggest using closed as in #485.

For example, a closed enum

closed enum Bool { False, True }

Could be used like this

int M(Bool b)
{
    switch (b)
    {
        case Bool.False: return 0;
        case Bool.True: return 1;
    }
}

This code would be considered OK by flow analysis, as the method returns for every possible value of the parameter b. Similarly a switch expression that handles every possible declared enumeration value of its switch expression of a closed enum type is considered exhaustive.

To ensure safety:

• The arithmetic operators on closed enum types are not defined. For example, there is no operator Bool +(Bool, int). Alternately, we might consider them defined but only usable in unsafe code.
• There is no conversion from some underlying type to the type Bool (or only in unsafe code)

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-26.md#discriminated-unions

[0xd4d]

If you add [Flags] to it, is that an error?

There must be a way to convert an integer to the closed enum (unsafe code).

[gafter]

I imagine you could add [Flags] to it, but it wouldn't have any effect other than possibly generating a warning. That attribute has no language meaning.

[alrz]

Since enum class is going to be the syntax for discriminated-unions, I vote enum struct for this. When there is no sub-member, it is practically a closed enum,

// if you can do this
enum struct Option<T> { Some(T), None }
// there's nothing to stop you from doing this
enum struct Bool { False, True }

I think this is more of a lowering strategy question - whether or not this should be emitted as a proper enum.

If we go with proper enums, we can't manage external values, so I think we will need to emit it as a struct anyways.

(some examples from F#)

[DavidArno]

@0xd4d,

There must be a way to convert an integer to the closed enum (unsafe code).

Why? I see no reason for closed enums to have any sort of fixed underlying value. If you want to convert, then use a switch expression:

Bool b = i switch {
    0 => False,
    _ => True
};

@alrz,
enum struct Option<T> { Some(T), None } is sort of how I envisioned DU's looking (and so agree that enum struct Bool { False, True } is a good choice for closed enums.

However, isn't there still an issue pattern matching on those as I'd imagined DU pattern matching would be type based, ie I'd write something like:

var x = option switch {
    Some v => v,
    None => default
};

But with enum struct Option<T> { Some(T), None }, Some isn't a subtype of Option.

I can invent some syntax here:

enum struct ShapeName { Point, Rectangle, Circle }

enum struct Shape 
{
    Point,  
    Rectangle(double Width, double Length),
    Circle(double Radius)
}

and I'm happy that this is consistent: a closed enum is a DU where the values have no parameters.

And I can imagine how I'd pattern match this stuff:

var area = shape switch {
    Point => 0,
    Rectangle { Width: var w, Length: var l } => w * l,
    Circle { Radius: var r } => Math.PI * r * r
};

But, I've no idea how this lot gets lowered by the compiler. For a struct, Shape, that contains a Circle "value", what does shape is Circle { Radius: var r } mean?

[dsaf]

Does it mean that existing enums will be updated to be somehow open? Otherwise the terminology is confusing.

[DavidArno]

@dsaf, existing enums are already open. For:

enum Shape { Point, Rectangle, Shape }

Shape can take any int value. Point, Rectangle, and Shape are little more than int constants.

But it would indeed be prudent to talk of them as being open enums to clarify the difference between existing enums and the closed variety.

[alrz]

@DavidArno

isn't there still an issue pattern matching on those as I'd imagined DU pattern matching would be type based

No they wouldn't be types (or members), if it's a struct DU, all parameters get emitted as flat fields into the struct, the rest is compiler magic. See what F# produces for your example as a value DU.

Though I can imagine C# could do a better job utilizing runtime support for FieldOffsetAttribute if possible (e.g. if all parameters are blittable) - minimizing the struct size.

[0xd4d]

@0xd4d,

There must be a way to convert an integer to the closed enum (unsafe code).

Why? I see no reason for closed enums to have any sort of fixed underlying value. If you want to convert, then use a switch expression:

Bool b = i switch {
    0 => False,
    _ => True
};

That's inefficient unless it's a trivial enum like the above Bool enum. Deserialization should be quick.

It should be as simple as it is today, except the programmer must verify that it's a valid value (or it's undefined behavior)

int value = 1;
if (InvalidValue(value)) throw ...;
unsafe { return (MyEnum)value; }

[alrz]

@0xd4d You can add it yourself, or have a generator to do it.

enum struct Bool { 
  False, True;
  public static explicit operator Bool(int v)
    => v switch { 0 => False, 1 => True, _ => throw InvalidCastEx };
}

I think unlike enums the corresponding "integer tag" is solely an implementation detail for DUs, so I don't think it's a good idea for the compiler to generate that out of the box.

[0xd4d]

@alrz Your code is the same as @DavidArno's code, a big switch statement which generates a lot of code and causes extra CPU usage at runtime. My solution to use a cast in an unsafe block has the same perf and code size as code we can already use today.

[DavidArno]

@alrz, "compiler magic" suits me. It's then @gafter's job to solve how to have struct-based DUs (including closed enums) work with a is X b, rather than mine 👍

@0xd4d, it's very unlikely that the solution to serialization of closed enums and other DUs is to convert them to ints. I guess it all depends on whether closed enums are implemented as a completely different solution to DUs or not. Doing so would be a mistake in my view. So I'd argue that any solution to serializing

enum struct Bool { False, True }

also has to be able to serialize

enum struct Option<T> { Some(T), None }

[alrz]

The only difference with the original proposal would be that False and True in that example wouldn't be compile-time constants so we can't use enum struct and enum interchangeably.

That is because an enum type as it exists today, is just a group of named constants with zero safety guarantees, I'm not sure if it makes sense to try to "fix" that or take the trade-off and favor struct DUs.

[333fred]

why not closed by default, and open as suggested syntax to open them?

Because open is the default today, and we can't change that.

[AartBluestoke]

@gafter would your intent to be that adding a new value to a "closed" enum is a breaking change? would this be binary compat breaking change, or just a "switch statement which must pick a value entered unknown territory by not finding a match" exceptional territory. eg: what if you wanted to add trinary logic "not provided by customer" to the bool enum (bad naming example)

[franchyd]

@AartBluestoke I believe its true, false and filenotfound.

[CyrusNajmabadi]

With DUs, for example, especially struct-based ones, it would have to be very carefully designed to avoid bizarre outcomes.

It's certainly a breaking change in your API. But the question on the consumption side is: is it possible for me to write code that is resilient to breaking changes. I would hope so. Especially something as basic as "in case of something i don't understand, bail out".

[TahirAhmadov]

I updated my post above to correct that point.
Another reason may be that it's probably impossible to prevent setting an invalid value using reflection, and that's yet another reason to still be able to handle unexpected values (in certain codebases).

[Eli-Black-Work]

@TahirAhmadov I think that we'd want to warn on all cases that don't have a default: label, in case a new .dll gets dropped in that has new values in the enum.

I think that would make your above list something like this:

• If has a default case, don't warn.
• If it's exhaustive and doesn't have a default case, warn.
• If it's not exhaustive and doesn't have a default case, error.

I feel like this pretty much kills the entire point of this proposal, though. I wonder if we should ping Gafter and ask his thoughts 🙂

[TahirAhmadov]

@Bosch-Eli-Black I don't think we need to go that far; the compiler will output a hidden default which throws something like UnexpectedEnumValueException, which is sufficient for 99% of use cases.

To the extent that either reflection or new DLL can introduce an unexpected value, an argument can even be made that you still shouldn't define a default case, and instead should just catch that exception with a try-catch. On the other hand, the try-catch is boilerplate, and it would be weird if default is specified but then purposefully not emitted and replaced with the built-in throw line. (Perhaps specifying a default in an exhaustive switch would be an error in that case.)

PS. Another approach to consider would be an even stricter version: the binder refuses to work if a DLL with an unexpected enum option is copied to the executable folder; and verification code is emitted (not sure if it's possible during IL emit or JIT) whenever any value is set to a closed enum type field/local/etc., making it "airtight". This is probably too expensive, though.

[Eli-Black-Work]

@TahirAhmadov Sorry, I just read through more of the thread, and I think now I understand things a bit better 🙂 Your list looks good to me! (#3179 (comment))

[Swiftly1]

Closed enum for me is a tool for error handling.

Error handling often requires additional informations (messages, args, etc)

I propose "constructor" syntax for those enums that's similar to the concept used in records

closed enum ValidationError
{ 
	EmptyString,
	UnexpectedCharacter(int index),
}

closed enum UserServiceError 
{ 
	NotFound,
	NonOkHttpStatusCode(int statusCode),
	NetworkError(string debug_log)
}

Random ass usage example:

public (bool Success, ValidationError? Error) Validate(string s)
{
	if (string.IsNullOrEmpty(s))
		return (false, ValidationError.EmptyString);
		
	if (s[0] != "[")
		return (false, ValidationError.UnexpectedCharacter(0));
		
	return (true, null);
}

var error = Validate("s").Error.Value;

switch (error)
{
	case EmptyString:
	{
		Console.WriteLine("empty");
	}
	case UnexpectedCharacter context:
	{
		Console.WriteLine($"unexpected character at {context.index}");
	}
}

[HaloFour]

@Swiftly1

See #113, as adding data elements definitely turns them into discriminated unions.

[AartBluestoke]

because even 'closed' enums can have added values, is it even physically possible to load a library of "closed" value and have that make sense?
If so, it makes loading any library version with a different Enum list a breaking change, but in many circumstances a newer version of a library is pulled in via dependencies.

library v1 has
closed enum Error{
Missing =1,
Forbidden =2}

library v2 has
library v1 has
closed enum Error{
Missing =1,
Forbidden =2,
Corrupted =3}

you compile against v1. loading v2 is now a binary breaking change, where you now unexpectedly fall through the previously "Complete" switch.

[NeilMacMullen]

Probably jumping in to beat a dead horse, but it seems to me that much confusion has been caused by using the term "enum" here. C#'s "original sin" IMO was to borrow the C concept of a list of values backed by integers and to allow the use of bitfield operations (Flags).

What I want almost all of the time when declaring an "enum" is actually a closed set of distinct tags that are strongly typed. It's usually desirable to be able to enumerate over that set. In other words, I want the strongly-typed enum pattern without confusing things by providing access to the backing type (which allows people to inject unrepresentable values).

[TahirAhmadov]

Probably jumping in to beat a dead horse, but it seems to me that much confusion has been caused by using the term "enum" here. C#'s "original sin" IMO was to borrow the C concept of a list of values backed by integers and to allow the use of bitfield operations (Flags).

What I want almost all of the time when declaring an "enum" is actually a closed set of distinct tags that are strongly typed. It's usually desirable to be able to enumerate over that set. In other words, I want the strongly-typed enum pattern without confusing things by providing access to the backing type (which allows people to inject unrepresentable values).

The int-backed C style enums (what is now enum in C#) have a number of benefits: they are simple; performant; the bitfield operations are desirable in some cases; easy convertibility to the underlying integer is frequently needed.
Yes, the ability to convert any arbitrary number to enum is a downside, but that's exactly what this proposal aims to solve.

[NeilMacMullen]

Yes, I appreciate that enums are what we have now just pointing out that when people talk about "closed sets" they may be coming at it from a perspective that is inconsistent with the assumption that the set must be based on a numeric backing field so perhaps it's better (or at least useful) to consider solving the general problem rather than 'just' patching up int-backed enums. For example #2849 is effectively a proposal for closed set of string-backed values. IOW at some point maybe it's better not to call these things "enums".

[TahirAhmadov]

I don't have any strong opinions on naming here, however my main point is that, the int and string backed "enums" (or however we name them) are needed irrespective of other proposals, specifically because in some cases you want to assign an undefined backing value to it, such as, in many network communication scenarios.
Having said that, it's downright impossible to rename the backing-value enums at this point, so even if everybody agreed that "enum" is a bad term for them, there isn't much to be done about that....

PS. Please see this comment of mine: #2849 (comment)
It shows why, I think, current C# enum is a good feature to have - and not a legacy baggage.

[Eirenarch]

I don't know if this is the right place to comment as it seems that the design has evolved a lot since what was discussed here but the discussion link of the proposal leads here.

The most recent version of this proposal #9011 has the closed enums with an integer backed value (which I think is preferable) but requires that 0 is a valid value. I think this is problematic for a very common real world scenario - storing the enum in a database while avoiding the bug of unassigned value. In theory the current open enum could result in an invalid value in the database in many ways but in practice the only way it happens is if you forget to initialize - i.e. the value 0. This problem has a simple solution - start from 1 and restrict the column in the database so that 0 is not a valid value. This solves the vast majority of enum bugs (i.e. turns them into runtime errors rather than corrupt data errors). If we want to use a closed enum we will have the problem of the default 0 again. We could keep using open enums for storing data in the database but then we lose the much desired switch exhaustiveness check. I understand why this 0 is required for closed enums but I hope you can think of some trick to solve the problem.

[CyrusNajmabadi]

NRTs do throw exception when the undefined value (null) appears. It doesn't prevent exceptions when an element from an array of nulls is used.

That's true for NRTs. But not true for NVTs. For NVTs you do get prevented with arrays, and you will get exceptions if you try to bypass it. This is exactly the reason that NVT was created 20 years ago. If you want these semantics, you can have it. Just use NVT.

Again the correct thing to do in this "vast minority of cases" is to throw an exception to prevent data corruption.

You keep mentioning data corruption. But that's a domain issue for you where you are stating that there is a value of the enum that is you both consider corrupt, but you are not willing to model that either in the enum itself (like Invalid = 0), nor are you willing to use NVT.

This is not a tenable position for us. If you believe there is such a concept in your domain model it in C#. The tools are there, and have been for ages :)

[HaloFour]

IIRC it's always been that way:

CS8509 - not handling all of the declared cases
CS8524 - not handling the entire range of the underlying integral value

You could make CS8509 an error, too, if you really want the compiler ensure that you're covering the declared cases.

[spydacarnage]

NRTs do throw exception when the undefined value (null) appears.

But since NRTs have come about, I can't remember the last time I had an NRE - as if the feature is designed to protect against them or something. Is it infallible? No. Was it advertised as such? Also no. Does it work in 99% of cases? Sure does!

The runtime doesn't have a problem with the undefined values for enums today

And it still won't, so carry on using them as you are and nothing needs to change. This proposal is for a new thing in the language, so the runtime can't have a problem with it today...

As business apps don't use the value of the enum for arithmetic

I'm not sure any apps should be using enum values directly for maths...

and the numeric value doesn't mean anything you can use the 0 to detect the bug of failing to provide a proper value. Then you can constrain your database not to accept the 0 and thus catch the bug before it corrupts the data forever.

So, what I'm reading is that in your business apps, the way you use enums, you don't use zero. That doesn't apply to everyone. I mean, if we're applying our own personal tastes to a global argument, I'd say you shouldn't declare an instance of an enum without setting your desired value for it (and yes, that means I don't declare empty arrays of value types), but everyone uses the language differently, and it cannot cater for every single use case. As has been pointed out, your use case is the significant minority, so unfortunately, that means on this occasion, you don't get exactly what you want.

TLDR: If closed enums don't work how you want them to, don't use them, and let the dev team get on with making something that the majority of users will use.

[NoahStolk]

what is the problem of throwing an exception on an undefined zero

Because this feature is intended to stop runtime exceptions by checking valid values at compile time, as NRTs did.

NRTs do throw exception when the undefined value (null) appears. It doesn't prevent exceptions when an element from an array of nulls is used. Again the correct thing to do in this "vast minority of cases" is to throw an exception to prevent data corruption.

if the enum is uninitialized (from the perspective of the programmer if that is an important distinction)

I think the core of the issue is that your definition of "uninitialized" is different to that of the language itself. As Cyrus has said (many times), that isn't going to change. Zero for an enum is initialized. It might not be user-initialized, but it is initialized.

The runtime doesn't have a problem with the undefined values for enums today but we still recognize the fact that an undefined value can be used as a problem. The goal should be to minimize bugs. Do we agree that a runtime exception is smaller bug than data corruption?

The vast majority of enums in the most common C# code - i.e. business apps should not have the value 0

Not that I want to derail the main thrust of this never-ending argument, why should business apps not use zero in enums??

As business apps don't use the value of the enum for arithmetic and the numeric value doesn't mean anything you can use the 0 to detect the bug of failing to provide a proper value. Then you can constrain your database not to accept the 0 and thus catch the bug before it corrupts the data forever.

I've written many business apps in the past and the way our team worked around this issue was:

1. All enums must define a 0 value.
2. Database write models (or really any model) must be immutable records where all properties are marked with the required keyword. This enforces proper initialization so it is impossible to forget to initialize any value, including enums. We had analyzers in place that enforced these rules for our models.

For more complex scenarios we had unit and integration tests validating our logic. I don't remember any case where we ran into the data corruption bugs you're describing.

Again, I don't think the existence of a default value is the problem here. You must validate your database transactions by enforcing proper initialization. This is not a problem with enums, but a problem with how you're constructing your objects.

Even if you really want all enums to exclude 0 as a valid value. You could still add a 0 member named Invalid. And yes, to get proper exhaustiveness you would have to always check for this invalid value, which I think is a good thing. Verbose and explicit code is better than shorter/readable code that's silently ignoring corrupt data. But that's just my personal thoughts and opinions, I'm not here to tell you how you should write software.

[Eirenarch]

what is the problem of throwing an exception on an undefined zero

Because this feature is intended to stop runtime exceptions by checking valid values at compile time, as NRTs did.

NRTs do throw exception when the undefined value (null) appears. It doesn't prevent exceptions when an element from an array of nulls is used. Again the correct thing to do in this "vast minority of cases" is to throw an exception to prevent data corruption.

if the enum is uninitialized (from the perspective of the programmer if that is an important distinction)

I think the core of the issue is that your definition of "uninitialized" is different to that of the language itself. As Cyrus has said (many times), that isn't going to change. Zero for an enum is initialized. It might not be user-initialized, but it is initialized.

The runtime doesn't have a problem with the undefined values for enums today but we still recognize the fact that an undefined value can be used as a problem. The goal should be to minimize bugs. Do we agree that a runtime exception is smaller bug than data corruption?

The vast majority of enums in the most common C# code - i.e. business apps should not have the value 0

Not that I want to derail the main thrust of this never-ending argument, why should business apps not use zero in enums??

As business apps don't use the value of the enum for arithmetic and the numeric value doesn't mean anything you can use the 0 to detect the bug of failing to provide a proper value. Then you can constrain your database not to accept the 0 and thus catch the bug before it corrupts the data forever.

I've written many business apps in the past and the way our team worked around this issue was:

1. All enums must define a 0 value.
2. Database write models (or really any model) must be immutable records where all properties are marked with the required keyword. This enforces proper initialization so it is impossible to forget to initialize any value, including enums. We had analyzers in place that enforced these rules for our models.

For more complex scenarios we had unit and integration tests validating our logic. I don't remember any case where we ran into the data corruption bugs you're describing.

Again, I don't think the existence of a default value is the problem here. You must validate your database transactions by enforcing proper initialization. This is not a problem with enums, but a problem with how you're constructing your objects.

Even if you really want all enums to exclude 0 as a valid value. You could still add a 0 member named Invalid. And yes, to get proper exhaustiveness you would have to always check for this invalid value, which I think is a good thing. Verbose and explicit code is better than shorter/readable code that's silently ignoring corrupt data. But that's just my personal thoughts and opinions, I'm not here to tell you how you should write software.

In a previous comment I said that defining Invalid value for 0 is what I would do because it is the right thing to do but that would require more code than the wrong thing (normal value for 0). Your efforts are noble and you should aim to force initialization but no guarantee is stronger than a restriction on the database side and an uninitialized value can slip from many places for example deserialization.