Added: C# Language Design Notes for Jul 24 and 26, 2017

[MadsTorgersen]

C# Language Design Notes for Jul 24 and 26, 2017

We started putting a series of stakes in the ground for nullable reference types, based on the evolving strawman proposal here. We're doing our first implementation of the feature based on this, and can then refine as we learn things from usage.

1. Goals
2. Nullable reference types
3. Rarely-null members

Please discuss!

[benaadams]

Consider overlap with nullable ref types?

e.g. ref? #497

[alrz]

I don't understand why ? should be on ref, when you have nullable types, ref can be added just like what we have now, e.g. ref object vs ref object?.

[iam3yal]

@MadsTorgersen The the following is really weird:

public void M(string name!) { ... }

// equivalent to
public void M(string name)
{
    if (name is null) throw new ArgumentNullException(nameof(name));
    ...
}

Many times, in this case, you may want to also check that the name is empty and throw so why would you add a special syntax for one but not the other?

Yes, checking for nulls is probably more common but it also really weird that we have special way to express it in the language whereas we can't do it for say strings and arrays which probably comes after checking for nulls.

I'd really prefer to have proper support for contracts in the language then to poison the code with a special syntax, I mean what's next? !"" and ![]... and don't worry this isn't a joke people will ask for it.

[benaadams]

I don't understand why ? should be on ref, when you have nullable types, ref can be added just like what we have now, e.g. ref object vs ref object?.

@alrz that's a different thing that is null; its saying the pointer points at null; vs the pointer variable itself is null

You can't do

static ref int Find(int[] array, int value)
{
    for (int i = 0; i < array.Length; i++)
    {
        if (array[i] == value) return ref array[i];
    }
    
    // What do you return here? null, default etc?
}

Suggestion would be something like

static ref? int Find(int[] array, int value)
{
    for (int i = 0; i < array.Length; i++)
    {
        if (array[i] == value) return ref array[i];
    }
    
    return null; // or default
}

However a ref? return would want similar constraints in use to nullable reference types (compile error to assign to non-nullable ref or var without null checking first; same with accessing a property of it.

[alrz]

@benaadams

You can't initialize a by-reference variable with a "value", I'm thinking that it should always point to a "ref" so it doesn't even make sense to say return ref null. for that to work you should have a safe-to-return-by-ref variable, i.e. a field under the hood.

private static int dummy = default;
static ref int Find(int[] array, int value)
{
    for (int i = 0; i < array.Length; i++)
    {
        if (array[i] == value) return ref array[i];
    }
    
    return ref dummy;
}

In Rust, &0 (a reference to a value) works exactly like that, it introduces a variable with the said value and then it returns a ref to that variable. Are you suggesting that the compiler should introduce a field for every return ref null?

[benaadams]

That would result in an incorrect algo as you wouldn't know if it was found; so it would instead need to be

private static int dummy = default;
static (ref int, bool) Find(int[] array, int value)
{
    for (int i = 0; i < array.Length; i++)
    {
        if (array[i] == value) return (ref array[i], true);
    }
    
    return (ref dummy, false);
}

At which point you are creating an new ValueTuple for each return.

Are you suggesting that the compiler should introduce a field for every return ref null?

Looking at what's going on at a lower level its more like:

static IntPtr int Find(int[] array, int value)
{
    for (int i = 0; i < array.Length; i++)
    {
        if (array[i] == value) return &array[i];
    }
    
    return null; // or default
}

As a ref variable is a memory location; so no extra variable is introduced.

Except... a ref being null would be a violation of what a ref represents; hence the nullable ref ref?

A null ref? wouldn't be able to be assigned to or read from; but could be checked as to whether its null (and read post null check; compile error if not)

[MgSam]

The ! operator seems like a terrible idea to me. It undercuts any semblance of having a contract for nullability. If I declare a property string Name {get; set; } and then in some other class someone can just do c.Name = null!, then what is the point of the nullability tracking? It actually makes things worse in this case because it gives you a false sense of certainty about non-nullness when in fact you shouldn't have any.

If you have a "rarely null" object, maybe the right course of action is to rethink the design rather than add an operator that allows undercutting the feature.

[jnm2]

Am I understanding this correctly? This bothers me deeply:

The compiler code base has a property (BoundExpression.Type) that's non-null in 95% of the cases. In a few places under very specific and well-defined circumstances it is allowed to be null.

What's the guidance for this kind of property?

For this kind of member you want the burden to be put on the few, not the many. It should be declared as not null, and the few places that will use it as null anyway should explicitly circumvent the annotation.

If you are setting the property this is fine, but if you are getting the value, the property's contract promises that it will never return null. My code which relies on the static knowledge that the value is not null will make the reasonable assumption that the non-null annotation will mean something at runtime and I won't get an NRE by dotting off it.

[HaloFour]

Agreed. "Rarely nullable" is still nullable and justifying the violation of that contract smacks of nothing less than sloppy design. Expediency is never an excuse.

The fact that the Roslyn codebase exhibits some of these patterns is only evidence that it will also arise in other codebases written and maintained by developers less adept than I assume the C# team is. As such, Roslyn should strive to demonstrate idiomatic migration and use of this feature. It should be no less painful for you guys, especially as you are now debating through just how painful it should be. Swallow your own medicine, do it thoughtfully and encourage proper use of this feature.

[DavidArno]

@MgSam,

The ! operator seems like a terrible idea to me

Chatted with a couple of mates today about this feature. It's pretty much a "I want an NRE here, please" notation. "Dammit" seems appropriate, as folk are likely to utter variations on that word when it kills their app. "Bang" is a common name for ! and it'll make your app go bang more often than not.

We all settled on calling it the "Fail At Run Time" operator (Fart operator) as the consensus was that it really is a silly idea, so it needed a silly name.

This comment is retracted: see my reply to @Joe4evr for reasons why.

[DavidArno]

@HaloFour,

Having had time to thing about our discussion around ! yesterday and reading these meeting notes, I am now even more convinced than before that ! is absolutely undesirable. For those rare cases where a method has to handle a rarely null reference, the existing approach:

#pragma warning disable xxxx
T Foo()
{
    ...
}
#pragma warning enable xxxx

wins out in my view. It has the same effect. It is far more glaringly obvious that a special case, "rarely null", is being handled. And it's a pain to write, thus ensuring it will genuinely only be used in those edge cases where a rarely null reference absolutely is unavoidable without compromising the code in some other significant way.

[Joe4evr]

The idea behind !. is that not everybody does checks with the exact syntax of if (obj == null) (or !=), so the compiler will needlessly warn in certain cases.

public class Foo
{
    public Bar? Bar { get; }
    public bool HasABar => Bar != null;

    public async Task Baz()
    {
        if (HasABar)
        {
            //compiler would warn here because the flow analysis is ignorant of what the previous line does
            await Bar.DoTheThing();
        }
    }
}

await Bar!.DoTheThing() would tell the compiler "I have checked that the reference is not null, it's just in a way that you don't see."

I keep coming back to wanting a [ChecksNullFor] for these kinds of properties so that the compiler can understand these alternative checks and as such reduce usage of !..

[DavidArno]

@Joe4evr,

So we have a situation that flow analysis will be used to determine "nullness" of a reference. But that flow checking will only analyse the local context (eg the code within a method, not other methods/properties called by that code).

As a result, for folk that like to structure their code such that explicit expressions are replaced with small methods and properties, that flow analysis will fail. This is a bad place to be, as it discourages this good practice.

So the proposed solution is a shorthand "hey compiler, don't worry, I know what I'm doing" syntax that suppresses warnings resulting from the limited flow analysis that the compiler performs.

However, the fact remains, most developers really do not know what they are doing half the time, so they'll misuse this feature to suppress warnings they do not properly understand and they'll be left uttering "dammit"...

I am really beginning to sympathise with the team here. I really want this feature and I really want it to be done right. But it's clear that really doing it right is a nigh on impossible task. I still feel that ! is a bad idea, but I can see why the team have little choice on the matter. So I've retracted my previous "it's a silly idea" comment. It's not silly, it's just maybe the simplest and least bad idea of a bunch solutions to a near impossible challenge.

[DavidArno]

@Joe4evr,

Of course, the solution is to not allow null in one's code in the first place, such that your example can be simplified to:

public class Foo
{
    public Bar Bar { get; }

    public async Task Baz() => await Bar.DoTheThing();
}

but not only would re-educating folk to avoid null take a very long time, there are lots of folk out there who passionately believe null is a good thing (they are utterly wrong in my view, but that's the way it is), so such re-education may never happen.

[DavidArno]

@eyalsk,

@MadsTorgersen The the following is really weird: ...

I have to disagree. I was thinking through how the framework would cope with nullable reference types. By way of an example, I'll use Where:

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, 
                                                  Func<TSource, int, bool> predicate) 
{
    if (source == null) throw Error.ArgumentNull("source");
    if (predicate == null) throw Error.ArgumentNull("predicate");
    return WhereIterator<TSource>(source, predicate);
}

My initial thought was that this could just be changed to:

[assembly: NonNull]

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, 
                                                  Func<TSource, int, bool> predicate) =>
    WhereIterator<TSource>(source, predicate);

But that would break backward compatibility with previous language versions in a horrific way. So those null checks are needed. Those null checks are really common in code and need to be kept for backward compatibility, but since source and predicate aren't suffixed with ?, why should the author have to handle such nulls? So the use of ! in this case really makes a lot of sense to me. The method just becomes:

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source!, 
                                                  Func<TSource, int, bool> predicate!) =>
    WhereIterator<TSource>(source, predicate);

Which is a much neater version in my view.

Sure, when handling strings, one might check for empty strings too. When handling class Foo, one might equally check the state of Bar. The point being, those checks can still be made; it's just the need to check null is auto handled. And using throw expressions, even checking for empty strings becomes a neatly handled thing, as within the body of my method, I can be sure they really aren't null, without having to write those tests myself:

public string FullName(string givenName!, string familyName!) =>
    givenName.Length > 0 && familyName.Length > 0
        ? $"{givenName} {familyName}"
        : throw new BadNameException("Both given and family name must be specified");

To my mind, this is such a useful feature, it stands on its own and could even be introduced prior to nullable reference types being added to the language.

[iam3yal]

@DavidArno I see your point, I agree. ;)

p.s. I said it's weird and not that it isn't useful because my thoughts were about adding proper contracts support to the language but then this feature doesn't really negate the possibility so it's all good.