Added: C# Language Design Review, Oct 4, 2017

[MadsTorgersen]

C# Language Design Review, Oct 4, 2017

We looked at nullable reference types with the reviewers, Anders Hejlsberg and Kevin Pilch.

1. Overall philosophy
2. Switches
3. Libraries
4. Dotted names
5. Type narrowing
6. The dammit operator
7. Array covariance
8. Null warnings
9. Special methods
10. Conclusion

Please discuss!

[orthoxerox]

I kinda agree with "checking dotted names makes sense, since that's what people do", but at the same time I don't. Multithreaded programs are already hard to debug, and if developers are taught to rely on nullability checkers they will likely miss cases where dotted chains are mutated from another thread. Then nullability checks on dotten names will join other "C# gotchas".

What if there was a lightbulb fix for the warning?

if (foo.bar.baz != null) HandleNonNullable(foo.bar.baz);
//a lightbulb proposes to turn this into
if (foo.bar.baz is var baz && baz != null) HandleNonNullable(baz);
//of, if declaration expressions are implemented
if ((var baz = foo.bar.baz) != null) HandleNonNullable(baz);

[dsaf]

... It will help you find many bugs, but it will not guarantee anything. ... We should not think of the feature as dialable, with multiple switches or settings. We should design our way to the best balance, and stick to it. One switch: On or off.

Fair choice, but I would prefer philosophy like "the language design makes the guarantee possible, but a particular compiler/IDE implementation might choose to loosen/tighten it over the time; we recognize that C# developers come in all skill levels and thus deserve to have options".

var array = new string[10];

These are numerous in current code, and very often they are fine. Besides, the code one would have to write instead is quite unappetizing!

I don't know, Rust-style would look appetizing:

var array = new string[] = [""; 10];

[Joe4evr]

Is there any word on what this comment from back on the Roslyn thread mentioned, specifically the auto-insertion of a temp variable? I see so many people here being gung-ho about wanting a new construct like

if (foo.Bar.Baz is var baz)

or something so they have a null-check and local reference in one go, whereas that comment says the compiler could just take the existing != null construct and insert a new local on the developer's behalf.

[alrz]

It would be really nice to have an analyzer to "infer" nullability of types and annotate them accordingly. It wouldn't be a simple analyzer but it would be a great win in the adoption story.

[DavidArno]

On the other hand we should not warn on array creation expressions, even though they create a whole array of forbidden null values:

    var array = new string[10];

These are numerous in current code, and very often they are fine. Besides, the code one would have to write instead is quite unappetizing!

I don't get this logic. What is unappetizing about changing this to:

    var array = new string?[10]; // array is string?[]

which is all that would be needed to fix the warning? Or alternatively, lying to the compiler to suppress the warning:

    var array = new string[10]!; // array is string[]

Not warning on this would be irresponsible in my view as it creates a glaring hole in the reliability of any analysis. Given that it's irresponsible, along with the option of marking the array as T?, plus the suggestions from @dsaf here and @orthoxerox in #976, there is plenty of scope for addressing this one properly.

[DavidArno]

@Joe4evr,
The two suggestions:

if (foo.Bar.Baz is let baz)
if (foo.Bar.Baz is value baz)

would address the "null-checked variable without having specify the type" desire. I've checked the raw notes that @MadsTorgersen has put up but there's no mention of them (or anything similar). SO I'm guessing it hasn't been discussed yet.

[orthoxerox]

@dsaf I have created #976 to support creation of initialized arrays.

[jnm2]

We should not think of the feature as dialable, with multiple switches or settings. We should design our way to the best balance, and stick to it. One switch: On or off.

I must have missed something; this seems a dramatic departure from recent meetings.

Don't worry too much about unannotated libraries.

Same with this, but I'm pleased with this philosophy.

[jnm2]

I tend to agree with @orthoxerox's proposed solution to the multi-thread or intervening call situation. Not quite satisfied with the analyzer decision on this.

[MgSam]

I agree that nullability should be tracked across dotted chains. I do not agree that this should be scrapped or modified in some way because of the possibility of cross-thread modifications.

Most assumptions about the evaluation order of your code are off if you're writing that kind of shared-state multi-threaded code. Unless you're an expert in it you probably should be avoiding it anyway. It doesn't make sense to gimp new features because they won't work well in cross-thread contexts, or else we'd never get new features.

[scalablecory]

@MgSam threading is only an example of the problem. the real problem is any "other" code, which could be a hard-to-track race condition or it could be an innocuous method call. Consider:

if(foo.Bar != null)
{
    foo.ModifyBar();
    foo.Bar.Baz(); // proposal would have Bar still assumed non-null.
}

I actually don't mind this behavior; I'd rather have it catch some bugs than no bugs. I guess the counter-argument is that some devs will assume their code is safe if the compiler doesn't give a warning.

Interesting thought: if we could mark something as somehow "fully verified", Roslyn could potentially optimize things a little better -- simplest probably being to emit call rather than callvirt.

[MgSam]

I understand that. I agree with the sentiment in the proposal. Code like you example is the exception, and its better to make the feature useful in the majority case. If people can't do

if(foo.Bar == null) return;

foo.Bar.Baz();
foo.Bar.Bark();
foo.Bar.Items.Add(...);

without getting warnings all over the place they're just going to turn the feature off (or ignore the warnings) and never look back.

[jnm2]

@MgSam Poor example because calling foo.Bar.*() won't likely cause foo.Bar to become null. More likely:

if (foo.Bar == null) return;

foo.Baz();
foo.Bark();
foo.Bar.Items.Add(...);

[jnm2]

No lie, I just got bitten by this:

foo.Bar.Baz();
if (foo.Bar.Qux) // Bam, foo.Bar is null.
{

That's code I wrote in February.

[yaakov-h]

If we do, what does it take to invalidate assumptions about a dotted name? In principle, any intervening call can cause a property to change. Even another thread could do that!

FWIW, the behaviour I've observed from Code Contracts - and this behaviour works quite well in practice - is:

1. Ignore multi-threading. Assume a single thread when running flow analysis. (Perhaps we could be smarter and take special note of variables marked as volatile).
2. Any method can mutate any other property of the current object, unless there's a postcondition that guarantees same-ness, or the method is marked as [Pure]. In this case, dotting still works until the first non-Pure method invocation.
3. Property getters are assumed to not mutate the object.
4. Property setters remove any assumptions about the property's state, unless the value being assigned has the same assumptions (e.g. assumed non-null).

[mihailik]

I wonder if the only feasible way — is for C# compiler to analyze IL and flows of the binary DLL dependencies.

[Joe4evr]

can libraries declare non-null inputs and skip manual null checks?

Yes. Also, another part of the proposal is an optional mechanism to let the compiler generate a runtime check for your method parameters, if you really wanted to be sure.

can T generic parameter be decorated with !?

Uhm? You can put ? on the declaration, but not ! AFAIK.

real life code: BCL collections, RX, Mono.Cecil, ASP.NET — how much effort is to upgrade?

Probably quite a bit, but I take it that it's part of the effort. If nothing else, I can see MS starting off with the core types, and adding more annotations over time.

existing (non-null-aware) comm/serialization libraries in combination with null-aware classes;

Not all that much you can do here, I think. The main idea is that from the moment you get an object ready to consume that the annotations will show which properties can and cannot be expected to contain null. Serializers could update to take the new attributes into account, but overall I don't think it's that worrisome.

taking complex data between nullability-aware and legacy code — how much effort is it to me as developer?

IIRC, if you opt-in but one of your dependencies did not, that dependency is considered "null-oblivious" and the compiler won't give warnings when calling in/dereferencing returns from those APIs (exact same behavior as today).

[mihailik]

@Joe4evr lack of C# use cases leads to haphazard guesses. That's why it would be prudent for the compiler team to do that early.

I wouldn't want to discuss your assumptions on how it might work, as they paint a rather grim prospect (lots of effort to refactor existing code etc.) Hoping C# code examples would be sunnier.

[yaakov-h]

@mihailik:

This needs more lifelike use cases — pinpoint valuadd beyond simplistic 'billion dollar' meme

Does it really? Let me provide an example. I work on an extraordinarily large .NET enterprise application. For one project, we invested rather heavily in Code Contracts, primarily for the purposes of detecting null references early and preventing us shipping such bugs.

This has been incredibly valuable (though at this point, the cost of Code Contracts is starting to outweigh the benefits). I can count on my hands the number of times that we've had a NullReferenceException in production, and half of them were due to the Code Contracts rewriter inserting nulls into our IL, or accidentally removing code from our IL.

By contrast, NullReferenceExceptions make up a fair chunk of our automatic exception reports from other products. It's not enormous, but it's still time that could be better spent on other things.

Why would you want to pay developers to clean up mistakes in arrears when you can have the computer prevent that mistake in advance?

Similarly, I often have to dig into documentation or into implementation code to figure out if a function can actually return null, and if so, what I should do about it. But, when writing Swift for some small personal iOS projects, I have absolutely none of these concerns. Something is either Optional, or not. If it is, I have to unwrap it - it's part of the method signature, and I always have to check for None/nil, or if there's actually something inside. (Unless I know better than the compiler, then I can shut it up with !.)

After using a language with strict nullability rules for about 30 minutes, you really can't go back to languages where anything could be an object, or could be null, whoops we won't tell you until runtime.

Cost is palpable: inflicting this on real codebase would create pollution with ? and !

Would it really? The general case - and hence the problem with nulls - is that I expect a value to be non-null in a majority of cases. If you could have a look at the codebase I mentioned above, you'd see that almost every single reference typed field or parameter has Contract.Requires(foo != null) or Contract.Invariant(foo != null).

? would be reserved for cases where you know it can be null and accept it, which is not the common case.

! would be reserved for cases where you know it can be null, but is not. This should be fairly rare if the flow analysis is sophisticated enough to make this feature valuable and not cumbersome.

Adding ? or ! is a non-binding declaration of intent (other C# features of that nature: CLS, code contracts)

Code Contracts technically aren't binding unless you jump through a very long series of hoops. Also, it's not a "C# feature".

I agree on this point, however. I would love it if, like Swift, nullability was part of the method signature and runtime enforcable. However, I do acknowledge that doing so would fully break compatibility on every level, and is not worth the cost for .NET.

can libraries declare non-null inputs and skip manual null checks?

I would argue that this should only be done on public API surface (including private implementations of public interfaces), and perhaps should be left as manual. Once you're into the internals of a library, the compiler should have been able to prove already that your value is not null, so further checks are worthless.

[mihailik]

@yaakov-h totally 100% agree with your long journey with Contracts, and the ultimate judgement (and the general industry sentiment):

the cost of Code Contracts is starting to outweigh the benefits

By the way, thanks for your intuition/speculation, but can it replace a concrete C# example?

* can libraries declare non-null inputs and skip manual null checks? ...

Those are not questions to answer/argue in English on the thread — they are points to clarify in C# examples.

I would argue that this should only . . .

[Joe4evr]

Those are not questions to answer/argue in English on the thread — they are points to clarify in C# examples.

Is it really that hard to understand?

public void Foo(string! s) //compiler-generated null-check
{
}

[mihailik]

@Joe4evr now you're on the right track — would be awesome if your example looked more real, and showed valueadd.

[yaakov-h]

@mihailik

I would argue...

😄

I think regarding examples though, you're barking a bit up the wrong tree at this stage. You don't need a large complicated function to see what one small tweak would do and how it should behave. For flow analysis, perhaps, but not for syntactic markers.

[mihailik]

The beauty of nullability is too bright for mortal eyes.
Genesis 9:22

[jnm2]

Lol... wut...

[mihailik]

Real C# code examples for a C# feature is a reasonable requirement, neither lol nor wut nor barking at a tree.

[jnm2]

The last three things you had said still don't make sense together...

[HaloFour]

@mihailik

Real C# code examples for a C# feature is a reasonable requirement, neither lol nor wut nor barking at a tree.

That's what the strawman proposals are for, to test the various flavors of the implementation against real-world code bases.

This conversation is well past the point of gathering requirements and use cases. The value-add has already been established. We're at the point of hammering out the nitty gritty details and measuring the impact to developers who wish to opt-in.

[alrz]

9. Special methods

Is Debug.Assert a special method? consider this pre-NRT code,

void M(string o) { // or string?
  Debug.Assert(o != null);
  // ...
  var x = o.Length; // No warnings on debug?
}

Also it would be helpful to take existing null checks into account when we warn possible nulls,

void M(string o) {
  var x = o?.Length; // there is a null check in pre-NRT code so nulls are expected
  // ...
  var y = o.Length; // this is possible null dereference!
}

We could first suggest to convert string to string? but I'd expect to get warnings when I opted-in, without code changes.

[jnm2]

Debug.Assert is not present in release builds, yet I would rather always have the warning or it would take me by surprise.