Smoother generics with Nullable<> and NRT

[AndreasHoffmann2]

Hi,

I have used and loved generics for a long time now. But when it comes to NRT, I often get the feeling that the language does not offer the full potential yet. There is a lot of friction between the world of nullable structs and NRT. And IMO, this should be reduced as much as possible.

One example is generic code that should be able to handle "everything nullable":

private class CollectNotNull<T> {
	private List<T> ValuesInternal = new();
	public void MaybeAdd(T? key) {
		if (key != null) {
			ValuesInternal.Add(key);
		}
	}
	public IEnumerable<T> Values => ValuesInternal;
}

It is a class that should collect all values that are not null. But how do I use it with a struct?

private void Usages() {
	int? i = 0;
	CollectNotNull<int?> c1 = new();
	c1.MaybeAdd(i);
	IEnumerable<int> values1 = c1.Values; //Error: the return-type is IEnumerable<int?> 

	CollectNotNull<int> c2 = new();
	c2.MaybeAdd(i); //Error: Nullable<int> is not assignable to int.
	IEnumerable<int> values2 = c2.Values;
}

It just does not work. Or am I overlooking something?

Things get even worse when I use a dictionary inside CollectNotNull like Dictionary<T, object>. Then, I need to add a constraint: where T : notnull, which does exclude the definition of c1 already.

I am by no means an expert on the way, this could be changed. I am just a user reporting what I feel is a weakness. A weakness that forces me to duplicate code.

The example above is just an artificial example. I met this kind of restriction at different points in the real world.

[HaloFour]

This is an accepted aspect of the design of NRTs as they had to be necessarily different from NVTs and that does affect how they work with generics. With NVTs the wrapper struct Nullable<T> is required in order to convey the additional nullity information. With NRTs the type already implies nullability and the type could not be changed without breaking existing code. So while string? and string are technically the same type, int? and int are two different types as far as the runtime is concerned. I'm not sure what options might exist for trying to smooth that out which wouldn't potentially incur serious breaking changes.

[AndreasHoffmann2]

I'm not sure about the possibility of non-breaking changes either. My first thought was to let the compiler generate two classes and choose depending on the generic type argument.
But this does not scale well, if you have many such type arguments: If you have 10 like this, this would be 1024 classes :-(

[RenderMichael]

As long as a new keyword is used (i.e. where T : nullable, boiling down to where T : class? or Nullable<>) I don't quite see the breaking change here

[TonyValenti]

I am Curious - why is Nullable constrained to structs?

[CyrusNajmabadi]

Because it's intrinsic to the runtime. The runtime will actually recognize null as a value for it, despite nullable itself being a struct. So Nullable literally is a way of deeply supporting the entire underlying struct values plus exactly one more for null.

With reference types that's non sensical. Null is already in the domain of the reference type, so Nullable couldn't add it as well. And it would be entirely ambiguous what null was. Is it that the nullable has no value? Or it has a value and the value is null.

[HaloFour]

I think it could still make sense but only if the runtime would guarantee that the nullity of the wrapper struct was determined by the nullity of the reference.

Nullable<string> nullable = new Nullable<string>("foo");
Debug.Assert(nullable.HasValue == true);

Nullable<string> nullable = new Nullable<string>(null);
Debug.Assert(nullable.HasValue == false);

Nullable<string> nullable = (Nullable<string>) null;
Debug.Assert(nullable.HasValue == false);

This could help with generic situations specifically but I think the ship has sailed for NRTs in general, unless the runtime could treat string and Nullable<string> as the same type in some way.

[FaustVX]

That could help a lot with nullable generic with may be just a single method void Method<T>(T? value) {} instead of constraining the generic with where T : struct and where T : class (which can only works when there is a T? parameter)

[TonyValenti]

I was curious about nullable because in my code I sometimes will use a generic "box" class that is basically the same as nullable except it is a class and it is not constrained to structs.

This allows me to treat classes and structs the same way.

[fabianoliver]

I agree the current solution struggles a bit with generics in particular. I haven't through this through too deeply yet, but would it make sense to have some syntax to refer to the non-nullable version of a generic type?

For the sake of the example, say it's just the type parameter followed by an exclamation mark - might have to choose something else if this clashes with existing use cases for null forgivability etc., but anyway. Let's say the compiler does the following:

1. T being Nullable<X>, T! would translate for X. Example: T = Nullable<int>, then T! = int
2. T being a plain value type X, then T! translates to X. Example: T = int, then T! = int
3. T being a nullable reference type X?, then T! translates to X. E.g. T = string?, then T! = string
4. T being a non-nullable reference type X, then T! translates to X. E.g. T = string, then T! = string

Then maybe you could have something like

static IEnumerable<T!> WhereNotNull<T>(this IEnumerable<T> items)
{
  foreach(var item in items)
    if (item is not null)
      yield return (T!)item;
}

Which I think would enable usage such as:

  IEnumerable<int> a = Enumerable.Empty<int>().WhereNotNull();
  IEnumerable<int> b = Enumerable.Empty<int?>().WhereNotNull();
  IEnumerable<string> c = Enumerable.Empty<string>().WhereNotNull();
  IEnumerable<string> d = Enumerable.Empty<string?>().WhereNotNull();

Similarly (even if the ship might have already sailed for this one), maybe instead of where TKey : notnull, we could have

interface IDictionary<TKey, TValue>
{
  ICollection<TKey!> Keys {get;}
  void Add(TKey! key, TValue value);
  // .. etc ..
}

[HaloFour]

I think the biggest issue with this is that the signature can't support the type being conditional based on the generic type arguments. The return type can't be int sometimes and int? other times. The body within the method cannot be different based on the generic type arguments either, but it would have to be in order to support int vs. int?.

[HaloFour]

Hmm, what exactly is "different"?

The C# language hides the fact that (int) item means two different things if item is int vs. int?. For the former it's elided away as there is no need for a cast, but in the latter case it's a method call to the Value property get accessor. So the IL in those two cases ends up being necessarily different.

[iam3yal]

One of the features that is really missing from generics is generic specialization both at the type level and constraint level that part of it is discussed here #6308.

[TahirAhmadov]

Having discussed this at length, extending Nullable<T> to allow classes, or creating a new type which can be used for unconstrained generic nullables, addresses some scenarios, but not others; like @CyrusNajmabadi had pointed out, a List<T?> list field is not compatible with List<string?> when T is string.
I think the only "real" solution would be what @AndreasHoffmann2 said here (and I had previously suggested myself). And just like he wrote, it causes 2^X versions to be generated, where X is the number of generic arguments, but only those which are used to create a nullable variable of some sort; meaning, if there are 10 unconstrained generic arguments, but only 2 are used like T1? and T2?, then we only need 4 versions. The problem isn't even the # of versions, but the entire system of having generic code with multiple versions; this would be an involved addition to the runtime. Another difficulty is that there are complicated generic nesting rules already in place; nested types inherit generics, and methods can have their own generic arguments; it would be expensive to make it so that all of that stuff works reliably.

[AartBluestoke]

would be possible for the T! in "return (T!)item;"
to compile to
....
return (T)item is Nullable<> a? a.Value:item;

as long as the JIT can optimise the always true or always false cast based on the template type, it should come with no substantial runtime cost ...