Uniform Array Initialization

[jeffanders]

Uniform Array Initialization

C# already has a number of different array creation expressions. However, none of them work well with possible future support for non-nullable references for variable length arrays.

I propose a new form of array creation expression that can be used in general or specifically with non-nullable reference to guarantee that a variable length array is initialized with non-null references. The uniform array creation expression takes the following form:

new T[n] expression;

For example:

string[] GetWords(int wordCount)
{
    return new string[wordCount] "Hello";
}

If non-nullable references, as currently proposed, were enabled for this assembly then this will return an array of non-nullable strings of length wordCount where every element is a reference to the string "Hello".

This by itself does not seem particularly useful but used as a basis for building an array by starting with a non-nullable "default" value and replacing individual elements with other non-nullable references we can have the compiler provide the same sort of non-nullable guarantees for every element of an array created using this syntax.

To give a more relevant example, imagine a function that takes an array of nullable objects and returns an array of non-nullable objects and for performance reasons we want to guarantee that this function does not allocate more than necessary. Using this form of array creation this function could be implemented as follows:

object[] ExcludeNulls(object?[] objects)
{
    int nonNullCount = 0;
    object? firstObject = null;
    foreach (object? o in objects)
    {
        if (o != null)
        {
            nonNullCount++;
            if (firstObject == null)    
                firstObject = o;
        }
    }

    if (firstObject != null) // By flow analysis C# would treat firstObject as non-nullable inside if
    {
        object[] returnObjects = new object[nonNullCount] firstObject;
        nonNullCount = 0;
        foreach (object? o in objects)
        {
            if (o != null) // again, o is non-nullable inside if
                returnObjects[nonNullCount++] = o;
        }
        return returnObjects;
    }

    return new object[0] {};
}

At no point in the above code was it necessary to perform any cast to throw away nullability but we have ended up with an array of guaranteed non-nullable objects.

Drawbacks

There are a number of drawbacks to this approach.

1. This potentially trades pervasive nulls in arrays with pervasive object aliasing where the same object is present at multiple indices within the array. This is not a problem for immutable objects like string, but for mutable objects any mutable calls performed element-wise that are not idempotent could lead to possibly even more obscure errors than a NullReferenceException. Usually careful construction of the array should avoid this problem, but logic errors could unintentionally leave multiple references to the "default" element at multiple indices.
2. This will typically require some sort of gymnastics to both establish a base element value with which to initialize the array and the number of elements. Although it should be noted, excluding coding style, the ExcludeNulls function above would not really be any different in C# today under the constraint that it be minimally allocating.

[bondsbw]

Perhaps this could have an implicitly typed form:

new[n] expression;

e.g.

new[3] "Hello"; // equivalent to new[] { "Hello", "Hello", "Hello" };

[jeffanders]

@bondsbw Definitely, I had imagined that as a natural extension of this proposal. This should work for all "default" expressions except for null. That is to say that new[3] null would be an error and the intent could already be achieved today via new object[3]

[theunrepentantgeek]

You explicitly call out the aliasing drawback, with the (reasonable) assumption that the same reference would be used for all array members.

Prior to reading the drawbacks, however, I'd already written code like this in my head:

var buckets = new HashBucket[10] new HashBucket();

This would then end up with the same HashBucket() for every element of the array, not quite the intent - and it would be challenging to spot the issue.

You'd almost want to restrict this for use with immutable types only.

[brandon942]

I like the performance benefits of the idea as it can avoid writing the memory twice unnecessarily if done right (first time with zeros on allocation, second time with desired values). It'd work well with IEnumerble.

var a = new int[20] GetValues();

[jeffanders]

@brandon942 unfortunately I don't think this will have the performance benefits you are hoping for. I believe it is the CLR that guarantees allocated memory will be zeroed and while the C# language has similar guarantees for member variables it leans on the CLR to fulfill that guarantee as far as I understand it. Having said that the zeroing most likely occurs in a background GC thread so that allocations take place from an already zeroed heap region and therefore have near zero amortized cost.

Your idea of being able to initialise an array from IEnumerable is interesting but I think the problem would be that when allocating the array you indicate how many elements it will contain and there is no guarantee the IEnumerable will have exactly that many values (or a finite number of values for that matter). So I think the Enumerable.ToArray() extension method will continue to fulfill that functionality.

[brandon942]

@jeffanders Are you sure the GC pre allocates memory for "potential" use? I find that hard to believe. Allocations by user code would have almost no cost then if that was true. What if you allocate a bigger memory chunk then what is pre-allocated?
The default IEnumerable behavior can be to either to IEnumerator.Reset() and repeat when the end of the IEnumerable is reached or just zero the rest of the memory. If you fill all the memory and the IEnumerable has not ended you just stop enummerating.

[jeffanders]

@brandon942 Yes, the garbage collector reserves memory in heap segments that it pre-zeroes and when it performs heap compaction it zeroes the compacted free space. So allocations are near zero cost involving bump pointer allocation. So either there is enough free space to allocate or a compacting GC occurs or extra heap segments are created. So while allocation is cheap, high rates of allocation, large objects or large live object graphs put pressure on the GC requiring a lot of memory scanning amd rewriting.

See https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/fundamentals in particular

Reclaims objects that are no longer being used, clears their memory, and keeps the memory available for future allocations. Managed objects automatically get clean content to start with, so their constructors do not have to initialize every data field

[theunrepentantgeek]

Looping and repeating to fill the array would be somewhat startling to many developers. Also, not every implementation of IEnumerator actually implements .Reset() at all (it's use is currently fairly rare, so in most cases this is not an issue). Some implementations are inherently non-resettable (e.g. streaming data from a network socket or database connection).

[brandon942]

@theunrepentantgeek Reset isn't necessary anyway. If you have counted the size of the IEnumerable you can just copy what you have written.