[Proposal]: [Zero-length fixed buffers]

[john-h-k]

Zero-length fixed buffers

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: Not Started

Summary

Allow fixed-buffers to have a length of 0 if the fixed buffer is the final field in a struct of a sequential layout. Currently this code generates CS1665: Fixed size buffers must have a length greater than zero.

unsafe struct Value
{
    public fixed byte Buffer[0];
}

Motivation

This is a common scenario when working with native code (as is one of the most frequent uses of fixed-buffers in general).
It also is useful in some high-performance data stream designs.

For example, in a data stream

internal struct CommandClearDepthStencil
{
    public ViewHandle View;
    public float Depth;
    public byte Stencil;
    public DepthStencilClearFlags Flags;

    public uint RectangleCount;
    // rectangles or
}

// Addressing done as
int* pFirstRectangle = (int*)(&cmd->RectangleCount + sizeof(uint));

The alternative to the above design (using a 1-sized fixed buffer) leads to incorrect sizeof values.

Could be replaced with

internal struct CommandClearDepthStencil
{
    public ViewHandle View;
    public float Depth;
    public byte Stencil;
    public DepthStencilClearFlags Flags;

    public uint RectangleCount;
    public fixed int Rectangles[0]; // or Rectangles[];
}

// Addressing done as
int* pFirstRectangle = &cmd->Rectangles[0];

Detailed design

This would legalise having a fixed buffer length of size 0 when the containing type is a struct with sequential layout and the fixed buffer is the final field of the type. If the type was auto layout, or the buffer was not the final field, it would still generate CS1665.

Drawbacks

Niche feature. Can be confusing.
They require very special handling, and can easily break. However, this should be clear via the unsafe requirement, and they can't cause easily cause managed heap corruption (which seems to be the requirement for unsafe code). Implicit compiler copies will cut the struct and remove any excess data, so using these values requires extreme care. However, this is already the case in their uses cases, they just generally use a size of 1 or no field, and manually address, which is arguably more bug-prone.

Alternatives

Alternative syntax could be the same as C++ flexible-array members - so it would have no length specifier (but effectively be the same as 0 for all intents and purposes).

Not doing it is not particularly high impact. It is just a relatively low-effort change that helps some native interop scenarios.

Unresolved questions

Should it be allowed in explicit-layout structs?

Design meetings

N/A

[tannergooding]

The alternative to the above design (using a 1-sized fixed buffer) leads to incorrect sizeof values.

It might be good to know how often this is a "correctness" thing. That is, how often is length "actually" zero.

Due to these types being "variable length", you can't safely represent them in the type system (even in C/C++) and so therefore doing anything with these types "by value" will lead to incorrect or undesirable behavior.

Even in C/C++, such types are effectively required to be created via malloc, to be copied via memcpy, and to always be passed/handled via pointers because the type doesn't correctly take into account that there are RectangleCount (or RectangleCount - 1, in the case the fixed buffer uses 1 instead of 0) fields after the end of the struct.

So you will always need semi-custom logic to account for this and the only real question is whether or not those n extra bytes actually matter for the case where Count is really 0 and that is a "valid" state

[KalleOlaviNiemitalo]

Would the following structure and the Tail array be aligned to a multiple of 64 bits?
If so, how would the alignment requirement be represented in metadata?

struct Flex
{
    internal IntPtr Head;
    internal fixed long Tail[0];
}

[tannergooding]

It would follow the default packing for those types on a given platform. Most currently use 8-byte packing for long, but some may use 4-byte packing instead.

You can always override to a smaller (but not a larger) size using [StructLayout(LayoutKind.Sequential, Pack = #)]

[acaly]

It also is useful in some high-performance data stream designs.

I agree. I have encountered similar problems. Since C# supports Span, it would be attractive to serialize some unamanged data into a large byte array by simply copying with Span. However, you must always calculate the size of an element if that element may contain such a fixed buffer, and the (int*)(&cmd->RectangleCount + sizeof(uint)); doesn't seem too complicated compared with those other size calculations you would have to write.

[dgrunwald]

The address calculation with (int*)(&cmd->RectangleCount + sizeof(uint)) doesn't work if there's padding between RectangleCount and the array.

[KalleOlaviNiemitalo]

Currently, compiling this

using System;

unsafe struct Flex
{
    internal IntPtr Head;
    internal fixed long Tail[3];
}

results in:

.class private sequential ansi sealed beforefieldinit Flex
       extends [mscorlib]System.ValueType
{
  .class sequential ansi sealed nested public beforefieldinit '<Tail>e__FixedBuffer'
         extends [mscorlib]System.ValueType
  {
    .pack 0
    .size 24
    .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 ) 
    .custom instance void [mscorlib]System.Runtime.CompilerServices.UnsafeValueTypeAttribute::.ctor() = ( 01 00 00 00 ) 
    .field public int64 FixedElementField
  } // end of class '<Tail>e__FixedBuffer'

  .field assembly native int Head
  .field assembly valuetype Flex/'<Tail>e__FixedBuffer' Tail
  .custom instance void [mscorlib]System.Runtime.CompilerServices.FixedBufferAttribute::.ctor(class [mscorlib]System.Type,
                                                                                              int32) = ( 01 00 59 53 79 73 74 65 6D 2E 49 6E 74 36 34 2C   // ..YSystem.Int64,
                                                                                                         20 6D 73 63 6F 72 6C 69 62 2C 20 56 65 72 73 69   //  mscorlib, Versi
                                                                                                         6F 6E 3D 34 2E 30 2E 30 2E 30 2C 20 43 75 6C 74   // on=4.0.0.0, Cult
                                                                                                         75 72 65 3D 6E 65 75 74 72 61 6C 2C 20 50 75 62   // ure=neutral, Pub
                                                                                                         6C 69 63 4B 65 79 54 6F 6B 65 6E 3D 62 37 37 61   // licKeyToken=b77a
                                                                                                         35 63 35 36 31 39 33 34 65 30 38 39 03 00 00 00   // 5c561934e089....
                                                                                                         00 00 ) 
} // end of class Flex

If the source code had a zero-length fixed buffer internal fixed long Tail[0] instead, then would the metadata just replace .size 24 with .size 0 but preserve .field public int64 FixedElementField? In which case, FixedElementField would then extend beyond the size of <Tail>e__FixedBuffer and might require some validation to be disabled in the CLR.

If that's not how the metadata would be implemented, then how would the alignment requirement of Tail be represented?