Support sizeof(x) where x is a fixed buffer

[The-Futurist]

I'm astonished that sizeof(x) doesn't compile when x is a fixed buffer that's within scope. Since such buffers have compile time constant sizes it should be almost trivial to implement sizeof here, can this be considered?

The only way I can see is to reflect upon the containing type, get the field with the name of fixed buffer field, get its FixedBuffer attribute, get that attributes constructor args and eventually get at the declared length!

Such code (even if done statically once) is very cumbersome and fragile (should MS ever alter that FixedBuffer attribute or whatever). I can use a static dictionary keyed on containing type and fixed buffer field name and cache the length there (the first time its requested) but even so a dictionary lookup for a simple integer constant that's fixed and known at compile time? really?

(At least nameof works as expected! but sizeof and typeof give rather unhelpful error messages)

[The-Futurist]

So this is what developers have to write if they want to try and maximize performance of code using fixed buffers (which are often used mainly for performance reasons!)

    public static class FixedBuffer
    {
        private static ConcurrentDictionary<string, int> buffer_lengths = new ConcurrentDictionary<string, int>();
       
        public static int GetCapacity<T>(string FixedBufferName) where T : struct
        {
            var key = $"{typeof(T).Name}.{FixedBufferName}";

            if (buffer_lengths.TryGetValue(key, out int Capacity))
                return Capacity;

            var fields = typeof(T).GetFields().Where(f => f.Name == FixedBufferName);

            if (!fields.Any())
                throw new ArgumentException($"No field with that name exists within the type: {typeof(T).Name}", nameof(FixedBufferName));

            var args = fields.First().CustomAttributes.First().ConstructorArguments;

            buffer_lengths.TryAdd(key, (int)args[1].Value);

            return buffer_lengths[key];
        }
    }

Can we therefore consider adding sizeof support for fields that are declared as fixed [] ?

Since fixed buffers have a very contrived type definition (I personally think fixed buffers should be implemented as a kind of stand alone primitive type) its not possible to pass a type name into sizeof so the declared buffer identifier makes sense, in this case.

[juliusfriedman]

In the same scope... that wouldn't require a lookup as you created the buffer in that scope? You know it's size...

[The-Futurist]

@juliusfriedman

In the same scope... that wouldn't require a lookup as you created the buffer in that scope? You know it's size...

True but we must repeat the value everywhere we use that size creating fragility should anyone alter the size in one place and not others. Once can create constants and so on to address this but since its size is fixed compile time constant it strikes me as easy to deliver this.

[juliusfriedman]

Really, it seems like the fragility comes in the code which consumed it...

Provide an example IRL where this occurs as I can't even contrive it...

int mySize = fragility;

One would simply refer to mySize.

[The-Futurist]

@juliusfriedman

Take a look at this code (just posted here):

public interface IFixedBuffer
{
    int FixedBufferLength { get; }
}

public static FixedBufferExtensions
{
    public static ToString<T>(this T buffer) where T : IFixedBuffer
   {
       var length = buffer.FixedBufferLength;
       ...
   }
}

using System;

public unsafe struct AString_32 : IFixedSizeBuffer
{
    public fixed char chars[32];
    
    public ReadOnlySpan<Char> AsSpan()
    {
        fixed (char*  c = chars)
        {
            return new ReadOnlySpan<Char>(c, 64);
        }
    }

    public int FixedBufferLength => 32;
}

The 32 appears twice (actually three times), the 64 appears once, changing one necessitates changing the others. Then imagine you have many versions of this kind of type and the fragility becomes more apparent.

This could be coded as:

public interface IFixedBuffer
{
    int FixedBufferLength { get; }
}

public static FixedBufferExtensions
{
    public static ToString<T>(this T buffer) where T : IFixedBuffer
   {
       var length = buffer.FixedBufferLength;
       ...
   }
}

using System;

public unsafe struct AString_32 : IFixedSizeBuffer
{
    public fixed char chars[32];
    
    public ReadOnlySpan<Char> AsSpan()
    {
        fixed (char*  c = chars)
        {
            return new ReadOnlySpan<Char>(c, sizeof(Char) * sizeof(chars));
        }
    }

    public int FixedBufferLength => sizeof(chars);
}

This may seem to be grumbling BUT the buffer size is a known compile time constant, so why not?

[YairHalberstadt]

@Korporal

public interface IFixedBuffer
{
    int FixedBufferLength { get; }
}

public static FixedBufferExtensions
{
    public static ToString<T>(this T buffer) where T : IFixedBuffer
   {
       var length = buffer.FixedBufferLength;
       ...
   }
}

public unsafe struct AString_32 : IFixedSizeBuffer
{
    public fixed char chars[FixedBufferLength];
    
    public ReadOnlySpan<Char> AsSpan()
    {
        fixed (char*  c = chars)
        {
            return new ReadOnlySpan<Char>(c, FixedBufferLength * 2);
        }
    }

    public int FixedBufferLength => 32;
}

[juliusfriedman]

I don't follow:

1. sizeof(AString_32) is known and can be calculated.
2. Why doesn't AString_32 have a const size so it only have to be changed in one place? e.g. the const which would be used to declare chars...

[The-Futurist]

@YairHalberstadt - That cannot compile - the fixed buffer bounds [ ] must be a compile time constant.

[The-Futurist]

I don't follow:

1. sizeof(AString_32) is known and can be calculated.
2. Why doesn't AString_32 have a const size so it only have to be changed in one place? e.g. the const which would be used to declare chars...

@juliusfriedman

The code is sample only (it won't compile for other reasons) but post what you mean?

[YairHalberstadt]

@Korporal

public unsafe struct AString_32 : IFixedSizeBuffer
{
    public fixed char chars[LENGTH];
    
    public ReadOnlySpan<Char> AsSpan()
    {
        fixed (char*  c = chars)
        {
            return new ReadOnlySpan<Char>(c, FixedBufferLength * 2);
        }
    }

    public int FixedBufferLength => LENGTH;
    const int LENGTH = 32;
}

[The-Futurist]

The point I'm making is not that there are no alternatives but that the length/capacity of a fixed buffer is a fixed compile time constant and in this case it makes sense to be able to treat it as we can other types that also have a fixed compile time size.

In other words why is a fixed buffer the only fixed sized type that cannot be used with sizeof?

[The-Futurist]

@Korporal

public unsafe struct AString_32 : IFixedSizeBuffer
{
    public fixed char chars[LENGTH];
    
    public ReadOnlySpan<Char> AsSpan()
    {
        fixed (char*  c = chars)
        {
            return new ReadOnlySpan<Char>(c, FixedBufferLength * 2);
        }
    }

    public int FixedBufferLength => LENGTH;
    const int LENGTH = 32;
}

@YairHalberstadt - Of course that is an option, but we don't have to do this for other fixed size types.

[juliusfriedman]

@Korporal this also works but I would likely use the pattern @YairHalberstadt showed since as you stated the value is known at compile time.

 public interface IFixedBuffer
    {
        int FixedBufferLength { get; }
    }


    public unsafe struct AString_32 : IFixedBuffer
    {
        public fixed char chars[32];

        public ReadOnlySpan<Char> AsSpan()
        {
            fixed (char* c = chars)
            {
                return new ReadOnlySpan<Char>(c, sizeof(Char) * sizeof(AString_32));
            }
        }

        public int FixedBufferLength => sizeof(AString_32);
    }

    static void Main(string[] args)
    {
        Console.WriteLine(System.Runtime.InteropServices.Marshal.SizeOf<AString_32>()); //64

        AString_32 string32;

        Console.WriteLine(string32.FixedBufferLength); //64

        Console.ReadKey();
    }

[The-Futurist]

@juliusfriedman

@Korporal this also works

 public interface IFixedBuffer
    {
        int FixedBufferLength { get; }
    }


    public unsafe struct AString_32 : IFixedBuffer
    {
        public fixed char chars[32];

        public ReadOnlySpan<Char> AsSpan()
        {
            fixed (char* c = chars)
            {
                return new ReadOnlySpan<Char>(c, sizeof(Char) * sizeof(AString_32));
            }
        }

        public int FixedBufferLength => sizeof(AString_32);
    }

    static void Main(string[] args)
    {
        Console.WriteLine(System.Runtime.InteropServices.Marshal.SizeOf<AString_32>()); //64

        AString_32 string32;

        Console.WriteLine(string32.FixedBufferLength); //64

        Console.ReadKey();
    }

Until one adds other fields to the struct...

[juliusfriedman]

@Korporal, if I added more fields my code would reflect the mathematical paradigm to allow the offset to be calculated e.g. offset-of or otherwise.

[The-Futurist]

@Korporal, Tis a small world after all ;)

But.. there is no way sizeof support alone can allow one to pass pointers to structures... right?

There is the unmanaged constraint which is proposed to do that as far as I know but I would suspect in such cases there is little difference from passing a fixed buffer or any other type there when you think about it as the call site will only pass the pointer and the invocation sees only the pointer with no knowledge of size, one would still need to pass the length.

@juliusfriedman Not sure I understand you Julius. The unmanaged constraint lets us take a pointer to a struct generically (prior to it this was not possible) this is a huge step forward (despite a bug I found recently).

[juliusfriedman]

@Korporal, not sure how to further explain it...

@HaloFour, I believe that is a much better approach!

I am also pretty sure there is a proposal for that as well.

[CyrusNajmabadi]

Do the compiler/language teams have such a thing as a "low hanging fruit" category.

Yes. It's called: a LDM member wants to champion this. Meaning, they've assessed all of that and made such a determination.

[CyrusNajmabadi]

I can see there being support for treating a fixed buffer as a span via implicit conversion. That would give you an easier API for working with these buffers (including treating fixed char buffers like strings) as well as easy access to their length.

Agreed. I would definitely support such a proposal if it were created.

[tannergooding]

Agreed. I would definitely support such a proposal if it were created.

Implicit or explicit conversion to Span should probably just be tacked on as part of the support for the already championed/general-purpose fixed-sized-buffer proposal (https://github.com/dotnet/csharplang/blob/master/proposals/fixed-sized-buffers.md).

[HaloFour]

@tannergooding

Implicit or explicit conversion to Span should probably just be tacked on as part of the support for the already championed/general-purpose fixed-sized-buffer proposal

Should a proposal be created at least to track the work to add it to that other proposal? Otherwise we're just all relying on someone remembering to bring it up at some LD meeting? 😁

[The-Futurist]

Out of curiosity where approx in the Roslyn source is the parsing and validation of sizeof(X) expression? I see too that sizeof is represented as an IL instruction (wonder why it wasn't just replaced at compile time with a constant).

Just curious about what I would need to change if I were to (experimentally) implement this in a test build of the compiler.

Thx

[CyrusNajmabadi]

Hey @Korporal . Parsing is handled here:

https://github.com/dotnet/roslyn/blob/master/src/Compilers/CSharp/Portable/Parser/LanguageParser.cs#L9980-L9988

Semantic analysis and binding of that expr happens in:

https://github.com/dotnet/roslyn/blob/3bc2bdea33e1da702a7203afe036d8575e931f89/src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs#L1139

Actual handling of it in terms of IL seems to happen here:

https://github.com/dotnet/roslyn/blob/3bc2bdea33e1da702a7203afe036d8575e931f89/src/Compilers/CSharp/Portable/CodeGen/EmitExpression.cs#L2964-L2969

I hope this helps!

[CyrusNajmabadi]

(wonder why it wasn't just replaced at compile time with a constant).

Because the compiler doesn't know the size the type is outside of some basic types. There is no relation between the IL you compile against and the IL you run against, so the type may have a totally different size at runtime.

[The-Futurist]

(wonder why it wasn't just replaced at compile time with a constant).

Because the compiler doesn't know the size the type is outside of some basic types. There is no relation between the IL you compile against and the IL you run against, so the type may have a totally different size at runtime.

@CyrusNajmabadi

OK I understand - to an extent - (platform agnostic). But surely a Byte is universal as is Int16 and so on? I personally don't know of any hardware that has a byte that isn't 8 bits or a 16 bit signed it that isn't 16 bits, am I missing something?

[tannergooding]

But surely a Byte is universal as is Int16 and so on

At least as far as the C# spec is concerned, the various primitives are constant and are handled that way: https://github.com/dotnet/csharplang/blob/master/spec/unsafe-code.md#the-sizeof-operator

However, all other structs are just handled via the sizeof IL instruction; which the JIT will treat as a runtime constant anyways. And which, outside of some specific scenarios which the C# language spec currently doesn't define, can only be a runtime constant due to platform-specific differences (generally layout and padding).

[The-Futurist]

@tannergooding - Thanks for clarifying!

[CyrusNajmabadi]

OK I understand - to an extent - (platform agnostic). But surely a Byte is universal as is Int16 and so on? I personally don't know of any hardware that has a byte that isn't 8 bits or a 16 bit signed it that isn't 16 bits, am I missing something?

As mentioned: Because the compiler doesn't know the size the type is outside of some basic types

'byte' would be one of those basic types.

[CyrusNajmabadi]

Weird... why did my post show up so much later...

[GSPP]

I just had a real-world case where this would have been useful:

        const int MAX_PATH = 260;

        [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
        unsafe struct WIN32_FIND_DATA
        {
            //...
            fixed char _cFileName[MAX_PATH];

            public string GetFileName()
            {
                fixed (char* ptr = _cFileName)
                {
                    var length = 0;
                    for (int i = 0; i < MAX_PATH; i++)
                    {
                        if (ptr[i] == '\0')
                        {
                            length = i;
                            break;
                        }
                    }

                    return new string(ptr, 0, length);
                }
            }
        }

Here, MAX_PATH appears twice. I'd rather have written:

for (int i = 0; i < sizeof(_cFileName) / sizeof(char); i++)

The intention of this for loop is not to iterate MAX_PATH times. The intention is to iterate over the whole buffer regardless of its size. If I could have used sizeof, the intent would have been more clear.

Also, any changes to the size of the buffer would no longer require changes in two places.