Proposal: "Field Pointers"

[dmitriyse]

Currently the next code is not allowed:

    public class MyClass
    {
        private ref int _myIntRef;
    }

This feature can save some amount of "virtual calls" in a performance critical parts. So some code can receive more than 10x speed-up.

Probably no any change to CLR is required.
That how it's possible:

internal struct ByReferenceInObject<T>
{
        /// Object should be stored to allow implicitly pin memory
        /// Type of object does not play any role, just any reference-type.
        private Object _obj;

        /// Offset of the T from the start of _obj memory.
        private uint _offset;

        // ...

        public ref T Value
        {
            [Intrinsic]
            get
            {
                throw new NotSupportedException();
            }
        }
}

// Example:
public class MyClass
{
      private ByReferenceInObject<int>  _myIntRef;
}

/// Or with syntax sugar
public class MyClass
{
      /// objref - is a new keyword.
      private objref int  _myIntRef;
}

// Or with even more syntax sugar
public class MyClass
{
      private &int  _myIntRef;
}

Speedup example:

/// ------------------- Currently available "slow" sloution: -----------------------
public interface IIntStorage
{
    void SaveResult(int value);
}

/// This method should works without knowlage of int storage implementation.
/// But it's so performance critical place where every IL instruction took into account.
/// Assuming that this method cannot be inlined, just to avoid non-relative suggestions.
public void DoSomePerformanceCriticalOperation(IIntStorage resultHolder)
{
        int i = 4 * 42; // Calculating modern answer to everything.

        // OMG! VIRTUAL CALL. Performance disaster,
        resultHolder.SaveResult(i);
}

public class A: IIntStorage
{
      private string _myStringField1;
      private int _myResult:
      public void SaveResult(int value)
      {
           _myResult = value;
      } 

      public void DoSomething()
      {
           /// ...
           DoSomePerformanceCriticalOperation(this);
           /// ...
      }
}

public class B: IIntStorage
{
      private Url _myUrl;
      private bool _someFlag;
      private int _myResult;
     
      public void SaveResult(int value)
      {
           _myResult = value;
      }

      public void DoAnotherThing()
      {
           /// ...
           DoSomePerformanceCriticalOperation(this);
           /// ...
      }

}

/// --------------- Fast version based on the new "reference/pointer" type ----------
/// Assuming that this method cannot be inlined, just to avoid non relative suggestions!
public void DoSomePerformanceCriticalOperation(objref int result)
{
       // No any virtual call just "memory write"
       result = 4 * 42;
}

public class A
{
      private string _myStringField1;
      private int _myResult:

      public void DoSomething()
      {
           // ...

           // Yep, this is C++-like reference over GC object.
           // C# compiler should know offset of _myResult field relative to the start of the object,
           // otherwise additional CLR support is required.

           DoSomePerformanceCriticalOperation(&_myResult);

           // ...
      }
}

public class B
{
      private Url _myUrl;
      private bool _someFlag;
      private int _myResult;

      public void DoAnotherThing()
      {
           // ...

           // Yep, this is C++-like reference over GC object.
           // C# compiler should know offset of _myResult field relative to the start of the object,
           // otherwise additional CLR support is required.
           DoSomePerformanceCriticalOperation(&_myResult);

           // ...
      }
}

Update (03/21/2018) :
The closest possible implemenation with the current C#:

    using System;
    using System.Runtime.CompilerServices;
    using System.Runtime.InteropServices;
    using System.Threading.Tasks;

    [StructLayout(LayoutKind.Auto)]
    public struct ByRefInObject<T>
    {
        private object _object;

        private  int _fieldOffset;

        /// <summary>
        /// This is a workaround constructor, should be used until C# support or IL intrinsic support.
        /// </summary>
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        unsafe public ByRefInObject(object @object, ref T fieldRef)
        {
            _object = @object;

            // Hope that JIT will infer this method as a "partial interruptable" and will not perform GC in-between.
            _fieldOffset = (int)((ulong)Unsafe.AsPointer(ref fieldRef) - *(ulong*)Unsafe.AsPointer(ref @object));
        }

        /// <summary>
        /// This constructor should be used by a compiler or with IL intrinsic "ldfla" to get field address.
        /// </summary>
        unsafe public ByRefInObject(object @object, int fieldOffset)
        {
            _object = @object;
            _fieldOffset = fieldOffset;
        }

        public object Object => _object;

        public int FieldOffset => _fieldOffset;

        unsafe public ref T Value
        {
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            get
            {
                // TODO: Implement me in IL without Unsafe.

                // Hope that JIT will infer this method as a "partial interruptable" and will not perform GC in-between.
                return ref Unsafe.AsRef<T>(*(byte**)Unsafe.AsPointer(ref _object) + _fieldOffset);
            }
        }
    }

    public class MyTest
    {
        public int MyField;
    }

    public class Program
    {
        static void Main(string[] args)
        {
            var myTest = new MyTest();
            var myTransport = new Tuple<ByRefInObject<int>>(new ByRefInObject<int>(myTest, ref myTest.MyField));
            Task.Factory.StartNew(
                () =>
                    {
                        myTransport.Item1.Value = 100;
                    }).GetAwaiter().GetResult();

            // Prints:
            // 100
            Console.WriteLine(myTest.MyField);
        }
    }

WIth an implementation of ByRefInObject on the IL, the constructor and the "ByRefInObject.Value" property will be able to fully inlined by the JIT in the calling code and give ability to speedup code even before C# language support.

OMG this struct is already can be added to System.Runtime.CompilerServices.Unsafe package!
TODO: Post issue to CoreFX repo.

P. S.
Optimizations like type generic methods, devirtualization, inlining, etc. in many cases cannot be applied.
For example event loops: you cannot specialize event handlers.

[jnm2]

So, basically Span<T> with length 1? Since what you're asking requires not only a C# change but also a CLR change, would your purposes be served if you could use ByReference<T>?

[HaloFour]

I don't see what the "fast version" of this proposal is doing that couldn't be done today with your normal ref parameter. There is no storage of the ref in a field, just passing of the field address so that it can be manipulated directly. That's been supported since C# 1.0.

As for the broader subject of ref fields, that is a much more complicated beast addressed in part by "ref-like types". This is intentionally severely limited to ref struct types as this requires special CLR and compiler support and the compiler must ensure that the ref can't outlive what its referencing.

It might be nice to have a simple way to go from ref T to a Span<T>, though.

[dmitriyse]

So, basically Span with length 1? Since what you're asking requires not only a C# change but also a CLR change, would your purposes be served if you could use ByReference?

No, "purposes" is a performance. Virtual call is very expensive on the instruction level optimization. So working with pointers is up to 10 times is faster.

Span with length 1 is just a sort of ref-like value type, so it's can be stored only inside calling stack, not in the GC object.

[dmitriyse]

I don't see what the "fast version" of this proposal is doing that couldn't be done today with your normal ref parameter. There is no storage of the ref in a field, just passing of the field address so that it can be manipulated directly. That's been supported since C# 1.0.

Getting pointer to a value type field is supported since C# 1.0, but only with object pinning.
That pointers cannot be "saved" inside another object.
So the key suggestion of this proposal to store not the 64/32 bit value of direct memory pointer, but "GC object reference + offset".
Size of the new proposed pointers is 96 (64+32) / 64 (32+32) bit exactly to workaround volatility of GC objects memory location.
Whenewer you want to read or write to the target of 96/64 bit pointer you pin (implicitly) GC object and locating requered target field by the offset part of the pointer.

Writing/Reading through such pointers is a faster operation than call to some virtual method that will do the work.

[jnm2]

So the key suggestion of this proposal to store not the 64/32 bit value, but "GC object reference + offset"

I think this is exactly what Span<> does.

Virtual call is very expensive on the instruction level optimization. So working with pointers is up to 10 times is faster.

Using Span<> doesn't incur virtual calls, does it? It's a JIT intrinsic. Either way, refs must be slower than pointers because refs aren't necessarily pinned.

[dmitriyse]

@jnm2

    public class MyTest
    {
        private Span<int> _myRef; // Error CS8345 Field or auto-implemented property cannot be of type 'Span<int>' unless it is an instance member of a ref struct.	
    }

[HaloFour]

@dmitriyse

That pointers cannot be "saved" inside another object.

Indeed, that would be inherently unsafe. The CLR was designed to forbid this and the current "ref-like" restrictions coupled with Span JIT intrinsics are designed very specifically to enable very narrow but proven safe scenarios.

So the key suggestion of this proposal to store not the 64/32 bit value, but "GC object reference + offset".

Your proposal includes zero implementation details. If that's how you envision something like this being implemented I'd suggest actually including that in your proposal.

Writing/Reading through such pointers is a faster operation than call to some virtual method that will do the work.

I don't see what virtual calls have to do with any of this. The fact that you've decided to use interfaces to abstract storage is a particular detail of your solution.

[mikedn]

internal struct ByReferenceInObject

This should probably be first proposed in the corefx repository as it may be possible to implement this without any special language support. It also tends to be related to features like Span and Memory (Memory is rather similar except it only works with arrays).
Once the framework agree to add such a feature language support in the form of a dedicated keyword may be added if the feature is found to be very useful (I for one doubt that this is useful enough to deserve a keyword).

[BreyerW]

It was already requested by someone from Unity because they need something like that for their new and shiny EntityComponentSystem which was designed with performance in mind but i cant find exact thread, i dont remember title nor author and looking for it by hand is too much with so many open issues.

Now more interesting part is that i belive in the same thread @VSadov threw extremely loose idea of reference that live only on heap and only there (exact opposite of ´Span´) which would imply it could be stored on fields but likely couldnt point to any local variables, only to another field of class or field of struct that is already stored in class. This should fullfil your need in most cases. I might be wrong with these assumpions though since it was @VSadov idea and he didnt bother to explain in more details what he had in mind with references living on heap only.

EDIT found it #1147 and it was Vsadov not CyrusNajmabadi

[dmitriyse]

That pointers cannot be "saved" inside another object.
Indeed, that would be inherently unsafe.
DotNet/C# is already unsafe!
I can store any pointer at any place through IntPtr.

This proposal is exactly about to go through the same way (design very specifically to enable very narrow but proven safe scenarios ....) as with "ref" variables but without stack-only live limitation.
please forgot about Span.
we already have this:

    public class MyTest
    {
        private int _myField;

        public ref int GetRef()
        {
            return ref _myField;
        }
    }
public void Main()
{
       var t = new MyTest();
       
       // It's totally safe.
       ref int r = ref t.GetRef();
       r = 5;
}

But in this case "ref int r" is a 64/32 bit direct pointer under the hood and "stack only" limitation save us from invalid memory read/write.

@HaloFour
Let's discuss the proposal but not possible workarounds and properties of the current solutions.

This is what is possible now without additional C# or probably CLR support:
Lets discuss how to solve !!!PROBLEM!!! described here and implement the proposal:

    using System.Runtime.CompilerServices;
    using System.Runtime.InteropServices;
    using System.Threading.Tasks;

    public struct ByReferenceInObject<T>
    {
        private object _obj;

        private uint _offset;

        public ByReferenceInObject(object obj, uint offset)
        {
            _obj = obj;
            _offset = offset;
        }

        public ref T GetRef()
        {
            unsafe
            {
                // !!! THE PROBLEM !!!: Somehow we needs receive object pointer with implicit pinning 
               // Span can do this !!!!
                IntPtr p;
                // THATS WRONG:
                GCHandle h = GCHandle.Alloc(_obj, GCHandleType.WeakTrackResurrection);
                p = Marshal.ReadIntPtr(GCHandle.ToIntPtr(h));
                h.Free();

                // IntPtr p         CAN ALREADY BE BROKEN HERE
                return ref Unsafe.AsRef<T>((void*)(p + (int)_offset));
            }
        }
    }

    public class MyTest
    {
        private string _myString;
        public int MyField;

        public static uint MyFieldOffset;

        static MyTest()
        {
            var t = new MyTest();
            unsafe
            {
                fixed (int* pf = &t.MyField)
                {
                    GCHandle h = GCHandle.Alloc(t, GCHandleType.WeakTrackResurrection);
                    IntPtr p = Marshal.ReadIntPtr(GCHandle.ToIntPtr(h));
                    MyFieldOffset = (uint)((ulong)pf - (ulong)p);
                    h.Free();
                }
            }
        }

        public ByReferenceInObject<int> GetFieldRef()
        {
            // Offset should be known by the C# compiler.
            return new ByReferenceInObject<int>(this, MyFieldOffset);
        }
    }

    public class MyRefStorage
    {
        public ByReferenceInObject<int> StoredRef;
    }

    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(MyTest.MyFieldOffset);
            var t = new MyTest();
            var s = new MyRefStorage() { StoredRef = t.GetFieldRef() };
            Task.Factory.StartNew(
                () =>
                    {
                        ref int m = ref s.StoredRef.GetRef();
                        m = 4 * 42;
                    }).GetAwaiter().GetResult();

            Console.WriteLine(t.MyField);

            // Output is:
            // 16
            // 168
        }
    }

[dmitriyse]

EDIT found it #1147 and it was Vsadov not CyrusNajmabadi

I see that is different proposal.

[mikedn]

I don't see why would you need a GC handle in GetRef, that would probably make it too expensive to be useful.

Besides, the main problem with implementations like ByReferenceInObject is multithreading and you don't show a way to avoid that.

[gafter]

The CLR has a concept of "managed pointer" that I think can do this. It isn't exposed in C# today, but it is used to implement #1351

[mikedn]

The CLR has a concept of "managed pointer" that I think can do this. It isn't exposed in C# today

Nah, CLR's "managed pointer" is C#'s ref and they both have the same limitation - you can't define a class field of managed pointer type.

[dmitriyse]

@gafter Thank you for the answer, yes #1351 is very similar to this proposal (if we treat object fields as variable length array and offset as an index inside this imaginary array). So in IL this proposal can already be implemented, but not in C# yet.
Probably some CoreFX demand can push this issue to the "champion" state.

[dmitriyse]

@hypeartist Your example does not related to the problem of this proposal.
You are storing unsafe direct memory pointer to a stack variable.
But this proposal is about "managed pointer" to a GC object field. This "managed pointer" (96/64 bit) itself can be stored in a GC object field or in the stack.

[hypeartist]

@dmitriyse Got it now! Sory. Del.

[dmitriyse]

With @gafter help I found how to implement it in IL.
This is the idea illustration (but on the C# with help of Unsafe):

    using System;
    using System.Runtime.CompilerServices;
    using System.Runtime.InteropServices;
    using System.Threading.Tasks;

    [StructLayout(LayoutKind.Auto)]
    public struct ByRefInObject<T>
    {
        private object _object;

        private  int _fieldOffset;

        /// <summary>
        /// This is a workaround constructor, should be used until C# support or IL intrinsic support.
        /// </summary>
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        unsafe public ByRefInObject(object @object, ref T fieldRef)
        {
            _object = @object;

            // Hope that JIT will infer this method as a "partial interruptable" and will not perform GC in-between.
            _fieldOffset = (int)((ulong)Unsafe.AsPointer(ref fieldRef) - *(ulong*)Unsafe.AsPointer(ref @object));
        }

        /// <summary>
        /// This constructor should be used by a compiler or with IL intrinsic "ldfla" to get field address.
        /// </summary>
        unsafe public ByRefInObject(object @object, int fieldOffset)
        {
            _object = @object;
            _fieldOffset = fieldOffset;
        }

        public object Object => _object;

        public int FieldOffset => _fieldOffset;

        unsafe public ref T Value
        {
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            get
            {
                // TODO: Implement me in IL without Unsafe.

                // Hope that JIT will infer this method as a "partial interruptable" and will not perform GC in-between.
                return ref Unsafe.AsRef<T>(*(byte**)Unsafe.AsPointer(ref _object) + _fieldOffset);
            }
        }
    }

    public class MyTest
    {
        public int MyField;
    }

    public class Program
    {
        static void Main(string[] args)
        {
            var myTest = new MyTest();
            var myTransport = new Tuple<ByRefInObject<int>>(new ByRefInObject<int>(myTest, ref myTest.MyField));
            Task.Factory.StartNew(
                () =>
                    {
                        myTransport.Item1.Value = 100;
                    }).GetAwaiter().GetResult();

            // Prints:
            // 100
            Console.WriteLine(myTest.MyField);
        }
    }

[mikedn]

Hope that JIT will infer this method as a "partial interruptable" and will not perform GC in-between.

Most likely not. That kind of code is a GC hole plain and simple.

TODO: Implement me in IL without Unsafe.

There's no actual need for that. Though an addition to Unsafe would be helpful it's still possible to implement this without IL and without introducing GC holes. Something like below should do it:

public struct ByRefInObject<T>
{
    private readonly RawObject _object;
    private readonly IntPtr _fieldOffset;

    private class RawObject
    {
        public byte data;
    }

    public ByRefInObject(object obj, ref T fieldRef)
    {
        _object = Unsafe.As<RawObject>(obj);
        _fieldOffset = Unsafe.ByteOffset(ref Unsafe.As<T, byte>(ref fieldRef), ref Unsafe.As<RawObject>(obj).data);
    }

    public ref T Value => ref Unsafe.As<byte, T>(ref Unsafe.AddByteOffset(ref _object.data, _fieldOffset));
}

An Unsafe method that gives you a ref to the first data byte of an object would avoid the need for the RawObject workaround.

But none of this solves the main problem such an implementation has - it's not thread safe. And as far as I can tell this isn't easily solvable.

[dmitriyse]

@mikedn Please illustrate what the thread-safety problem do you mean.

[mikedn]

Please illustrate what the thread-safety problem do you mean.

Loading/storing such a struct is not atomic. You can end up in a situation where a thread stores to a field of type ByRefInObject while another thread is loading from the same field. The load thread my end up with a ByRefInObject that contains a reference to an object and the field offset from a different object.

This is a pretty well known issue that had an impact on similar types such as Span and Memory. Those avoid the issue in various ways (e.g. span is confined to stack so not subject to thread races) but it's not clear what can be done, if anything, in the case of ByRefInObject.

[dmitriyse]

Got it. @mikedn thank you very much for you help.
We can use 128 bit CAS operation (cmpxchg16b x86 instruction) with help of this proposal https://github.com/dotnet/corefx/issues/10481

But there are a lot of another cases to open a door to the "unmanaged erros hell".

We can just describe in the documentation about a dangerousity of using the "Field pointers" in multi-a multi-threaded environment.

[mikedn]

We can use 128 bit CAS operation (cmpxchg16b x86 instruction) with help of this proposal

Yes, but then you're moving into the realm of runtime support. Not to mention that performance won't be exactly great - replacing a virtual call with a CAS operation? Hmm, that might not be a win.

But there are a lot of another cases to open a door to the "unmanaged erros hell".
We can just describe in the documentation about a dangerousity of using the "Field pointers" in multi-a multi-threaded environment.

Then it's not clear what's the value of having this as a language feature. It can be a framework feature, to be used only in those cases where you need to extract the last drop of performance.

[dmitriyse]

@mikedn I agree with your concerns. And language support is not "so required" if "Compiler intrisinsic" will appear in the languge (dotnet/roslyn#11475).

CAS is not an option. So only only cautions in the document is possible.

But If the feature becomes popular not only across a framework developers than it needs to be added into the language.

[zpodlovics]

In order to support multi-threading a (formally) defined memory modell will be required for CLR. Right now there is no formal memory model definition exists for CLR. How it supposed to work in multi-core environment on different architectures? Learn from other peaple/platform mistakes [1] and do not introduce unsafe constructs instead provide an one level higher abstraction something like VarHandles [2].

Please note: you can have object fields address relatively easly from [3] [4] using the ldflda instruction. You can even define StructLayout attribute for class with explicit offsets.

[1] https://www.infoq.com/articles/A-Post-Apocalyptic-sun.misc.Unsafe-World
[2] http://gee.cs.oswego.edu/dl/html/j9mm.html
[3] http://mattwarren.org/2014/07/04/know-thy-net-object-memory-layout/
[4] https://github.com/kevin-montrose/Jil/blob/519a0c552e9fb93a4df94eed0b2f9804271f2fef/Jil/Serialize/Utils.cs#L320

[dmitriyse]

@zpodlovics, thanks. Very interesting links. I will inspect Java experience.