How close can we get to a good smart pointer implementation in C#?

[SamPruden]

Context

The Unity game engine's recent move towards its Burst compiler has created a number of scenarios where C++ style features may be useful in C#.

For those that don't know, Unity now includes the Burst compiler, which allows specific sections of C# code to be run through LLVM and be heavily optimised. This is a big part of their push towards the DOTS (Data Oriented Technology Stack) which is a major overhaul of how the engine works and brings in major performance improvements. Burst compiled code has some significant restrictions, one of which is that it can't touch managed types.

This means that memory is all managed through custom struct container types which don't work with garbage collection. The IDisposable pattern is used to deallocate them. This makes ownership management quite tricky because we're back to the early C++ days of manually keeping track of making sure each allocated chunk of memory is deallocated exactly once.

Question

C++'s solution to this is smart pointers. I'm wondering how close we can get to a good smart pointer implementation in C#, and what feasible changes to the language may get us closer if necessary.

The problem of managing ownership of IDisposable types isn't new, but doing so without using classes and where performance is of paramount importance is unusual.

I don't have extensive C++ experience, so my intuition for the subtle requirements here isn't good. My understanding of smart pointers is that they rely heavily on being able to override the copy behaviour of structs. unique_ptr simply prevents implicit copying and only allows explicit APIs to copy and transfer ownership in a controlled way. shared_ptr overrides copying to implicitly increment a reference counter. In both cases, a destructor is implicitly called when the smart pointer goes out of scope. For unique_ptr this deallocates the memory, and for shared_ptr this decrements the reference counter and deallocates if zero.

There are two major blockers to doing this in C#: We cannot modify copy behaviour, and we cannot implement finalizers for structs.

We could attempt to approximate this with some form of struct CountedReference<T> type which provides Clone and Dispose methods to explicitly fulfil the roles of copying and destructing respectively. This would be a bit clumsy to use, but fairly safe if it weren't for the fact that it would still be easy to accidentally make implicit copies of this type without calling Clone. That vulnerability means that we would only ever have a (dangerous) pretence of safety and I predict that bugs arising from accidentally copying a counted reference without using Clone would be common.

There may be some trick using the ref like behaviours recently added to C#, but I'm not aware of what it would be.

My prediction is that the best that we can do today is some version of the above mentioned CountedReference<T> combined with an analyser that attempts to catch all implicit copies. The basic cases for that analyser are easy, but I'm not sure how feasible it is to definitely catch every copy. Even if this is possible, we would still have an ugly scenario when we want to implicitly copy a struct that contains one of these smart pointer types. We may find that many of our structs suddenly require Clone methods to handle this. I predict that that would lead to people lazily avoiding using the smart pointer, which isn't exactly a pit of success.

What is the best way to achieve smart pointer like behaviour in C#, whilst being as fast as possible and without touching managed types, and whilst being robust to mistakes by developers who only have experience in managed / GCed environments like C#? Are there any feasible language feature additions which may help here? Copy overloading would be great, but it probably isn't feasible.

Other techniques for safely handling ownership (single and multiple) of IDisposable types are also of interest! They must be very fast, and they must not touch managed types. I'm interested in this academically and for the sake of experimentally implementing a solution for my own Unity project, but if a good design can be found then I'd also like to propose that Unity adopts it into their framework.

[canton7]

See also (in no particular order):

• [Proposal] Deterministic finalizers on structs and classes #3450
• https://docs.microsoft.com/en-gb/archive/blogs/brada/resource-management
• https://blog.paranoidcoding.com/2019/12/02/borrowing.html
• Proposal: Destructible Types roslyn#161

[SamPruden]

If you're comparing directly to C++'s smart pointers, it's worth noting that you also need move semantics, otherwise you end up with auto_ptr instead of shared_ptr.

Good point that I forgot to add! I'm treating C++s smart pointers as an aspiration, but trying to remain realistic. Whilst I would of course love to see features added that made this all seamless, I'm also looking for practical solutions for today as the current issue in the Unity ecosystem is one I'm butting up against as a practical matter and is becoming a maintenance problem.

Those links - particularly the email and Jared Par's blog post - have done a fantastic job of making me even more bleak about the prospect of these features being seamlessly added to the language.

[SamPruden]

Oh, I see that you edited out the need for move semantics. I thought you were right about that, but I'm not very familiar. Did you edit that because you believe that it's not actually a requirement?

[canton7]

I removed that because I wanted my message to stand on its own as a set of links, rather than also starting a dicussion about move semantics 😄. I also missed that you started down the route of addressing move semantics through the use of an analyzer in your original post.

[SamPruden]

Even if this is possible, we would still have an ugly scenario when we want to implicitly copy a struct that contains one of these smart pointer types. We may find that many of our structs suddenly require Clone methods to handle this. I predict that that would lead to people lazily avoiding using the smart pointer, which isn't exactly a pit of success.

Perhaps an optimised reflection based extension method would be able to deal with this. It could act like a normal struct copy but call the appropriate method on any member which is a smart pointer. This would need to be a deep clone to catch deep smart pointers.

As whether or not a particular struct contains a smart pointer could be considered an implementation detail, it may have to become standard to use this extension method for practically all struct copies. This is getting quite unwieldy and I have concerns over whether it could be anywhere near as fast as it would need to be.

I don't think that this is on track to be a "good" implementation by any reasonable definition.

[SamPruden]

I think a good definition for a "good enough" implementation may be that it is not completely safe, but has a deep and wide pit of success.

[HaloFour]

https://github.com/dotnet/csharplang/blob/master/meetings/2015/LDM-2015-02-11.md#1-destructible-types

[sharwell]

The closest we can get today is ReferenceCountedDisposable<T>. Performance-wise, you could do a bit better by using a lock-free implementation, but to date this hasn't been a measured problem in Roslyn.

If C# had non-defaultable value types and a way to enforce non-copyable value types, the overhead would be substantially reduced as shown in dotnet/roslyn#42710. This just leaves the overhead of the StrongBox<int>, which is difficult to eliminate (at least in the way std::make_shared works), but perhaps the type could handle cases where the underlying type provides its own reference count as a field.

[Joe4evr]

Non-defaultable value types would have some wider value (heh) outside of an implementation for this, so I'm a bit hopeful at least that can be realized eventually.

[SamPruden]

The closest we can get today is ReferenceCountedDisposable<T>.

This doesn't work for the Unity use case out of the box because it's class based, but it's a little bit similar to an idea I've been playing around with. Sketched very crudely, that looks something like this:

// Being given this implies that you own it.
// When giving it to something else, passing it directly implies giving up your own ownership.
// Passing it with .Copy() implies joint ownership.
// It's considered WRONG to pass it without copy to two separate places, because they would have conflicting ownership.
readonly unsafe struct OwnedReference<T> : IDisposable
    where T : unmanaged, IDisposable
{
    private readonly T* valuePtr;
    private readonly int* countPtr;

    public OwnedReference(T* valuePtr) {
        this.valuePtr = valuePtr;
        // However I want to handle allocation of a count
        this.countPtr = malloc(...);
    }
    
    private OwnedReference(T* valuePtr, int* countPtr)
    {
        this.valuePtr = valuePtr;
        this.countPtr = countPtr;
    }

    public T Value => *this.valuePtr;
    
    public OwnedReference<T> Copy()
    {
        *countPtr = *countPtr + 1;
        return new OwnedReference<T>(this.valuePtr, this.countPtr);
    }
    
    public void Dispose() {
        *countPtr = *countPtr - 1;
        if (*countPtr <= 0) 
        {
            this.valuePtr->Dispose();
            // However I want to handle freeing
            free(countPtr); 
        }
    }
}

I could do a decent job of enforcing this with analyzers in basic situations, but it would be a mess if it's stored as field data in another type which gets copied. Perhaps a standard interface IOwned implying these semantics would be a better option, then objects that have fields of this type could also implement that interface. That may quickly cascade to being implemented on a large swathe of types.

Threading probably shouldn't be an issue in this case, because Unity's DOTS has a separate way of handling that.

I went down this road of investigation a few months back and ultimately decided it was a bad idea, but stupidly I didn't keep notes on why... I haven't seen the problem yet this time around, but I fear that I may hit some blocking issue a little further down the road.

Performance-wise, you could do a bit better by using a lock-free implementation, but to date this hasn't been a measured problem in Roslyn

The specific use case here is the Unity game engine, and even more specifically their new high performance framework. In that context, every drop of performance matters. I would hope that a solution could generally be removed completely by inlining - remember, a lot of this code is also going through LLVM on -O3! I know that's weird for C#.

This just leaves the overhead of the StrongBox, which is difficult to eliminate (at least in the way std::make_shared works), but perhaps the type could handle cases where the underlying type provides its own reference count as a field

Having the underlying type optionally provide a field to store the ref count is interesting. It's a little bit cumbersome, but that's one solution to avoiding the double allocation! I've had it vaguely in the back of my mind that I might be able to persuade Unity to do some IL rewriting magic - they're already doing plenty - to combine allocations for some common patterns, but that wouldn't cover everything.

[sharwell]

This doesn't work for the Unity use case out of the box because it's class based

Restricted to the features available in C# today, class-based is a requirement for these semantics. If this doesn't work in Unity, then the question isn't really about what C# supports anymore, but what Unity supports.

I would hope that a solution could generally be removed completely by inlining

Most overhead can be eliminated in theory, but nothing close to what's been demonstrated for modern reference-counted garbage collectors. For me, smart pointers are about safety first and performance second. Closing the performance gap is a noble goal, but safety must be maintained in the process.

[SamPruden]

Restricted to the features available in C# today, class-based is a requirement for these semantics. If this doesn't work in Unity, then the question isn't really about what C# supports anymore, but what Unity supports.

I would argue my question is about what C# can do while using unmanaged types, but at that point we're getting into a discussion about the semantics of discussing semantics...

It's certainly true that we're restricted by Unity here, but we're also restricted by C#. I'm just trying to find something practical given the restrictions coming from each end.

Most overhead can be eliminated in theory, but nothing close to what's been demonstrated for modern reference-counted garbage collectors. For me, smart pointers are about safety first and performance second. Closing the performance gap is a noble goal, but safety must be maintained in the process.

I've had some luck in basic tests getting Clang to eliminate ref counter increment/decrement pairs on ref counted pointers so that the counter memory access is skipped entirely, so it may be possible, but my experience in that area is minimal and I'm sure that you know far better than I do!

The thing with safety first is that it's at odds with how people generally approach game development. It needs to be lighting fast, or people will eschew safety for speed, and maintainability and reliability will suffer.

[The-Futurist]

@SamPruden This is only peripherally connected with your issue, but I've tried to build something that does some low level operations with pointers before and got to a stage where my work was halted because of this problem which has still not been fixed.

The type ItemHolder could be regarded as a wrapper around pointers, much as a smart pointer would be.

The actual issue now is here.

The use case here was the desire to manage large areas of process address space that sits outside of the CLR.