[Proposal] Ref and ref-like parameters of async methods

[IS4Code]

Summary

This is a follow-up to #1331, extending the relaxation of ref and ref struct variables in async methods to method parameters as well, with the same overall semantics. Under this proposal, ref/in/out parameters or parameters of ref-like types shall be usable in async methods (possibly only for compatible builders; Task/ValueTask at minimum), and shall have the same behaviour as locals, i.e. be available up to an await boundary (and require reassignment afterwards to be usable again).

This proposal does not cover iterator methods, since their code always executes lazily.

Motivation

Consider a general message-receiving component:

public interface IMessageReceiver
{
    ValueTask ReceiveMessageAsync(in Message message);
}

Such an interface is designed with these goals in mind:

• The method returns a ValueTask so it can optimize synchronous paths (indeed many receivers might not need to implement it asynchronously at all).
• Message is a struct because its construction might happen over a local, and the same local can be reused for multiple sequential messages, reducing GC overhead (even using an object pool complicates reasoning about the lifetime of messages ‒ using a reference here protects against unwanted escaping).
• Since Message is a potentially bulky object with many fields gradually added as the application grows, it is passed by-reference, because not all receivers need to access all its fields (but the interface must stay the same, so the method can't request these fields explicitly).

In this context, the receivers are supposed to process the messages as they arrive, sometimes pass the references around into nested receivers, while a single dispatcher constructs the messages in-place (in an allocation-less style). Most receivers aren't asynchronous, and most messages do not need to persist past the ReceiveMessageAsync call, justifying this design. Since message is immutable, all receivers are always able to read all relevant fields up front using the reference, and do any processing later.

The issue appears immediately when trying to implement this method via async even when there is no await:

public async ValueTask ReceiveMessageAsync(in Message message) // error CS1988: Async methods cannot have ref, in or out parameters
{
    Log(message.Sender);
}

Such a pattern is somewhat common to capture any exceptions into the returned ValueTask, as most callers expect. Even in this case, with 100% synchronous execution (and hence message being available the whole time), the parameter is currently not allowed.

The usefulness of this pattern is not restricted to await-less code, of course:

public async ValueTask ReceiveMessageAsync(in Message message)
{
    var (sender, recipient) = (message.Sender, message.Recipient);
    await LogMessageAsync(sender, recipient);
    // do something else with the variables
}

Here all relevant data is extracted from the message first and utilized later, even after await.

Currently, one needs to go through a local function to implement such a method conveniently:

public ValueTask ReceiveMessageAsync(in Message message)
{
    var (sender, recipient) = (message.Sender, message.Recipient);
    return Inner();
    async ValueTask Inner()
    {
        await LogMessageAsync(sender, recipient);
        // ...
    }
}

However this can still be prone to issues regarding exception-wrapping and other potentially erroneous semantics, such as using using in the outer method, and it is far from perfect ‒ since message must not be accessed within Inner at all, everything must be stored up front.

Benefits

All in all, this proposal would bring in:

• Convenience of implementation of methods designed for both asynchronous and memory-efficient handling of inputs.
• Correctness of such implementation, as opposed to the current solution using Inner (in regards to using).
• Avoiding a performance sink the present workaround could cause with runtime async. The C# compiler could always generate optimal code for the runtime.

Design

The overall design is similar to that of local variables:

• async methods can have parameters that are ref/out/in or ref-like.
• Such parameters are considered assigned from the start of the method right until the first await.
• After an await boundary, they are considered definitely unassigned. Consistently with byref locals, byref parameters can be reassigned to be usable again, e.g. param = ref expr;.

That is, the semantics of such parameters within the implementation of an async method are functionally identical to that of locals defined immediately at the start of the method.

out

Unlike other cases, out parameters pose requirements on the implementation itself. In the case of an out parameter, its value must be definitely assigned by the point of the first await, in accordance with what the caller expects from calling the method without await.

Reference escaping and scoping

Like in normal methods, these relaxed rules make it possible for references from parameters (or references to this or its members) to escape via ref or out parameters. Such an async method should have the exact same characteristics as a non-async method sharing the initial portion of the implementation up until the first await boundary, i.e. what can be achieved using the Inner function as above, again matching what the caller expects.

Examples

Span processing

This method takes an input span of an arbitrary length, identifies the length of the thing serialized within data, parses it, adjusts the span accordingly so it points after the thing in data, and handles the thing off to additional processing.

public async ValueTask ParseAndProcessAsync(ref ReadOnlySpan<byte> data)
{
    int length = /* find the end of a serialized thing in data */;
    using var thing = Thing.Parse(data.Slice(0, length));
    data = data.Slice(length);
    await ProcessAsync(thing);
}

This mechanism ensures the extent of the thing within data is known even before the final processing, it also correctly wraps any exceptions during parsing in the resulting ValueTask, and it ensures cleanup of thing at end of the processing.

Possible extensions

• Parameters of pointer types are currently also prohibited, which is something that could be lifted as well, without any substantial redesigning necessary (this was more of a safety check than a limitation, considering things like int*[] param are allowed anyway). Such a thing could potentially be changed to a warning.
• Currently, ref/ref-like locals cannot be used in a local function or anything else that could generate a closure. In case this restriction is lifted, it could also apply to async functions referencing outside variables or parameters, due to the similarity between a closure and a ref parameter.

[jnm2]

I will champion this.

[PaulMaget]

This would be huge for TryXAsync(out result) methods.
A current solution is to use tuples (bool success, T result), but it's not ideal.

[IS4Code]

This wouldn't (and can't) be a replacement for returning a tuple (or record or similar). References can only be manipulated across synchronous code.

[PaulMaget]

I don't understand. How do you plan to make out parameters possible in async methods?

[IS4Code]

Same as all other by-ref parameters, with the added condition that the method's implementation must assign it by the point of the first await in the flow.

This is written in the proposal. Which part of it is confusing?

[PaulMaget]

Why would TryXAsync(out result) not work?

[IS4Code]

Would it not? I think TryXAsync(out result) should definitely work fine. It doesn't matter to the caller whether the method is async or not.

The only requirement is that result must be assigned synchronously, so that you can observe it before you await the resulting task.