Optional Async Partial Method that returns a Task

[MicrosoftHawk]

Currently, partial methods that are optional can only have a return type of void or a non-awaitable async void, which limits their usefulness. I use partial methods to add custom logic to generated code when necessary. However, I face a problem when I want to use awaitable async methods in this custom code. As a solution, I propose introducing an optional Async Partial Method that returns a Task instead of void.

[HaloFour]

You could use source generators to fill in a default implementation of the method that returns a completed task.

[HaloFour]

Perhaps, but this is a workable solution. With void partial methods that don't have an implementation the compiler is free to completely elide the method and attempted invocations of that method. That cannot be the case here as you need something that you can await, which isn't specific to dealing with Task either.

[MicrosoftHawk]

It could be easily possible to return a completed task in case an implementation is not found?

[CyrusNajmabadi]

Note: when an implementation is not found, it's not like there's a synthesized no-op method method that hte compiler actually calls. Rather, the call doesn't happen at all. This is very relevant as it means that things like argument evaluation does not happen.

So it's unclear what it would mean to "return a completed task" here as there is no actual operation/work happening.

--

That said, i am sympathetic here. a Task-returning method is the async equivalent of a void method. So it does seem unfortunate that there's no way to expression this concept today for partial methods.

[MicrosoftHawk]

Thank you for your explanation, I appreciate your detailed insight.

I'm trying to get this feature added to C#. Is there a way to bring this discussion to the attention of the c# language developers, or do we have to wait for this discussion to be noticed ?

[CyrusNajmabadi]

I am one of the language developers :-)

[robertmclaws]

Is it possible to open this back up for discussion? I believe being able to specify Task instead of void on a partial method should not require a default implementation in the codebase. The whole point of a partial method is to elide the call if it isn't implemented. A default implementation defeats that purpose. It should be something up for consideration in C# v12 or later.

[theunrepentantgeek]

If you really want to have a partial method that does something asynchronously, then you can, it's just not very pretty.

Partial methods can have ref parameters, so you could use one of those to return the task.

 partial void Trigger(ref Task<int>? task);

Then, at the point of consumption, normal rules for partial methods apply:

var count = 0;

Task<int>? task = null;
Trigger(ref task);
if (task != null) {
    count = await task;
}

The thing I like about this approach is that it's explicit - you know by reading the code that something is going on; given this pattern is unusual, that's a tremendous plus.

You also get to lean into the compiler's optimization: if the call to Trigger() is omitted, some of the tests will become constant values. I don't know however whether if (false) { ... } would be optimized away though, the presence of the await might get in the way.

[CyrusNajmabadi]

Well, I guess that would be the tradeoff then: if you want a partial to be async then it can't be "free". The developer would have to make the conscious decision to incur the cost.

It's a possibility, Though, IMO, it's unlikely to happen. This sort of thing feels like it causes more problems than it solves. Namely that it gives people the confidence that it's living up to the purpose of partial-methods, but in reality ends up violating the intent of it (in no way that the user can see.

I agree with @theunrepentantgeek . Best is to be explicit here and have your code clearly indicate it understands and gets different semantics around this method when it's there or not.

if you don't want the if-null check you can also do:

Task<int>? task = null;
Trigger(ref task);
await (task ?? Task.CompletedTask);

This, again, has the benefit that the code explicitly clear about what behavior you're getting. For example, in this place you are indicating you want await Task.CompletedTask as hte semantics if the partial is missing. For other use cases, others might want other behavior. etc.

[HaloFour]

The one issue with that approach is that it makes it so that these partial methods can't be implemented as async methods, which makes it more difficult for the consumer.

[CyrusNajmabadi]

internally they can just do this:

partial void Foo(ref Task task)
{
    task = FooAsync();
    return;

    async Task FooAsync()
    {
        ...
    }
}

It's not pretty. But partial-methods were never intended to be :)

Primary use case is for generation tasks where some generator spits out thousands of these, but you only want to pay the price for the tiny handful you actually provide impls for.

[MicrosoftHawk]

I think the discussion is not going in the direction I hoped for.

I still believe there should be a an async implementation of a partial method. It would make c# more powerful and flexible for generated code.

Considering the compiler's ability to recognize and omit unimplemented regular partial methods, it seems reasonable to expect it to do the same for asynchronous partial methods.

For now I will use the solution the TheUnrepundantGeek and CyrusNajMaBadi have provided. I think some eyebrows will be lifted when other developers look at this generated code.

[Applesauce314]

There appears to be a very good discussion on this, I hope the below is a useful contribution. I know there are still problems to consider, but i think this feature would be a plus to the language.

First, partial methods are supposed to be zero cost (both in IL/metadata and at runtime) when not implemented. This would violate both of those. You'd pay the IL/metadata price, and you'd incur cost at runtime.
Halo's point though is correct, we could pretend that if you had partial MyTask Whatever(); that the impl was partial async MyTask Whatever() { } and let the async machinery take hold. But then you're still paying the price of invocation and metadata/IL (including all the statemachinery :'( ) which goes against the purpose we created partial methods for in the first place.

Is that really the case though? Would it not be more accurate to say that the purpose of partial methods is to support generated code? The Zero cost requirement is a desired implementation detail, not the purpose, and while that goal is a good one for any language feature, pretty much everything you can do with async/await is technically possible without those keywords, but that non-zero cost feature greatly improves developer efficiency. It is also worth considering that many of the scenarios which would benefit from this feature are already incurring these costs, but without the benefits to developer productivity, so while from a theoretical perspective it is not zero cost, from a practical perspective it may be.

This, again, has the benefit that the code explicitly clear about what behavior you're getting. For example, in this place you are indicating you want await Task.CompletedTask as [the] semantics if the partial is missing. For other use cases, others might want other behavior. etc.

it would only be unclear if undocumented though right? if the documentation clearly calls out what it does, it will be "obvious" what the code will do ( for example
if the Task is only awaited (e.g. await AfterLoad(); then it will be if the line of code did not exist;
if the Task is observed (e.g. await AfterLoad.ConfigureAwait(false)) or var taskStoredForLater = AfterLoad(); then it will be as if the method returned Task.CompletedTask
)
It would be completely clear what was happening.

internally they can just do this:

partial void Foo(ref Task task)
{
    task = FooAsync();
    return;

    async Task FooAsync()
    {
        ...
    }
}

It's not pretty. But partial-methods were never intended to be :)

Primary use case is for generation tasks where some generator spits out thousands of these, but you only want to pay the price for the tiny handful you actually provide impls for.

The problem with this code is it puts the weirdness on the developer side instead of on the generated code side. From that perspective the above is definitely inferior to partial Task Foo(); It is probably generally true that the developers writing source generated code typically have more advanced knowledge of the language, and they are trying to use the generation to hide complexity using the generated code.

[HaloFour]

You need a clean and simple solution to "async methods must return something" and "partial methods must never return something."

[CyrusNajmabadi]

it is surprising given Microsoft's focus on async

We can both want something, and yet discover there's no effective solution that can suffice in a particular complex space.

that there is no clean and simple way to do partial async methods.

Right. As you can see from this conversation alone, no one can think of a clean or simple way to do this. Fundamentally, the ideas are in conflict. 'partial' is about literal zero cost with absolutely no runtime impact. Whereas async/await/Task is about a non-free way of tracking asynchronous work (involving objects on the heap, runtime checks, callbacks, etc.). The systems, quite literally, have fundamentally conflicting properties that are very hard to reconcile.

[HaloFour]

Closest thing I could think of would be:

1. support partial ValueTask methods
2. if the partial method is not implemented:
   a. if the consuming code simply awaits the result, elide the call completely
   b. if the consuming code does anything else with the result, replace the call with default(ValueTask)

This forces the compiler to make more assumptions than async/await currently does, and doesn't work on Task or other task-likes given there isn't a known way to obtain a completed task for an arbitrary task type. That could be done via extending AsyncMethodBuilder to have a method that returns a completed task.

Given that partial methods are typically used in cases where there are dozens if not hundreds of hooks, having unimplemented hooks cost something would be simply untenable.

[CyrusNajmabadi]

I'd say the most viable path would be that a partial method could return any type T. However, if the partial method is not implemented, a call to it is replaced with a direct default(T). It's then up to the program to work with such a value sensibly.

As it would turn out, this would just kinda work already with default(ValueTask) (though certainly not with Task, unless we added support for await?). But it seems like such a niche thing to do (esp. given the seeming little community interest here).

[MicrosoftHawk]

I have developed a web application in which many components, such as CRUD operations, authorization, and validation, are automatically generated.

This is great because it allows me to focus on writing the custom code. For example, I have a validation class that performs standard validation for all fields in an entity. Additionally, I have a partial "CustomValidation" method that I can use whenever I need to implement custom validation.

Similarly, when retrieving data for the index page from my API controller, default all entities are returned. With the help of a custom partial method, I can apply various filters if that is required.

The only issue I encounter is that I consistently use asynchronous programming (async) throughout the application. However, when it comes to the partial method, I face difficulties in utilizing async the way it should be.

I think partial methods are a niche because they are not widely known and that their usefullness because of not being able to return an awaitable task makes them limited.

[robertmclaws]

I also just want to point out that nested functions are not unit testable on their own. Forcing developers through that pattern is a net-negative.

[HaloFour]

If you don't unit test the private implementation details, how do you ensure the stuff exposed to the public also works properly in all permutations?

By testing the combinations of inputs possible through the public surface. You don't test the private implementation directly. Testing the implementation details only makes for brittle test suites. By your argument, private accessibility shouldn't exist either.

[robertmclaws]

You keep saying things like "you don't". I do. Others do. Whether you are supposed to or not is up for debate.

By my argument, most of my internals are marked "internal" so they can be tested... internally. See how that logic neatly lines up? Whether "private" exists for anyone else is a matter of choice. I don't force my design methodologies onto others unless they work at my company.

And test suites are supposed to be brittle. If you change something, I believe you should be forced to think about why and what impact it will have. It's how I maintain high quality levels across all of my products. YMMV.

[CyrusNajmabadi]

A local function is not a testing anti pattern. By that logic, any logic in a method would be an anti pattern since you would not be able to test that snippet of logic independently from the rest of the method.

No testing methodology advocates for this. And we're certainly not going to contort the language for this approach to testing.

[CyrusNajmabadi]

And test suites are supposed to be brittle. If you change something

That seems like exactly not what would be desired. The tests catch issues that arise as you change code, precisely because they text the end consumption experience if the component, not the impl details (which can and will change an the time). If your tests have to change in this way, they lose their value in ensuring consistent behavior and no regressions.

[robertmclaws]

And we're certainly not going to contort the language for this approach to testing.

Nobody was suggesting that you do. But the suggested workaround is ugly, and a more elegant solution would be ideal than pushing the issue off on developers, as @Applesauce314 mentioned.

[MicrosoftHawk]

Sorry I marked an answer but It's probably up to the experts to selecte an answer :)

[iam3yal]

@MicrosoftHawk Hello, it's really up to you so if you think that one of the answers is the answer to your question then feel free to mark it as such.

[xiety]

Example gist of an empty async method source generator as a temporary workaround: https://gist.github.com/xiety/0ae30d718bbf6942e876a5dc230c573f

[MicrosoftHawk]

I noticed your puzzled smiley :) and now, of course, I feel compelled to take a closer look at your suggestion. The main reason I didnt is mostly because of my laziness, the apparent complexity, and my lack of familiarity with this type of source generator.

I am curious about whether this approach would be effective, considering that you can't define an async partial method without encountering a syntax error. I also would not easily be able to see where my custom code is, as every partial method would have an (empty) implementation.

[kjkrum]

@MicrosoftHawk If I understand what you want, a source generator could do it transparently without the burden of adding it to the language. If you declare a partial method that returns Task but don't provide an implementation, the generator would detect the error in the first compilation pass and generate an implementation that returns Task.CompletedTask. Such a generator could be packaged as a NuGet package and you'd get this behavior simply by adding it to your project.

[TonyValenti]

I have kind of run into situation similar to this.

The way I handle it is having my generated classes derive from a base class that has empty virtual methods that do nothing.

In the event that I want to enhance the generated code, I just add a partial class that overrides the empty base class method.

[MicrosoftHawk]

Good morning, Tony,

I wanted to respond to your suggestion, but of course, I forgot.

The issue with having a base class with empty virtual methods is that code generation relies on expected class and method names. In most cases, I won't need to add custom code, but in some cases, I will.

When I do need to override a virtual method, I must first create a new class that inherits from the original one. However, the generated classes that depend on the original class still do not execute the custom code.

If I generate all the classes and base classes, I end up with an additional layer of classes that often add nothing but complexity.

The suggested solution:

Task task = null;
CustomIndexQuery(ref task, ref query, ref parameters);
await (task ?? Task.CompletedTask);

Is indeed a good solution. I could even return something from the Task if it's required.

Stubborn as I am, I still believe that the async partial method functionality should be a part of C#. It would strengthen the language, but I suspect it could take years before that becomes a reality.

[danielmeza]

What about to use atributes to indicate when a partial method should be implemented or not?