Passing through IAsyncEnumerable<T> within an async method body

[Mike737377]

Using an async on a method signature with a return of IAsyncEnumerable forces you to yield each item back instead of being able to pass through the original async enumerable.

This is not just an issue isolated to myself but seems to be a common theme on stackoverflow

• https://stackoverflow.com/questions/59876417/pass-through-for-iasyncenumerable
• https://stackoverflow.com/questions/65917317/why-am-i-not-allowed-to-return-an-iasyncenumerable-in-a-method-returning-an-iasy
• https://stackoverflow.com/questions/59689529/return-iasyncenumerable-from-an-async-method

Steps to reproduce

Given the following code example.

public async IAsyncEnumerable<ComplexObject> Execute(IAsyncEnumerable<ComplexObject> data)
{
    // do something which requires an await here
    
    return data;  //causes compile error CS1622
}

Expected behaviour

Code compiles and passes through IAsyncEnumerable<ComplexObject> data

Actual behaviour

Compile error CS1622

Workaround

Iterating through the enumerable will allow compilation

public async IAsyncEnumerable<ComplexObject> Execute(IAsyncEnumerable<ComplexObject> data)
{
    await foreach (var item in data)
        yield return item;
}

Scenario

We could easily workaround the above by just returning an IEnumerable in most cases, however imagine the scenario where we need to do some kind of aggregation with the data in a pipeline design pattern which returns IAsyncEnumerable.

public async IAsyncEnumerable<ComplexObject> Execute(IAsyncEnumerable<ComplexObject> data)
{
    var totalAmount = await data.SumAsync(x => x.Amount);
    // do something with this aggregated data 

    // forced to write a foreach which yields each item
    await foreach (var item in data)
        yield return item;
}

Questions

1. Why do we have to do this? I assume this is because of some source generation around async, but couldn't the compiler synthesize the yielding if it see's it the return as the same IAsyncEnumerable? Forcing developers to do this increases code verbosity and reduces clarity on what that method block is attempting to achieve.
2. Is there a better work around than having to iterate through the collection and yielding each item? I assume there would be extra cpu cycles used for the enumeration as well as bytes allocated.

[HaloFour]

Remove the async modifier if you don't want the method to behave as an iterator:

public IAsyncEnumerable<ComplexObject> Execute(IAsyncEnumerable<ComplexObject> data)
{
    return data;
}

As an iterator you are expected to yield each individual value. You can't return an IAsyncEnumerable<T> just as synchronous iterators can't return IEnumerable<T> and normal async methods can't return Task<T>.

[Mike737377]

I understand if you don't use an await in your body you can do this, but what if you use an await like I listed in the scenario section?

I'll update my repo steps to be clearer.

[HaloFour]

I think that falls under these discussions:

#378
#5303

[Mike737377]

#5303 is definitely a workaround.

But is there a better way to do it now so as to not waste cycles and allocations? (probably minor)

[HaloFour]

The allocation is unavoidable as the method is returning a different IAsyncEnumerable<T> that has to encapsulate the additional awaits within the iterator body.

[YairHalberstadt]

Return a Task<IAsyncEnumerable<T>>

[Mike737377]

Interesting working around if you can change the return signature unfortunately doesn't work in my case because its an ETL pipeline with IAsyncEnumerable as the return,

[YairHalberstadt]

Then create a local method which returns a Task<IAsyncEnumerable<T>> and then write a custom IAsyncEnumerable<T> implementation which wraps a Task<IAsyncEnumerable<T>>

[YairHalberstadt]

Actually I can't see any way to make that more efficient than what the compiler would generate, so no point.

[alrz]

I wouldn't have though this has built-in support in aspnetcore.. looks like a hack but it works.
I wonder which one is more efficient, iterating over IAsyncEnumerable or returning a Task<IAE>.

[CyrusNajmabadi]

Why do we have to do this? I assume this is because of some source generation around async, but couldn't the compiler synthesize the yielding if it see's it the return as the same IAsyncEnumerable?

Possibly. Though this might mask some bug. Perhaps you have an IAsyncEnumerable<object> and you meant to yield the entire enumerable as the next object in the sequence. Having the code be explicit makes it very clear what the intent was.

[Mike737377]

Interesting point, never thought about weak type scenarios, Although I would have thought it would have been relatively trivial to check for object and dynamic types as part of the syntesis whilst other scenarios would cause the relevant type saferty errors.