Preparing for System.Linq.AsyncEnumerable

Ix.NET/Documentation/adr/0002-System-Linq-Async-In-Net10.md

@@ -0,0 +1,50 @@
+# Migration of core `IAsyncEnumerable<T>` LINQ to runtime libraries
+
+.NET 10.0 provides LINQ support for `IAsyncEnumerable<T>` in the runtime class libraries. This effectively renders most of `System.Linq.Async` irrelevant. However, enabling a smooth transition to .NET 10.0 for existing users of this library is not entirely straightforward. This document describes how this will work.
+
+## Status
+
+Proposed.
+
+## Authors
+
+@idg10 ([Ian Griffiths](https://endjin.com/who-we-are/our-people/ian-griffiths/))
+
+
+## Context
+
+As an accident of history, the Rx.NET repository ended up being the de facto implementation of LINQ for `IAsyncEnumerable<T>` from 2019 when .NET Core 3 shipped up until late 2025 when .NET 10 shipped.
+
+This happened because Rx.NET had effectively been the incubator in which `IAsyncEnumerable<T>` was originally developed. Back before .NET Core 3.0, there was no such interface built into .NET, but Rx _did_ define this interface as part of its 'interactive extensions for .NET' feature. It also implented common LINQ operators for that interface.
+
+.NET Core 3.0 defined its own version of this `IAsyncEnumerable<T>`, but the .NET team did not choose to implement LINQ for it. Since the Rx.NET repository already had a fairly complete implentation of LINQ for its original version of `IAsyncEnumerable<T>`, it took only a fairly small amount of work to adapt this to the new version of `IAsyncEnumerable<T>` built into .NET. Thus `System.Linq.Async` was born.
+
+In .NET 10.0, the .NET team decided to take ownership of this functionality. For various reasons they did not simply adopt the existing code. (One reason is that .NET class library design guidelines have evolved over time, and some of the methods in Rx's `System.Linq.Async` did not align with those guidelines.) So the .NET team took the decision that they were not going to maintain backwards compatibility with the existing Rx.NET-originated `System.Linq.Async` library. Instead, there is a new `System.Linq.AsyncEnumerable` library that defines equivalent functionality, but implemented from scratch, and fully in conformance with current .NET class library design guidelines.
+
+Most of the API changes fall into one of these categories:
+
+1. Where `System.Linq.Async` defined methods taking an `IComparer<T>` and an associated overload without the `IComparer<T>`, `System.Linq.AsyncEnumerable` only defines the overload that takes the `IComparer<T>`, making it optional with a default value of `null`
+2. For certain operators (e.g. min, max, sum) `System.Linq.Async` defined methods operating directly on numerical sequences, and also ones that operate on sequences of any type, taking an addition argument to project each element to a numeric value; in `System.Linq.AsyncEnumerable`, these projection-based variants either have a different name (e.g. `MaxByAsync`) or simply don't exist (as with `SumAsync`)
+3. `System.Linq.Async` offered some adapters (`ToEnumerable`, `ToObservable`) that handled async operations in potentially risky ways (sync over async, or fire-and-forget respectively); `System.Linq.AsyncEnumerable` has chosen simply not to implement these at all
+
+There are also a couple of cases where functionality simply has not been reproduced. `System.Linq.Async` provides an `AsAsyncEnumerable` to enable deliberate type erasure.  TBD.
+
+
+## Decision
+
+The next `System.Linq.Async` release will:
+
+1. add a reference to `System.Linq.AsyncEnumerable`
+2. remove from publicly visible API (ref assemblies) all `IAsyncEnumerable<T>` extension methods for which direct replacements exist
+3. add [Obsolete] attribute for all remaining members of `AsyncEnumerable`
+4. mark `IAsyncGrouping` as obsolete
+5. TBD `IAsyncIListProvider` relocate?
+6. continue to provide the full API in the `lib` assemblies to provide binary compatibility
+
+Note that not all of the XxxAwaitAsync and XxxAwaitWithCancellationAsync are handled the same way. In some cases, these now have replacements. E.g. System.Linq.AsyncEnumerable replaces AggregateAwaitAsync with an overload of AggregateAsync. So System.Linq.Async's ref assembly continues to make AggregateAwaitAsync available but marks it as Obsolete, telling you to use AggregateAsync instead. But there are some methods for which no replacement exists. We move these into System.Interactive.Async, because that's where `IAsyncEnumerable<T>` features that have no equivalents in the .NET runtime libraries live. E.g., although `System.Linq.AsyncEnumerable` defines `AverageAsync`, it does not offer the same range of functionality as `System.Linq.Async` previously did: overloads taking selectors (both sync and async). These methods become hidden in `System.Linq.Async` (available only for binary compatibility) and they have moved to `AsyncEnumerableEx` in `System.Interactive.Async`, and `System.Linq.Async` now adds a transitive reference to `System.Interactive.Async` in order to ensure continued source compatibility until such time as people update their NuGet references.
+
+combinations:
+
+* Method hidden in ref, available in `System.Linq.AsyncEnumerable`
+* Method hidden in ref, available in `System.Interactive.Async`
+* Method visible but marked as `Obsolete`

Ix.NET/Source/ApiCompare/ApiCompare.csproj

@@ -5,6 +5,20 @@
     <TargetFramework>net8.0</TargetFramework>
   </PropertyGroup>
 
+  <ItemGroup Condition="'$(TargetFramework)' != 'net10.0'">
+    <PackageReference Include="System.Linq.AsyncEnumerable" Version="10.0.0-rc.1.25451.107"
+                      Aliases="SystemLinqAsyncEnumerable" />
+  </ItemGroup>
+
+  <Target Name="_SetAliasOnBuiltInSystemLinqAsyncEnumerable" BeforeTargets="ResolveAssemblyReferences">
+
+    <ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
+      <Reference Condition="'%(Reference.AssemblyName)' == 'System.Linq.AsyncEnumerable'">
+        <Aliases>SystemLinqAsyncEnumerable</Aliases>
+      </Reference>
+    </ItemGroup>
+  </Target>
+
   <ItemGroup>
     <ProjectReference Include="..\System.Interactive.Async.Providers\System.Interactive.Async.Providers.csproj" />
     <ProjectReference Include="..\System.Interactive.Async\System.Interactive.Async.csproj" />

Ix.NET/Source/Benchmarks.System.Interactive/Benchmarks.System.Interactive.csproj

@@ -2,7 +2,7 @@
 
   <PropertyGroup>
     <OutputType>Exe</OutputType>
-    <TargetFramework>net6.0</TargetFramework>
+    <TargetFramework>net8.0</TargetFramework>
     <Optimize>true</Optimize>
     <Configurations>Current Sources;Ix.net 3.1.1;Ix.net 3.2</Configurations>
   </PropertyGroup>

Ix.NET/Source/FasterLinq/FasterLinq.csproj

@@ -2,7 +2,7 @@
 
   <PropertyGroup>
     <OutputType>Exe</OutputType>
-    <TargetFramework>net6.0</TargetFramework>
+    <TargetFramework>net8.0</TargetFramework>
 
     <NoWarn>$(NoWarn);IDE0007;IDE0034;IDE0040;IDE0063;IDE0090;IDE1006</NoWarn>
   </PropertyGroup>

Ix.NET/Source/Playground/Playground.csproj

@@ -14,4 +14,22 @@
     <ProjectReference Include="..\System.Linq.Async\System.Linq.Async.csproj" />
   </ItemGroup>
 
+  <!--
+  Since this includes code that uses the legacy System.Linq.Async package, we need prevent the compiler from using the .NET runtime library
+  System.Linq.AsyncEnumerable package.
+  So although we get this references transitively (or automatically on .NET 10.0+) we need to put them explicitly here to set aliases.
+  -->
+  <ItemGroup Condition="'$(TargetFramework)' != 'net10.0'">
+    <PackageReference Include="System.Linq.AsyncEnumerable" Version="10.0.0-rc.1.25451.107"
+                      Aliases="SystemLinqAsyncEnumerable" />
+  </ItemGroup>
+
+  <Target Name="_SetAliasOnBuiltInSystemLinqAsyncEnumerable" BeforeTargets="ResolveAssemblyReferences">
+
+    <ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
+      <Reference Condition="'%(Reference.AssemblyName)' == 'System.Linq.AsyncEnumerable'">
+        <Aliases>SystemLinqAsyncEnumerable</Aliases>
+      </Reference>
+    </ItemGroup>
+  </Target>
 </Project>

Ix.NET/Source/System.Interactive.Async.Providers.Tests/System.Interactive.Async.Providers.Tests.csproj

@@ -1,7 +1,7 @@
 <Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
-    <TargetFrameworks>net48;net8.0;net6.0</TargetFrameworks>
+    <TargetFrameworks>net48;net10.0;net8.0</TargetFrameworks>
     <NoWarn>$(NoWarn);CS0618</NoWarn>
   </PropertyGroup>
 
@@ -15,6 +15,25 @@
     <ProjectReference Include="..\System.Interactive.Async.Providers\System.Interactive.Async.Providers.csproj" />
     <ProjectReference Include="..\System.Linq.Async\System.Linq.Async.csproj" />
   </ItemGroup>
+
+  <!--
+  Since this tests the System.Interactive.Async.Providers package, which has a dependency on the legacy System.Linq.Async.Queryable package,
+  we need prevent the compiler from using the .NET runtime library System.Linq.AsyncEnumerable package.
+  So although we get this references transitively (or automatically on .NET 10.0+) we need to put them explicitly here to set aliases.
+  -->
+  <ItemGroup Condition="'$(TargetFramework)' != 'net10.0'">
+    <PackageReference Include="System.Linq.AsyncEnumerable" Version="10.0.0-rc.1.25451.107"
+                      Aliases="SystemLinqAsyncEnumerable" />
+  </ItemGroup>
+
+  <Target Name="_SetAliasOnBuiltInSystemLinqAsyncEnumerable" BeforeTargets="ResolveAssemblyReferences">
+
+    <ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
+      <Reference Condition="'%(Reference.AssemblyName)' == 'System.Linq.AsyncEnumerable'">
+        <Aliases>SystemLinqAsyncEnumerable</Aliases>
+      </Reference>
+    </ItemGroup>
+  </Target>
 
   <ItemGroup>
     <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />

Ix.NET/Source/System.Interactive.Async.Providers/System.Interactive.Async.Providers.csproj

@@ -3,7 +3,7 @@
   <PropertyGroup>
     <Description>Interactive Extensions Async Providers Library used to build query providers and express queries over async enumerable sequences.</Description>
     <AssemblyTitle>Interactive Extensions - Async Providers Library</AssemblyTitle>
-    <TargetFrameworks>net48;netstandard2.0;netstandard2.1;net6.0</TargetFrameworks>
+    <TargetFrameworks>net48;netstandard2.0;netstandard2.1;net8.0</TargetFrameworks>
     <PackageTags>Ix;Interactive;Extensions;Enumerable;Asynchronous</PackageTags>
   </PropertyGroup>
 

Ix.NET/Source/System.Interactive.Async.Tests/System.Interactive.Async.Tests.csproj

@@ -1,7 +1,7 @@
 <Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
-    <TargetFrameworks>net48;net8.0;net6.0</TargetFrameworks>
+    <TargetFrameworks>net48;net10.0;net8.0</TargetFrameworks>
 
     <!--
       CA1510: Use ArgumentNullException.ThrowIfNull - not available on .NET 4.8
@@ -21,6 +21,25 @@
     <ProjectReference Include="..\System.Interactive.Async.Providers\System.Interactive.Async.Providers.csproj" />
     <ProjectReference Include="..\System.Linq.Async\System.Linq.Async.csproj" />
   </ItemGroup>
+
+  <!--
+  Since this tests the System.Interactive.Async.Providers package, which has a dependency on the legacy System.Linq.Async.Queryable package,
+  we need prevent the compiler from using the .NET runtime library System.Linq.AsyncEnumerable package.
+  So although we get this references transitively (or automatically on .NET 10.0+) we need to put them explicitly here to set aliases.
+  -->
+  <ItemGroup Condition="'$(TargetFramework)' != 'net10.0'">
+    <PackageReference Include="System.Linq.AsyncEnumerable" Version="10.0.0-rc.1.25451.107"
+                      Aliases="SystemLinqAsyncEnumerable" />
+  </ItemGroup>
+
+  <Target Name="_SetAliasOnBuiltInSystemLinqAsyncEnumerable" BeforeTargets="ResolveAssemblyReferences">
+
+    <ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
+      <Reference Condition="'%(Reference.AssemblyName)' == 'System.Linq.AsyncEnumerable'">
+        <Aliases>SystemLinqAsyncEnumerable</Aliases>
+      </Reference>
+    </ItemGroup>
+  </Target>
 
   <ItemGroup>
     <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />

Ix.NET/Source/System.Interactive.Async/System.Interactive.Async.csproj

@@ -3,7 +3,7 @@
   <PropertyGroup>
     <Description>Interactive Extensions Async Library used to express queries over asynchronous enumerable sequences.</Description>
     <AssemblyTitle>Interactive Extensions - Async Library</AssemblyTitle>
-    <TargetFrameworks>net48;netstandard2.0;netstandard2.1;net6.0</TargetFrameworks>
+    <TargetFrameworks>net48;netstandard2.0;netstandard2.1;net8.0</TargetFrameworks>
     <PackageTags>Ix;Interactive;Extensions;Enumerable;Asynchronous</PackageTags>
   </PropertyGroup>
 
@@ -24,8 +24,31 @@
     <EmbeddedResource Include="Properties\System.Interactive.Async.rd.xml" />
   </ItemGroup>
 
+  <ItemGroup Condition="'$(TargetFramework)' != 'net10.0'">
+    <PackageReference Include="System.Linq.AsyncEnumerable" Version="10.0.0-rc.1.25451.107" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="..\System.Linq.Async.SourceGenerator\System.Linq.Async.SourceGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" Private="false" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <Compile Update="System\Linq\Operators\Average.Generated.cs">
+      <DependentUpon>Average.Generated.tt</DependentUpon>
+      <DesignTime>True</DesignTime>
+      <AutoGen>True</AutoGen>
+    </Compile>
+  </ItemGroup>
+
+  <ItemGroup>
+    <None Update="System\Linq\Operators\Average.Generated.tt">
+      <LastGenOutput>Average.Generated.cs</LastGenOutput>
+      <Generator>TextTemplatingFileGenerator</Generator>
+    </None>
+  </ItemGroup>
+
   <ItemGroup>
-    <ProjectReference Include="..\System.Linq.Async\System.Linq.Async.csproj" />
+    <Service Include="{508349b6-6b84-4df5-91f0-309beebad82d}" />
   </ItemGroup>
-  
+
 </Project>

Ix.NET/Source/System.Interactive.Async/System/Linq/IAsyncIListProvider.cs

@@ -0,0 +1,52 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT License.
+// See the LICENSE file in the project root for more information. 
+
+using System.Collections.Generic;
+using System.Threading;
+using System.Threading.Tasks;
+
+namespace System.Linq
+{
+    /// <summary>
+    /// An iterator that can produce an array or <see cref="List{TElement}"/> through an optimized path.
+    /// </summary>
+    /// <remarks>
+    /// This interface is primarily used for internal purposes as an optimization for LINQ operators. Its use is discouraged.
+    /// It was made public because it was originally defined in the <c>System.Linq.Async</c> package but also used in
+    /// <c>System.Interactive.Async</c>. Now that <c>System.Linq.Async</c> is being retired in favor of .NET 10.0's
+    /// <c>System.Linq.AsyncEnumerable</c>, the <c>System.Interactive.Async</c> package no longer takes a dependency on
+    /// <c>System.Linq.Async</c>, which is why it now defines its own version of this interface here. We can't put a type
+    /// forwarder in <c>System.Interactive.Async</c> to here because that would risk creating a circular dependency in
+    /// cases where an application managed to get out-of-sync versions of the two packages, so this interface is not
+    /// backwards compatible with the old one. If you were implementing this in your own types to get the associated
+    /// optimizations, be aware that this is not supported, but implementing this copy of the interface (in place of
+    /// the old version defined in the deprecated <c>System.Linq.Async</c> package) will continue to provide the
+    /// same (unsupported) behaviour.
+    /// </remarks>
+    internal interface IAsyncIListProvider<TElement> : IAsyncEnumerable<TElement>
+    {
+        /// <summary>
+        /// Produce an array of the sequence through an optimized path.
+        /// </summary>
+        /// <param name="cancellationToken"></param>
+        /// <returns>The array.</returns>
+        ValueTask<TElement[]> ToArrayAsync(CancellationToken cancellationToken);
+
+        /// <summary>
+        /// Produce a <see cref="List{TElement}"/> of the sequence through an optimized path.
+        /// </summary>
+        /// <param name="cancellationToken"></param>
+        /// <returns>The <see cref="List{TElement}"/>.</returns>
+        ValueTask<List<TElement>> ToListAsync(CancellationToken cancellationToken);
+
+        /// <summary>
+        /// Returns the count of elements in the sequence.
+        /// </summary>
+        /// <param name="onlyIfCheap">If true then the count should only be calculated if doing
+        /// so is quick (sure or likely to be constant time), otherwise -1 should be returned.</param>
+        /// <param name="cancellationToken"></param>
+        /// <returns>The number of elements.</returns>
+        ValueTask<int> GetCountAsync(bool onlyIfCheap, CancellationToken cancellationToken);
+    }
+}

Ix.NET/Source/System.Interactive.Async/System/Linq/Operators/AsAsyncEnumerable.cs

@@ -0,0 +1,26 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT License.
+// See the LICENSE file in the project root for more information. 
+
+using System.Collections.Generic;
+
+namespace System.Linq
+{
+    public static partial class AsyncEnumerableEx
+    {
+        // NB: Synchronous LINQ to Objects doesn't hide the implementation of the source either.
+
+        // Note: this was previously in System.Linq.Async, but since .NET 10.0's System.Linq.AsyncEnumerable chose not to
+        // implement it (even though Enumerable.AsEnumerable exists), we moved it into System.Interactive.Async so that
+        // it remains available even after developers remove their dependency on the deprecated System.Linq.Async.
+
+        /// <summary>
+        /// Hides the identity of an async-enumerable sequence.
+        /// </summary>
+        /// <typeparam name="TSource">The type of the elements in the source sequence.</typeparam>
+        /// <param name="source">An async-enumerable sequence whose identity to hide.</param>
+        /// <returns>An async-enumerable sequence that hides the identity of the source sequence.</returns>
+        /// <exception cref="ArgumentNullException"><paramref name="source"/> is null.</exception>
+        public static IAsyncEnumerable<TSource> AsAsyncEnumerable<TSource>(this IAsyncEnumerable<TSource> source) => source;
+    }
+}