From 3c499b464c10daca2913e2c9a3c7fea6585f023a Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Fri, 19 Jan 2024 16:51:06 +0000 Subject: [PATCH 01/19] Packaging ADR WIP --- ...0003-windows-tfms-and-desktop-framework.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md new file mode 100644 index 000000000..20ad0e8e3 --- /dev/null +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -0,0 +1,77 @@ +# Windows TFMs and Desktop Framework Dependencies + +When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add tens of megabytes to the deployable size of applications. It has caused some projects to abandon Rx entirely. + +The view of the Rx .NET maintainers is that projects using Rx should not be forced into this situation. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. + +Since this topic was first raised, we have discovered that there is a workaround to the problem. It is not necessary to change Rx in order to avoid the problem. Back when endjin took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed—Rx 6.0 can be used in a way that doesn't encounter these problems. So we now think that a better bet is to have a longer-term plan in which we can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. The process of deprecation could begin now, but it would likely be many years before we reach the ends state. + +This document explains the root causes of the problem, the current workaround, the eventual desired state of Rx .NET, and the path that will get us there. + + +## Status + +Draft + + +## Authors + +@idg10 ([Ian Griffiths](https://endjin.com/who-we-are/our-people/ian-griffiths/)). + + +## Context + +There are a few things we need to take into account: + +1. the long history of confusion in Rx's package structure before Rx 4.0 +2. the subtle problems that could occur when plug-ins use Rx +3. the [_great unification_](https://github.com/dotnet/reactive/issues/199) in Rx 4.0 that solved the first two problems +4. the new problem caused by the _great unification_: as described above, an .NET application that runs on Windows might get tens of megabytes larger as a result of adding a reference to `System.Reactive` + +### Rx's history of confusing packaging + +The first public previews of Rx appeared before NuGet was a thing. So it was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your installation process. By the time the first supported Rx release shipped, NuGet did exist, but it was early days, so for quite a while Rx was available both via NuGet and through an installable SDK. + +There were several different versions of .NET around at this time. Silverlight and Windows Phone both had their own runtimes, and a version of Rx was actually preinstalled on the latter. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance. The scheduler support was specialized to work as well as possible on each distinct target. + +This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. Understanding which component your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and was something of a barrier to adoption for new users. + +With Rx 3.0, things got a little simpler, with NuGet metapackages providing you with a single package you could reference for basic Rx usage, and packages appropriate for using specific UI frameworks with Rx. However, this led to a new problem. + +### Plug-in problems + +Because Rx has always supported many different runtimes, each component came in several forms. At one point, there were different copies of Rx for different versions of .NET Framework: there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. + +However there was a problem with plug-in systems. People ran into this in practice a few times writing extensions for Visual Studio. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the `net40` `System.Reactive.dll` file. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the `net45` copy of `System.Reactive.dll`. Visual Studio is capable of loading components compiled for older versions of .NET Framework, so it would happily load either of these. But if it ended up loading both, that would mean that each plug-in was trying to supply its own `System.Reactive.dll`. The first one to load would be able to use its copy, but when the second one tried to load, the .NET assembly resolver would notice that it was asking for a version of `System.Reactive.dll` that was already loaded. (The `net40` and `net45` builds both had the same version number.) So the second component would end up getting the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` but not the `net40` build. + +Rx 3.0 attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/issues/205). But this went on to cause [various new issues](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120). + + +### Rx 4.0's great unification + +Rx 4.0 tried a different approach: have a single Rx package, `System.Reactive`. + + +In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). + +The `System.Reactive` is a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: `net472`, `net6.0`, `net6.0-windows10.0.19041`, `netstandard2.0`, and `uap10.0.18362`. Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. + +This design was introduced in Rx 4.0. Before that, Rx.NET was split across multiple NuGet packages, and this caused a certain amount of confusion. The goal of the _great unification_ that happened with Rx 4.0 was that there would be just one NuGet package for Rx. If you reference that package, you get everything NuGet has to offer on whatever platform you are running on. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. + +This worked fine until .NET Core 3.1 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.1, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. + +But .NET Core 3.1 ended that simple relationship. The answer to the question "Which UI frameworks are available if I run on .NET Core 3.1?" the answer is, unfortunately, "It depends." + + + +https://github.com/dotnet/reactive/discussions/2038 + +https://github.com/AvaloniaUI/Avalonia/issues/9549 + + +Is size that big a deal? Ani Betts wanted to dismiss this whole topic on the grounds that in this day and age, 50MB is really nothing. But the reality is that people voted with their feet. This has been a big deal for some people, and we need to take it seriously. + + +## Decision + +## Consequences From 001e0e27e22eab817fdd349b3e072013f6e8f22d Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Tue, 30 Jan 2024 12:46:52 +0000 Subject: [PATCH 02/19] Adding more detail to packaging ADR --- ...0003-windows-tfms-and-desktop-framework.md | 197 +++++++++++++++++- 1 file changed, 191 insertions(+), 6 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 20ad0e8e3..26c8e501f 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -2,11 +2,13 @@ When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add tens of megabytes to the deployable size of applications. It has caused some projects to abandon Rx entirely. +For example, [Avalonia removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. + The view of the Rx .NET maintainers is that projects using Rx should not be forced into this situation. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. -Since this topic was first raised, we have discovered that there is a workaround to the problem. It is not necessary to change Rx in order to avoid the problem. Back when endjin took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed—Rx 6.0 can be used in a way that doesn't encounter these problems. So we now think that a better bet is to have a longer-term plan in which we can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. The process of deprecation could begin now, but it would likely be many years before we reach the ends state. +Since this topic was first raised, we have discovered that there is a [workaround](#the-workaround) to the problem. Back when [endjin](https://endjin.com) took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed. It seems that Rx 6.0 can be used in a way that doesn't encounter these problems, so we now think that a less radical, more gradual longer-term plan is a better bet. We can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. The process of deprecation could begin now, but it would likely be many years before we reach the ends state. -This document explains the root causes of the problem, the current workaround, the eventual desired state of Rx .NET, and the path that will get us there. +This document explains the root causes of the problem, the current [workaround](#the-workaround), the eventual desired state of Rx .NET, and the path that will get us there. ## Status @@ -21,14 +23,28 @@ Draft ## Context -There are a few things we need to take into account: +To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem). The proposed [workaround](#the-workaround) needs to be understood in detail. We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. + +The following sections address all of this before moving onto a [decision](#decision). + +### The problem + +The basic problem is described at the start of this document, but we can characterise it more precisely: + +> An application that references `System.Reactive` (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). The [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true` + +That "or transitively" is important but easily overlooked. Some developers have found themselves encountering this problem not because their applications use `System.Reactive` directly, but because they are using some library that depends on it. + + +### The road to the current problem + 1. the long history of confusion in Rx's package structure before Rx 4.0 2. the subtle problems that could occur when plug-ins use Rx 3. the [_great unification_](https://github.com/dotnet/reactive/issues/199) in Rx 4.0 that solved the first two problems 4. the new problem caused by the _great unification_: as described above, an .NET application that runs on Windows might get tens of megabytes larger as a result of adding a reference to `System.Reactive` -### Rx's history of confusing packaging +#### Rx's history of confusing packaging The first public previews of Rx appeared before NuGet was a thing. So it was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your installation process. By the time the first supported Rx release shipped, NuGet did exist, but it was early days, so for quite a while Rx was available both via NuGet and through an installable SDK. @@ -38,7 +54,7 @@ This was years before .NET Standard was introduced, and at the time, if you want With Rx 3.0, things got a little simpler, with NuGet metapackages providing you with a single package you could reference for basic Rx usage, and packages appropriate for using specific UI frameworks with Rx. However, this led to a new problem. -### Plug-in problems +#### Plug-in problems Because Rx has always supported many different runtimes, each component came in several forms. At one point, there were different copies of Rx for different versions of .NET Framework: there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. @@ -47,7 +63,7 @@ However there was a problem with plug-in systems. People ran into this in practi Rx 3.0 attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/issues/205). But this went on to cause [various new issues](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120). -### Rx 4.0's great unification +#### Rx 4.0's great unification Rx 4.0 tried a different approach: have a single Rx package, `System.Reactive`. @@ -58,11 +74,36 @@ The `System.Reactive` is a multi-target NuGet package. If you download the v6.0 This design was introduced in Rx 4.0. Before that, Rx.NET was split across multiple NuGet packages, and this caused a certain amount of confusion. The goal of the _great unification_ that happened with Rx 4.0 was that there would be just one NuGet package for Rx. If you reference that package, you get everything NuGet has to offer on whatever platform you are running on. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. + +#### Problems arising from the great unification + This worked fine until .NET Core 3.1 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.1, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. But .NET Core 3.1 ended that simple relationship. The answer to the question "Which UI frameworks are available if I run on .NET Core 3.1?" the answer is, unfortunately, "It depends." +### The workaround + +If your application has encountered [the problem](#the-problem), you add this to the `csproj`: + +```xml + + true + +``` + +This needs to go just the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. The problem afflicts only executables, not DLLs. + +Why not just set [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) back to `false`? That might work in a simple single-project setup, but there are cases involving libraries where it does not. For example, consider this dependency chain: + +* MyApp (with no direct `System.Reactive` dependency) + * SomeThirdPartyLib + * `System.Reactive` + +If `SomeThirdPartyLib` has target that's subject to this problem (e.g., it targets `net80-windows10.0.19041`), and if it did not also set `UseWPF` and `UseWindowsForms` to `false`, then that library will have a dependency on `Microsoft.WindowsDesktop.App` + +### Community input + https://github.com/dotnet/reactive/discussions/2038 @@ -72,6 +113,150 @@ https://github.com/AvaloniaUI/Avalonia/issues/9549 Is size that big a deal? Ani Betts wanted to dismiss this whole topic on the grounds that in this day and age, 50MB is really nothing. But the reality is that people voted with their feet. This has been a big deal for some people, and we need to take it seriously. +#### The peculiar faith in the power of breaking changes + + +#### Exploiting radical change as an opportunity + +If radical change is unavoidable (and that's a big "if") there is a view that it presents an opportunity to achieve things that would normally be impossible. This is essentially the 'never let a good crisis go to waste' mindset, a notable example being Chris Pulman's [proposal for a greater separation of concerns in Rx.NET](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7559424). + + +#### Windows version numbers + + +### Constraints + +Our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. + +These expectations are not unreasonable because Rx.NET has been positioned as a core, flexible piece of technology. Its widespread use has been strongly encouraged, and as its new(ish) maintainers, we at [endjin](https://endjin.com) continue to encourage this. By doing so we are effectively setting expectations around backwards compatibility similar to those that could reasonably apply to types in `Microsoft.Extensions.*` namespaces, and perhaps even to types in the .NET runtime libraries themselves. + +This goal creates some constraints. + +#### Can't remove types until a long Obsolete period + +The simplest thing we could do to solve the main problem this document describes would be to remove all UI-framework-specific types from the public API surface area of `System.Reactive`. This would entail simply removing the `net6.0-windows10.0.19041`, `net472`, and `uap10.0.18362` targets. Applications using .NET 6.0 or later would get the `net6.0` target, and everything else would use the `netstandard2.0` target. The UI-framework-specific types could be moved into UI-specific NuGet packages, so applications would not simply be left in the lurch: all functionality would remain available, it would simply be distributed slightly differently. + +Unfortunately, this would create some serious new problems. Consider an application that depends on two libraries that use different versions of Rx. Let's suppose LibraryBefore depends on Rx 6.0, and LibraryAfter depends on some hypothetical future Rx 7.0 that makes the change just described. So we have this sort of dependency tree: + +* `MyApp` + * `LibraryBefore` + * `System.Reactive` 6.0 + * `LibraryAfter` + * `System.Reactive` 7.0 (a hypothetical version with no UI-framework-specific features) + * `System.Reactive.Wpf` 7.0 (a hypothetical new library containing the WPF-specific features that are currently baked into System.Reactive 6.0) + +Suppose `LibraryBefore` is using some WPF-specific feature in Rx 6.0—let's say it calls the [`ObserveOnDispatcher` extension method](subscribeon-and-observeon-in-ui-applications). Since it depends on Rx 6.0, it's going to require that method, and its containing `DispatcherObservable` type, to be in `System.Reactive`. + +To fully understand why this creates a problem you need to think about what actually gets compiled into components. Here's how the use of `ObserveOnDispatcher` looks in the IL emitted for a library depending on Rx 6.0: + +```cil +call class [System.Runtime]System.IObservable`1 [System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher(class [System.Runtime]System.IObservable`1) +``` + +If you're not familiar with .NET's IL, I'll just break that down for you. the `call class` part indicates that we're calling a method defined by a class. The `call` instruction needs to identify a specific method. The raw binary for the IL does this with a metadata token—essentially a reference to a particular row in a table of methods. Compiled .NET components contain what is essentially a small, highly specialized relational database, and one of the tables is a list of every single method used by the component. A `call` instruction incorporates a number that's effectively an offset into that table. (I've left out a complication caused by the distinction between internally defined methods and imported ones, but that's more detail than is necessary here.) + +The IL shown above is how ILDASM, the IL disassembler, interprets it for us. Instead of just showing us the metadata token, it goes and finds the relevant row in the table. In fact it finds a bunch of related rows—there's a table for parameters, and it also has to go and find all of the rows corresponding to the various types referred to: in this case there's the return type, the type of the one and only normal parameter, and also a type argument because this is a generic method. + +In fact there's only one really important part in that IL, which I'll call out here: + +``` +[System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher +``` + +This essentially says that the method we want is: + +1. defined in the `System.Reactive` assembly +2. defined in the `System.Reactive.Linq.DispatcherObservable` class in that assembly +3. called `ObserveOnDispatcher` +4. a generic method with one type parameter, and we want to use `int32` (what C# calls `int`) as the argument to that type + +It's point 1 that matters here. This indicates that the method is defined in `System.Reactive`. That's what's going to cause us problems in this scenario. But why? + +With that in mind, let's get back to our example. We've established that `LibraryBefore` is going to contain at least one IL `call` instruction that indicates that it expects to find the `ObserveOnDispatcher` method in the `System.Reactive` assembly. + +What's `LibraryAfter` going to look like? Remember in this hypothetical scenario, Rx 7.0 has moved all WPF-specific types out of `System.Reactive` and into some new component we're calling `System.Reactive.Wpf` in this example. So code in `LibraryAfter` calling the exact same method (the `DispatcherObservable` class's `ObserveOnDispatcher` extension method) would look like this in IL: + +```cil +call class [System.Runtime]System.IObservable`1 [System.Reactive.Wpf]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher(class [System.Runtime]System.IObservable`1) +``` + +And again, I'll single out the one bit that matters: + +``` +[System.Reactive.Wpf]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher +``` + +This is almost the same as for `LibraryBefore`, with one critical change. We saw from point 1 in the preceding list that `LibraryBefore` says the method it wants is defined in the `System.Reactive` assembly. But `LibraryAfter` is looking for it in `System.Reactive.Wpf`. + +So what? + +Well, when an application uses two libraries that use two different versions of the same NuGet package, the .NET SDK _unifies_ the reference. In this case, both `LibraryBefore` and `LibraryAfter` use the `System.Reactive` NuGet package, but one wants v6.0.0 and the other wants some hypothetical future v7.0.0. + +Unification means that the .NET SDK picks exactly one version of each NuGet package. And the default is that the highest minimum requirement wins. (It's possible for `LibraryBefore` to impose an upper bound: it might state its version requirements as `>= 6.0.0` and `< 7.0.0`. In that case, this would cause a build failure because there's an unresolvable conflict. But most packages specify only a lower bound. When you add a dependency to `System.Reactive` 6.0.0, the .NET SDK interprets that as `>= 6.0.0` unless you say otherwise.) + +So `MyApp` is going to get `System.Reactive` 7.0.0. That's the version that will actually be loaded into memory when the application runs. + +What does that mean for the `LibraryBefore`? Well if it happens never to run the line of code that invokes `ObserveOnDispatcher`, there won't be a problem. But if it does, we'll get an exception when the CLR attempts to JIT compile the code that invokes that method. It will look at the IL and determine that the method the code wants to invoke is, as we saw earlier: + +``` +[System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher +``` + +The JIT compiler will then inspect the `System.Reactive` assembly and discover that it does not define a type called `System.Reactive.Linq.DispatcherObservable`. The JIT compiler will then throw an exception to report that the IL refers to a method that does not in fact exist. + +And that's why we can't just remove types from `System.Reactive`. + +What we can do is replace them with a type forwarder. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, we can do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." + +Doesn't a type forwarder solve the problem? Not really, because if the `System.Reactive` assembly contains type forwarders to some proposed `System.Reactive.Wpf` assembly, the .NET SDK will require the resulting `System.Reactive` NuGet package to have a dependency on `System.Reactive.Wpf`. + +And that gets us back to square one: if taking a dependency on `System.Reactive` causes you to acquire a transitive dependency on `System.Reactive.Wpf`, that means using Rx automatically opts you into use WPF whether you want it or not. + +It would be technically possible to meddle with the build system's normal behaviour in order to produce a `System.Reactive` assembly with a suitable type forwarder, but for the resulting NuGet package not to have the corresponding dependency. However, this is unsupported, and is likely to cause a lot of confusion for people who actually do want the WPF functionality, because adding a reference to just `System.Reactive` (which has been all that has been required for Rx v4 through v6) would still enable code using WPF features to compile when upgrading to this hypothetical form of v7, but it would result in runtime errors due to the `System.Reactive.Wpf` assembly not being found. So this is not an acceptable workaround. + +#### ...except for UWP + +We are considering making an exception to the constraint just discussed for UWP. The presence of UWP code causes considerable headaches because UWP is not a properly supported target. The modern .NET SDK build system doesn't fully recognize it, and we end up using the [`MSBuild.Sdk.Extras`](https://github.com/novotnyllc/MSBuildSdkExtras) package to work around this. That repository hasn't had an update since 2021, and it was originally written in the hope of being a stopgap while Microsoft got proper UWP support in place. Proper UWP support never arrived, mainly because UWP is a technology Microsoft has long been telling people not to use. + +We don't want to drop UWP support completely, but we are prepared to contemplate removing the UWP-specific target (`uap10.0.16299`). UWP has long supported .NET Standard 2.0, so Rx.NET would still be available. However, the UWP-specific types would no longer be in `System.Reactive`. (We would move them into a separate NuGet package.) + +This is problematic for all of the reasons just discussed in the preceding section. However, as far as we know UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. + + +### The design options + +The following sections describe the design choices that have been considered to date. + +#### Option 1: change nothing + +The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) + +#### Option 2: new main Rx package, demoting `System.Reactive` to a facade + +#### Option 3: `System.Reactive` remains the primary package and becomes a facade + +We could maintain `System.Reactive` as the face of Rx, but turn it into a facade, with all the real bits being elsewhere. This would give people the option to depend on, say, `System.Reactive.Common` or whatever, to be sure of avoiding any UI dependencies. However, this might not help with transitive dependencies. + + +#### Option 4: UI-framework specific packages, deprecating their + + +### Other options less seriously considered + +### Change everything + +We could do something similar to what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretense of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. + +You could argue that we've already done this. There's a whole new version of Rx at https://github.com/reaqtive/reaqtor that implements functionality not available in `System.Reactive`. (Most notably the ability to persist a subscription. In the ['reaqtive' implementation of Rx](https://reaqtive.net), operators that accumulate state over time, such as [`Aggregate`](https://introtorx.com/chapters/aggregation#aggregate), can migrate across machines, and be checkpointed, enabling reliable, persistent Rx subscriptions to run over the long term, potentially even for years.) The NuGet package names and the namespaces are completely different. There's no attempt to create any continuity here. + +An upshot of this is that there is no straightforward way to migrate from `System.Reactive` to the [reaqtive Rx](https://reaqtive.net). (The Azure SDK revamp has the same characteristic. You can't just change your NuGet package references: you need to change your code to use the newer libraries, because lots of things are just different.) + +Our view is that we don't want three versions of Rx. The split between `System.Reactive` and [reaqtive Rx](https://reaqtive.net) was essentially a _fait acompli_ by the time the latter was open sourced. And the use cases in which the latter's distinctive features are helpful are sufficiently specialized that in most cases it probably wouldn't make sense to try to migrate from one to the other. But to create yet another public API surface area for Rx in .NET would cause confusion. We don't think it offers enough benefit to offset that. + + +Another idea: could we introduce a later Windows-specific TFM, so that use of windows10.0.19041 becomes a sort of dead end? + + ## Decision ## Consequences From 52a9d296a7e3b84a4a08966a307862ded20099a3 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 31 Jan 2024 09:16:26 +0000 Subject: [PATCH 03/19] More work in progress on packaging ADR --- ...0003-windows-tfms-and-desktop-framework.md | 106 ++++++++++++++++-- .../images/0003-Rx-Core-2.2.0-contents.png | Bin 0 -> 34649 bytes 2 files changed, 97 insertions(+), 9 deletions(-) create mode 100644 Rx.NET/Documentation/adr/images/0003-Rx-Core-2.2.0-contents.png diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 26c8e501f..f35c4bdd3 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -38,37 +38,119 @@ That "or transitively" is important but easily overlooked. Some developers have ### The road to the current problem +This problem arose from a series of changes that were intended to solve other problems. We need to ensure that we don't reintroduce any of these older problems, so it's important to have a good understanding of the following factors that led to the current design: 1. the long history of confusion in Rx's package structure before Rx 4.0 2. the subtle problems that could occur when plug-ins use Rx 3. the [_great unification_](https://github.com/dotnet/reactive/issues/199) in Rx 4.0 that solved the first two problems -4. the new problem caused by the _great unification_: as described above, an .NET application that runs on Windows might get tens of megabytes larger as a result of adding a reference to `System.Reactive` +4. changes in .NET Core 3.0 which, in combination with the _great unification_, caused the problem that this ADR aims to solve #### Rx's history of confusing packaging -The first public previews of Rx appeared before NuGet was a thing. So it was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your installation process. By the time the first supported Rx release shipped, NuGet did exist, but it was early days, so for quite a while Rx was available both via NuGet and through an installable SDK. +The first public previews of Rx appeared before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your installation process. By the time the first supported Rx release shipped, NuGet did exist, but it was early days, so for quite a while Rx had two official distribution channels: NuGet and an installable SDK. -There were several different versions of .NET around at this time. Silverlight and Windows Phone both had their own runtimes, and a version of Rx was actually preinstalled on the latter. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance. The scheduler support was specialized to work as well as possible on each distinct target. +There were several different versions of .NET around at this time. Silverlight and Windows Phone both had their own runtimes, and the latter has a version of Rx preinstalled. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: + +* The scheduler support was specialized to work as well as possible on each distinct target +* Each platform had a different UI framework (or frameworks) available, so Rx's UI framework integration was different for each target + +Some of the differences in the first category were implementation details behind an API common to all versions, but there were some public API differences too. The second category was all about differences in the public API, although at this point the UI-framework-specific code was in separate assemblies. There was a common core to Rx's public API that the same across all platforms. + +This meant that it would be possible, in principle, to write a library that depended on Rx, and which could be used on all the same platforms that Rx supported. However, it wasn't entirely straightforward to do this back in 2012. This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. Understanding which component your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and was something of a barrier to adoption for new users. -With Rx 3.0, things got a little simpler, with NuGet metapackages providing you with a single package you could reference for basic Rx usage, and packages appropriate for using specific UI frameworks with Rx. However, this led to a new problem. +An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component, and the original idea behind this was that this would be a stable component that didn't need new releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was correct, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even though nothing changed. The defeated the entire purpose of having a separate component for the core interfaces. + +The other splits were also a bit hard to comprehend—it's not obvious why the LINQ parts of Rx are in their own package. In practice, anyone using Rx is going to use its LINQ features. + +The 'platform services' part is arguably slightly easier to understand because .NET developers at this time were generally aware that there were lots of flavours of .NET each with slightly different characteristics. Even then, understanding how that worked in practice was tricky, and this was just another bit of complexity that could make Rx harder to use. + +With Rx 3.0, things got a little simpler. It settled on NuGet as the distribution mechanism. Rx was still fragmented across multiple components at this point, but Rx 3.0's simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. There were additional packages appropriate for using specific UI frameworks with Rx. However, there was another problem that this did not address. #### Plug-in problems -Because Rx has always supported many different runtimes, each component came in several forms. At one point, there were different copies of Rx for different versions of .NET Framework: there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. +Because Rx has always supported many different runtimes, there were several different builds of each component. At one point, there were different copies of Rx for different versions of .NET Framework: there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. + +So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (The Rx 3.0 [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively) And then each of those packages contained multiple versions of its assembly. For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders + +![](./images/0003-Rx-Core-2.2.0-contents.png) + +It's the same story for [`Rx-Interfaces`](https://nuget.info/packages/Rx-Interfaces/2.2.0) and [`Rx-Linq`](https://nuget.info/packages/Rx-Linq/2.2.0). And it's almost the same for [`Rx-PlatformServices`](https://nuget.info/packages/Rx-PlatformServices/2.2.0) except for some reason that doesn't have the `portable-windows8+net45+wp8`. Each of these folders contains a version of the assembly for that package. So `Rx-Core` contains 8 copies of `System.Reactive.Core.dll`, `Rx-Interfaces` contains 8 copies of `System.Reactive.Interfaces.dll`, `Rx-Linq` contains 8 copies of `System.Reactive.Linq.dll`, and `Rx-Core` contains 7 copies of `System.Reactive.PlatformServices.dll`. So conceptually we've got 4 assemblies here, but because of all the different builds, there are actually 31 files! + +This fragmentation caused a problem with plug-in systems. People ran into this in practice a few times writing extensions for Visual Studio. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assembly in the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. + +Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. + +Here's what would happen. Let's say a we have two plug-ins, `PlugOneBuiltFor40` and `PlugInTwoBuildFor45`. Both were built with a reference to `Rx-Main` 2.2.0. That means that if we were to look at how these plug-ins looked on disk we'd see something like this: + +* `PlugInInstallationFolder` + * `PlugOneBuiltFor40` + * `PlugOneBuiltFor40.dll` + * `System.Reactive.Core.dll` v2.2.0 (`net40` build) + * `System.Reactive.Interfaces.dll` v2.2.0 (`net40` build) + * `System.Reactive.Linq.dll` v2.2.0 (`net40` build) + * `System.Reactive.PlatformServices.dll` v2.2.0 (`net40` build) + * `PlugInTwoBuildFor45` + * `PlugInTwoBuildFor45.dll` + * `System.Reactive.Core.dll` v2.2.0 (`net45` build) + * `System.Reactive.Interfaces.dll` v2.2.0 (`net45` build) + * `System.Reactive.Linq.dll` v2.2.0 (`net45` build) + * `System.Reactive.PlatformServices.dll` v2.2.0 (`net45` build) + +The critical thing to notice here is that for each of the Rx assemblies, we have two copies, one built for .NET 4.0 and one build for .NET 4.5, but crucially _they have the same version number_. They both from NuGet package version 2.2.0, and the assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Core.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. + +Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second one plug-in, `PlugInTwoBuildFor45` first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuildFor45` is asking for. Now that was the `net40` version, but the assembly resolver doesn't know that these are different. It assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuildFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` but not the `net40` build. + +This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that a .NET 4.0 component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugOneBuiltFor40` and `PlugInTwoBuildFor45` would end up using the `net45` build of the Rx components. And that would work just fine.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the entire point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. -However there was a problem with plug-in systems. People ran into this in practice a few times writing extensions for Visual Studio. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the `net40` `System.Reactive.dll` file. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the `net45` copy of `System.Reactive.dll`. Visual Studio is capable of loading components compiled for older versions of .NET Framework, so it would happily load either of these. But if it ended up loading both, that would mean that each plug-in was trying to supply its own `System.Reactive.dll`. The first one to load would be able to use its copy, but when the second one tried to load, the .NET assembly resolver would notice that it was asking for a version of `System.Reactive.dll` that was already loaded. (The `net40` and `net45` builds both had the same version number.) So the second component would end up getting the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` but not the `net40` build. +It's worth noting at this point that the problem I've just described doesn't need to affect applications using .NET (as opposed to .NET Framework). Back when it was still called .NET Core, .NET Core added the [`AssemblyLoadContext` type](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext) which makes it possible for different plug-ins each to load their own copies of assemblies, even when they have exactly the same full name as assemblies loaded by other plug-ins. But that didn't exist back in the Rx 2.0 or 3.0 days. + +Rx 3.0 attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have thought that this would use the fourth part that .NET assembly versions can have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: + +```cs +#if NETSTANDARD1_0 || WP8 +[assembly: AssemblyVersion("3.0.0.0")] +#elif NETSTANDARD1_1 || WINDOWS8 || NET45 || NETCORE45 +[assembly: AssemblyVersion("3.0.1000.0")] +#elif NETSTANDARD1_2 || WINDOWS81 || NET451 || NETCORE451 || WPA81 +[assembly: AssemblyVersion("3.0.2000.0")] +#elif NETSTANDARD1_3 || NET46 +[assembly: AssemblyVersion("3.0.3000.0")] +#elif NETSTANDARD1_4 || UAP10_0 || NETCORE50 || NET461 +[assembly: AssemblyVersion("3.0.4000.0")] +#elif NETSTANDARD1_5 || NET462 +[assembly: AssemblyVersion("3.0.5000.0")] +#elif NETSTANDARD1_6 || NETCOREAPP1_0 || NET463 +[assembly: AssemblyVersion("3.0.6000.0")] +#else // this is here to prevent the build system from complaining. It should never be hit +[assembly: AssemblyVersion("invalid")] +#endif +``` + +By time time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. + +Again, it's worth stepping over into .NET Core/modern .NET at this point to see how things are different there. Those have a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) + +Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. + +As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems. But if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. + +For example, if your application targetted .NET 4.6.2, and you were using two libraries that both depend on Rx 3.1.1, but one of those libraries offers only a `net45` target and the other offers only a `net461` target, they now disagree on the version of Rx they want. The first wants Rx components with version numbers of `3.0.1000.0`, while the second wants components with version numbers of `3.0.4000.0`. This could result in assembly version conflict reports when building the application. You might be able to solve this with assembly binding redirects, and you might even be able to get the build tools to generate those for you. But there were scenarios where that didn't always know what to do, and developers were left trying to understand all the history described to date in order to work out how to unpick the mess. And this also relies on the same "we can resolve it all when we build the application" assumption that is undermined in plug-in scenarios, so this could _still_ cause problems for plug-ins! + +The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different targets. These might be a mutually compatible combination (e.g., if you use components targetting `net40`, `net45`, and `net46`, they can all happily run on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. -Rx 3.0 attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/issues/205). But this went on to cause [various new issues](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120). #### Rx 4.0's great unification -Rx 4.0 tried a different approach: have a single Rx package, `System.Reactive`. +Rx 4.0 tried a different approach: have a single Rx package, `System.Reactive`. This removed all of the confusion that had been caused by Rx previously being split into four pieces. +By this time it was also possible to sidestep the plug-in problem because by now, there was no need to ship separate versions for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET 4.0 would be suboptimal on .NET 4.5. But by the time Rx 4.0 came out, .NET Framework 4.0 was old and there was no longer any need to support it. The oldest version of .NET Framework that it made sense to target at this point was .NET 4.6, and it turns out that none of the new features added in later versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targetting different versions of .NET Framework. -In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). +Since there was now just a single .NET Framework target (`net46`), that solved the original plug-in problems. And collapsing it all down to a single assembly, `System.Reactive`, solved all of the newer problems created by the earlier attempt to solve the plug-in problems. + +It was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. The `System.Reactive` is a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: `net472`, `net6.0`, `net6.0-windows10.0.19041`, `netstandard2.0`, and `uap10.0.18362`. Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. @@ -260,3 +342,9 @@ Another idea: could we introduce a later Windows-specific TFM, so that use of wi ## Decision ## Consequences + + + +Spare: + +In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). diff --git a/Rx.NET/Documentation/adr/images/0003-Rx-Core-2.2.0-contents.png b/Rx.NET/Documentation/adr/images/0003-Rx-Core-2.2.0-contents.png new file mode 100644 index 0000000000000000000000000000000000000000..0188fccf9fac970733dba0bbe00e2ca07bfe1c6b GIT binary patch literal 34649 zcmd43cQjmY-!48nQASDhE`sRMnbCtp)D#k(Afg*(^j;D*O^Hqfi6|j@FQa!+N3YQt zj8V^?@AE!yeV%pJ`+I-qtn-JpZEN;9{{ii}6>;EQjr;1lX)W znow!BHn7;(*#0|Vl*&<$^yqZLDV2%eaoh%B&&fD(*n#6qN)ix97$s+@28b9qGN|Gz z0>>_PYLGH9hbrMa5O4{{y(s)3?4?To_)_aa(6Oc=s+npWDMyVsn zSU*kF3Nd+gzUMrsaP4QTzdH6i)q{;PFF+8ZL2L+1w&4!Q&wG{f{Bhq`ohYdnQ1`9z zer?Essuwiia?7kU4|w3854&;A!*4w+opElO)s|WP=y@N62yBm{lCT%2Hn5Zty2LKH@u&VVnBGUtQdPtc_M zsMgMQ`Cz5*ZcTmx@jjv{JG(`wdI8tS@(uTr42sHUiT;>XW#sXobj^jnO+!TGZG5E{ zpD1-Vs4*|!UWIulQ2}H7minFmliQs|(@!{Z@+2OZA1uy-MAe3q-%ZL~KZ3 zjOFpp&#vcY1p42m-K?!F+LAjj7Gq<8DQNZP3oidWsOaJ4o(|j`EA-s?H8D*gx4n>P z;@ohIOlUE3<||eL-Z+gl35fSL?l;7#T-tpn(e^p|NHv)h;g1)7Itk=E{@>ly>#rF@ zIhdC42juo3Tci2$j#aas3x)2YsW+M&rGW{Dcb=BjW(fs`ikpNlDX{4-Yt1 zBwIjfuTWi|Ja7WH1GdhuK4uaS@%`pIsSf&31RFeGVJa2;nBR(Zz3tF1;{n9u^pF?h zxcXth>bdBq7gc9HPl*Mvm6bjhL^otWenuF5n#{rdZ*C zqJo;Pj!PK21afWd*IbB@3)u36Yhk0rUt#^zFhTOn7b_6rJy-4 znej3{B$Nmo2XlPxchE6=@!-RqDfE0GTha}Vu!5&+FF;SYfB39*`{DL{*gYR>|7nra zcO7&QQ(b?u@%y_$8V#_`r+{tF?)qo%VW&6uw5gkNXZ|(>?6c zv-#7G%l;6<3RUPnfBq5UmMJV%25s_Jh<)FtV;^lvb+y-k&hSU|O9`n38Q=kXYD_Zy z2?(SneXs8k>``v}*lG&h$EulJ;mSvbyQp8?fmbLtDMV)2y_{>h7q2*&BpnQm?_#I# zPd#psX^m7*u`O$!pD;9Iy%&YaLgv9|CHa0pgzPPuesyn^H0{n^f;I`Z#Go{u4dtI}><)ZiXIE zFuUS%tlO#d1jam5yH#wj&g%W;cza6B^aZfm1V7(`=Lf)zo0X+8jFYu$$+H^L(Qx5V)rA)0w=~6smby{{;vtFL)*&xx5m7a{`zpb`v6Cf-Ci;f&1)FU)S5=LOQ3kr{HO2B z*UqkN{3puSSozznfzY6OJj_QBB%vjdD%#?vHM$gnQHPm;%g+LAJl-bK0bknyd@VK+ zMdi3J?fsR5awz{h$>)ENxBgFG>V{3euBLr_T4H@R{*L3)5<dOu4 z%Yk1XNrBLTq9O@;DB_>CdM92oRZ{ZXFz4^?BB5g&6S`k(@@;*LKos{?5C1FgtqD_u zZm~<*eO$dWXCrwWTK$D(A9qaHq90G)5Vog5tbq@t@8rQTRkG%%1}4^Z+drdY*%ai3 z3MD`1%=t6Oev^3-fLKptQW$ZYj}>y{}@E%ekP`qG!1s8-knr{nqPodl~!PUvHmZF6rSU;pzl6-QIx{M6ng@+4e&X?rlAM z6s(aeLv3rD<{DuFtnA7uo7Ac2w&~XXXGd?=2iDra3KjMx4dbl7+oISBr6w_$qfwo79EHoH(dG9i-oS3$0d5f=n1@>VpdErW~Y(WyG1WyuTUiUYC_wEO_*7rt(;k?|du`6r_vK49vjE@q}3?OqLjlo~3|Zx zbh7$RDwE!y;32?Y>0}FZP{4HS7(cZ7_4^s@aUzX!)PSlC0iM@&)(}5u-YoFE7nSr9u}(I;H0ZaVpi$YF!MkD zE!4FOJhc}ht>FB5xlN6zmufJ$aqIW@B+@=B215WsA)QA;NQGW`c>Hj-M(jA(tkUw_ zm7IF{p}Inl=FX#1Ai>Hk4|Bn3nE zgSMth$_Hcz49&b|J)EE+dvTCM4d3&fs-4gF#je4AzIzQ-qegEKKbAx+?g>o2!tF`X5DQfX^!;w z6P6eJb#Kj8USM!NrTuZFaQyrEXYv^TL=07qcUKS&9x~z4-kjf5Y51=R(-{W)m~H-& zdg`P~`ag2J)ol%Cfk@ZRsVt??B^kjMFZ`7sWeCO-AP7xXyDl%u?1Ux-pNF{mbDkQQ zdauH@W#NKZ3FmB-68`5W9&;gV0jC^7w*Ywm^((&O?*K~Pd3zn@=V|qe6oFMq;K$XC zsUexqh`yIJ)(ZaA*aA}z`+jA1`Q^JY+jPELH_#s(%_lU_$rTUu1?;Se6H~VUM*DHOxm1%^FWy)$<}`Y`G{hKcHQgc@&d-I=&vMNlO@SMl^*zpguj6 z#Q5BYN;$X90x)?{G!mvQURn&)p|rO_cL^HYoIj>pywz_^VjV=SV*GS>_3NEdKR#nk zx*t(WyJ#Qhmle4n*SOjo8KewPj|bnF1xhN{2c-4ZLFk~s*1Y#xhzkjd$TwtJ%d8-N6Wu?PL-_1wcLTsaCD{Fun73#@d@e)LrxJ z2^`X|)1f+h)HQ)?DYmtHJ2bDnfD);00TO_Es;H_=ymj$MC(!L$ILH9sbr99jRP$y) zJ;r=tP-dMOy~uf@Bf`N(bT%`8Wz%$hxu@xqrwgotH^x?0R*o;??{Z2l6837Hi;Up- zZOs|*?5*WL`4MQF`Mmr7h4b}`!0E5=F~$uNQUX@}muNdapqk)#sK^@hFi4&Cdm8wR z^sBd*f7&3J0Dsg?zdCN_S%sr8F%~#dzXKKEA8m@Z4H?=aHn0jD+#Q#HfMr;V?L*h1WJU?Hw=QED9 z3a@?yeD)O^fXar%e>))XdnNiWi5z?KYKkMD`zw^o`sbPg3-Hh10i@`EgDm;`HUA%Y zmj6qazPER9aEL=$YTJFbkleB_M-;j$QMtWa`?h5CWT+du%R=+mVG&=LmIj{3G9E2W+}~ ztONZGfYV}0JD}8#OQHp$kWxP}sKAtJ$YDIMrN};$THmbPiL{FTnl&_co-~>CFA}Nv z8h7`4|5uA&pY;VGAiv_pN1{$~e z)DcLX(L=l5Y6r*;J8_ng=altMb|mZ1?hiaOHR~B$KFE)v(7k%pHjr21lbpyA=oH(G zG8GD$_Hh7gUdTFxI(iE=UARf@0SMKjiDYtF>J3QG_>{u|T{+`aZeZsB`()AzZ1gI7 zIaRBW=m#R&lKw9hv|zaz`L1S2MymWOzGP~q=0_eg$bH#k4tjY7}2 z_#OW?@az-qoJ{wcv+ETokNHuzTPx|hq*}L;6Ky;+7sDX7#BJ347up2MmB=Y9T@fRr zcUHY343rW)H{>xZG*tphaQx*mGJ&#oY7D4>fX;a6cuvLC@8EvjYNAnE!Q{&iT1z8K z0;SEkyIM2=y^AZs2Wp+b{bM3j?=qIVtMTHvYxF%&SEQs|X8c;!)H5|tIjH?jw3A4ZkEFoHXG@>cD*Ss;DyUc9_UkD~L9t=qL71)u-@1OwU+l7b5lSK3b-W$ja)^&B z4{V-F1f#w5yI>ddS#0?P;Fe`>0Ii_@4QjS#_}%+&K+P?HPD22&@l$n0$Hts%Crjm| zW%Lwqv<)RBf4g2$A6GSD>MLV)@gL#l5&*+~`J-nip8W+mEkyR2BhkDG^f%xDL6c6Y z=o1PWxI7^aMQtvSMrW8vdYM<;S1PN>h66DLRVYBmSA64J99a#IPk~QYCpoR#k^+ zyutJnL&q73wO)I>iGqYJA-0+RmA!0%7Y75rx+IedtT8|#*@uW@VFI{B)t$#QxC>>S ztmTdV=g;yFyut`4|07@MxY*GOP{2byz%4i4R1qmfs9phR1HfkGu~HTQBJmQyZu%_Uzk6XW*fYQ{}?$7>e zqX_R{(;Y+JgV;b^(nxZqHB+5>VC(z{yuLb^4qrUSzqrORbyfu!8vx7VX6X;{qt&Ju zzVfd6{yBh{WegIn3UzE%KCoMLeg2Zzo~m)Ji^zxGXx)f{ewxyBeK}NFYS#U+@d$2% z3)uk>c{Up73`i!4&Tl9@39E*07Um-ZSyyf~J6291yurY&&^YgG6j~B-qbQg!! zyU0y`LpXVD_SjWk;U?vy-Uo}aPDaC(4eS7R;drQun?*hJy_1qAVIoW^4mBWsep7H_ zvnq3sTxN^#aVEHTlv;(rW6m1OV}6W?0Nqt|nwGQqpl;-ZNE$Aw*mM|F9Q?!G^&I9O zSl+nx92S3T=^lVZbNqn3^%k{a?a||8wCm0b3Ek(&%6hN6E^co25!<$CAEHh!!^hm|>@ARfSsBCYazV@WAbU z*r|WSW?R#p+y6B-_H~7Z$Gz{G4bLa;|EL)hDUWCw-ka0QRt40_C`>fIBEkPoHe3^v zpu%(c{nTJesBYVGxidaq^IPoK0xz?OxYu!snFlqii1-SmSFR?LEG5gp1D^DkfW}h< zJsMJRePD;(yZ{g8!X#=Lrj>G!TjvavAGf>Nri$2XRM)5WI(xIOJ}f`kR7 z4k&v4U4Nn2SMJO)KJstWINLnYP=q*2+UwWral8&)(-6rFIQyfmdrK?_;4rc;F@u4X zeF*~T#?(3?J_D^I?nI5BU z&7Xs@OYH|v0ciouRyBots(*gKLtDjQYmd$+nyyxNlK|2Mcv4gd08_`Xkk~d^a}@}z z6#%UUx%xElK}Y^i%qkYW02FW8(`U~+KA8dO_Aji$lj*kU0p~j>!|H4u+>VQw#+CP)#yS`P zw@1$gUSIC|jQCCiFvTR-=b(emAn)K5NY+UAl7YyQW$IU;;axeaO6M*DCMBVuc8MO2@6Da&6<+PP{7y zO=q7x_U2}5y8Y8VXt4tt06+bwB=Bdh(A|F(b#o;D9U5<*6W=&iINvVE0>3)ZK<5i+ zQ8b8N&arpSOAhLDUN%Uh%XV9LwY zZs)P)x(N&yq)g=kkk~T-qGTET!|pQgWyE`LKlFuuTl3W8q#Qr5%y^%x<9CJy(M&|s zl9?iF0iP3fj1bn#c0^Qaois%c@S55v9j)_Db27& zb=2`IFR2s$q6?@|oaA;YK4`80ltf@%2Ito0j&mSv)nR=xlFd`i5{o8K7ot9m*H^xE z%W-m8q1dhSPkhoPt(a_>tmy@I0oRO&SD~y z0y-_lTcyUnE@{LhoM_Ep`!47L9=H&1mumCN&8RmE$c@#0a`|MDdfN{oX@_<5mji<8 zMXJ<3*}yx|X+fbUVq|dxwJx1?3^VG{g*+jIqCOnt^r8perHF2gzkh` zN4NY$KDI+Y?E2Hc^TS|b$#-BUPnop6loNA8N_%BkDBdj- zUKGRm1ljY}`Zvnl`P=^5!!=TFg?XBnD!%jIkW_n3p^FIlgLay9SB_Wq0Cx&oj*5pI zM%gPebZRNEyn)gN;zckbt?BaiWTMCQ33%Yo9xm zb)@yuckJMmwyLE>qN|!Ne>4C>YbCIQr-7PmCobudy5KP{+(mn!oHFg&IRM2Zl>xOM z7w{tM6|72QTK-yQ#q6qmA0)!+4-{6?%cXtydMAO(WQ(yjTBx=w_C8PN&=lRVR>CZ{ zmSU{M*7__=on6QKEndc4n;)&s-*6{(O~77cF|E4;x0DX8?Y@)Rp68e%ViBVCS-hJe zu=0c2_0^>bI2y)hL)jB<-zm`^-ivX8imV~cR-@#0Nk{6x0$`;Qc*Z&i+WycKvk(eK z9xD?!p$6n!?MS1DGT0D>H8y#QX`R=XP1h41&9o-&dNTm_Mm7YqE4sVQLYQZ+F9xrr ztB(+ZF<{J{`0(zk*J@=kiYei4hN>r zD%-1BD-gh4)me1K0E(cF7iZV>Cvc< zk~rjO0SiVaP7iche7DOy|KugrUV@n<`y}-9GkjjlgQAHhLaADs4-){Iqu;ZIfUeqV z5(va5L5yl*{ilh(8R7fu@%snCPRw{}?+HBs&ejW>m}C|p11#Q2iinuAFT-nal83Dg z+UX-!H@pFKkHm;Nb-TJi4dWN;xZ)ura{=d$=x_9~jKFTV|Cgd+EHV@eYgl^AkM&m! zG;{XdP?{a_RZ+Y>d(CPVSLg49uTR_{ze_|W<+t6DJ>B11}+oQQJ*u4jdc--MekMtMQ58kzuhtRi2hv}+)d|VZH+Qx?T2?TNl|6ByD7+LZlo;X8(fKD`m_Xd=c zSu6x(u^pKNfv32f)6~=5Fp$lNG8>4rXT0$?PI^x!8<3WsG6r`2qdFhQ%DLV4cJ}Z- z)EQX-`3P~L#|J0|!5b&Uf8HE650!Fv(myB4>p$TqA(uoDSShAGc<%;8uB!_>_>}1j zog@S?N2|X=KAUy6RCyQY#d6_PsqlO%xk)AQHvpkbNMzBs+-Bx)xL>RygA3XthIY<+ z&h&fEdQ4PTh|Far9Tr1^JwN&6XIL;7b9jj4JJT&3XDy;LD5*`kH-kiBKFD@`cFDU0 z2phmW=0XS+e=VOFGA9%A-gSHbx8?wqiiV;!hmPj^TL#9%vv?0goRHrgg#BzbFd60lS?5N#x-ZaD%8eA$fLZVs-O&EkDmbB z@#*%spYuoFKgy?Nfr;cDx!^QXn%$WSA2mosgDazLy&gR|F4OfGY$`@}6IEklI_WYGbE2(B!sbuH>L*Jio)ipN9K0TZF6^rG}kRM zY2X?RA8g3L6Hv7tb?@tKTLoC_7s}N%TaF9N=;V;^4HHE73f`v3^wCEWffmoGd{e5|{+rA?V@Alcqf zyF?&{El1S0IxI_n_j{^SZMVciyX?M2Qq3(~WA$R12^MEf;?9s(JXR@%HFG>gz?ShGA;`5ag4%}^ac)Dk&0Wfy^fi1Thc$n8wsZAYe| zFTUQv2eoiNu>>}_`lLv^+EmNGQ$jZh!?h2|E2UnEuXG@1=())h)Vk4R{5apzK*Ed) zA?_OoD8<8Yp*b>*+I~%LZ)_SHyzKiPo!u*);-| z)RQlC)kQb*sM*8yEVr4gXzuJS?l;Kpbu7rj8lvx?j#ifoMTj`0F-&WW_g`P_UDv9y zsjw!Z)*iU=MiMe+H_IH3;}NF1x+hxGTR9z|UG8OsFPpP19vpa*U?;CTNVQTk-AAg) z%Z^6l?J37MmgR0p2G9I*2W6ravInOum@e}2>v0y9F}?F9Ck8=&AiRj7_EKbd*AH0 zS1!?oOn~GDlN2*p1kAtJ?FO~$I@rI7OUDq)2%bPPO2>Ph!^r5hjY}->TK&QvaP-Iw zDi6^Q^TMuHWAXRyq5?q%$LN;WF=uT1<0{#U zy!dw~K4vSNkg#~qt?G-3f_{VPWdETh@plV0BXNq(awsBH_v$KGE^X{Q6y#)*a_go(dFCt+x4zhKuE%wahzvgCLfjFLFjkM4qYSC;@qSu10^cJ-gU?ADSh73AP9lA) z6L{N_#l*WN9;#3+%;1DAO3pZZ0Bh`SpoEA3I`o}o+U!ld^sc&_fFC_G$@q?dj?tZm zEm~9W_W8^h=@^~mPn4G~%_E#ej{GztBD+peTls6@gFvgn!{;pX0KNuqmPanh|?3})@bUmFP5vu!nnzm1^;@S%i_ zupqtcYJX}Ux3Rx@65-?yVYW*9>dewUU#(P&xAWXQ0;whp@be{79|M*+09Xq$w#p1Auf$x zMWuK%OAVhxP!+czU?jNyxnjNYsgFJPpsy(xjxH5v=yr@t@JHGB2atZ6>$RuN$|ZQT zk66GB;6}zyqQ~`2)XNC_NA-jsXbH4c2AjH9Zb1}>vY(44Avhkr=*u`EQ&14E3+{Y zH)Q5n*Kz*&1W%VFs+`>|(_nq0V}U!HDtGs^|2S`v)Pg0Xtl)8kc!7P$pNEbGqlz)+ zF*RyP9r^0mGFZqRz*Rh-bo7veePaao>b*JTsaLhJX3XvNSSgUGq5x>34k~gFe`k@Z zrGEepu{xl)+Kk(JqyGJ-pQ06Zw&+yoGcVpM6y=D|~?J9#?Wd za~2({{Di7~`GXy-$lc5p|8#)pl3=vHPJUoL5;w$7hR?Xuk$<|94j(J-zX7}jxIV}9 zmbizXMY>$)7;H0hR$pA6pc*)AXU#WJ7Tzx`3$h04$gmcW5x~xZPg-QxPpho(!tB}>tYgfyBt0Trht4gEU?bYe{ zyBP+&go=QNy-S%a=wCWs#PR?C>XP|?f}O8HAP{TP!fPMDGT4+iO4_F}(%s~svnvK< zhgsS;0q$=C)?f5sdsr}0_=?=g@&j%k)Kj&Xn0~){xB2o}l23VY^b1&~g$+62324Yw zeuM2DLeY_6Ej8jjZZytfsb@;YBNT9)&7{(`V6LJXuiN;yu}wXFwy!w}7ruQj{%PVl z)q#j-V@rC%q@}mlNXGJ;Sx{ja)*H)aSF@*2}AlTm{TY)NnRlOf(e(*cZ+Pr#ZF}prXsx$jbv3?TeJqS?fmXXYEQ6@k`h9p38@noF=`~%}DRyH?< zFNVXrr}3>k05fo%?H2GH@tV=2NTdV6mzDg@lx#r=W_9>p^z&hA#X<<_3lkHq@m{zG00rjBGRK{0*oyUWcr_sbQs&_Or+k~&<)d$cYJJLg@p8sWxY>?uZz z+V?7M3X1XTw~z9)Gu>j{{Qywky;$Rl{p;6nHSf!iF-s2>*X_#so-Rf@{`fTx{7qCO z4q#v#8vx3Dz}F41LDzV0VFka~3xNK&2t1#e-v+qSn27@DzS9LN^IG6LInZ#EH0%E) zrlj$#M+-e|FO1&4Gm7oVYVbLB>vgB4R7*1POxY@0U|E!3MDJA1I{|haGvqyN&@_>g z6fWSCdD_64HC3b;`g}s3`%5FRw*Of(ly`^g4Lx$d{T3pCy(p1sIodM2p7uG}{?k+E z{$~llWiapxCdK9je+KxBB524Oe9@KHm-E-Lnlp!hO|<%tPQW3JR6j0Y=5=Ya*A#B3 zVDZ_=uH0{Z=NQ#>-|-w!^z8u~L_@>%)%gh)c6~DV{27)1B#$_YCr)oq^fvT3=0mTr z^|<{jO=Ze2Cb+maOgJ+i4QGn%-2;TpAl56azbVB!u`J;7tRqj5oe(RStaI~r+?soy z<#i6o5mP{2YKLtTF`lN~E8Lp?^*z-l7%SG?(fa%nt4K4Q5i8*r0fuIwYk<$Iv&$=# z!2x9xv+IVqexM-u=N6jNJLq)(iIn)5n#`}OqlvCl@M)#6;RCxCfupdu1m&d^LM2!~ zA=a@`eN6m$X*ws)^+UMGZuO|PT#f1#@U&Ghrk||4PkW{)ggVQtI;Vm6zPpC8I`X@| zM0<9!cvZVf#*MN$^R-C9+`l~xiYTl0~ zXnNfX^wSzlS7vi!{eadjd|lT z^?=!Tcanj}+T^73z|^(lmQx)o*)|~7tm}U+^9M{FCc;p`Qmn6k7d}bk>;hVEJvMik zpXW#Bsx!NWHC&#|I5BJhJ!OJ`Nw`=_^ZaAG*$1FOXG|-4F?4AJckO+c^JiT!Ge4RV z{k|!)TI$AP3CH@%pddrV2D+sl5ISbLoBRw7INDW=122yU5*!y40zWG}#>r2czk?F$RAAHUfb_L&9mbB@K@0gzYEi%?LKt;h{zfPUMxWgub z5EZCGx;c@hAT9u>odA5-Ziv`b$U$n#K5IC)5oGX-4R)-H@YzNI-SyVx7sABM{8~ zSCJhfpj|cGp|piCB+6KL*O%HmgTE(*Eo3Xx(PfU3{gF*mcvasHCP3s-tEc_#=eLYD>t@ z_-+S!2^5$;*#-j;)V%_5bxY5>{^YIZEz zQ@$2@vr78x19VNHZQ<>H_LADmqo5`7@*U^ zr86ivXF_9;{ogo!js>E{X3?RbQ5Q>%4LeH3r+3Hb$ZwD*Nky=dvBWYE278Yg5HF8Z zm)gi?7t|XY}iLi$uh=`rWquh2%B$b zbJ1~ec6DHlkuw$B{Nx*lDQauzk}S~NOyxtw!PYUYq?(+Au>0O4^ltv%YWp`Gs~01H z4bDt+zChp5IpFIh1n4i%=F%T34W?a|>dcRxur~uO2y$NWF65$TtcaZ&b;_tBw${tR zL#%1;`!%-db2nqn^f>1(^WeJ7vR}=h^dMmV?t{uf_>l_4k%Eeb9et2w;J`9&# z{OSq)yB-#by(25$8+p$Vkb%fm4CDiRbAMIuaHfZ+X%PN`APWg-0%6a{Ia_pym|cP# zfjBAjUSd0VA>#g`W%Kj+$o{s|32ZkAa+Cfk<84M1C8HcGvrim8{U=hW8a__vMmC)a zI4Z;8MH>iPm;*$iTn-g+JRVmgFtlnb^Axu%i#F7|?Wv*Otb-iAJXcb3Hge{WAoJUd zbNbEIj}833L~jwKwZ+NpZurzZco$p+3RPiY@m_g{0m&>(vw0z1cTF`~C9*^bHq-6b z&_qSi=ZsvT#QV^~Th9zzOwSAGdlaQihSscQ$({ z(}xY2TS49&eaigN9T{yT_KCd4?_uO+WJoI5m5+w^!?^vx5Qkagk#tV1GW6qn^zMEfh>eYn0Ypv=e7 zq$BwqN^UC2^Nf826+B!9V@!RidzMzny-w>I{E}ObXK!9q$eM4bm?}{_In1*!E_Lt@ zpF#kow>XASOq`GdLM&bj$vTo&F|-9Bv7>Q3jFcR#M$q;_GR|ngo3=Gmft+|o)1KX~ zRMZ@awkY`l6_GyDezmP4r2u_yT+hsf5_NC1UG|m0FA*)dxgjyqtIaB>_G!nZp?;CP z#4FogZ7+pTibOfSEReBB8OnjsU`Xx!5#NH#=rs`T%j1snT~tw(=Zrf$wn8xrqxkYU z2-`u*AC@(kv^81{g&U+cSw+LWqGi|97|xnEXzQfb2nVDb;lbxoXNr$}^?HX0Ji-l0 z$rB@&^C+t0vV$GKm4l@$>;nfCzla*FOi`!|S?UZHh&}vKfmh5+ZnI3)E-q@(jjIgM z?yUhd)@JdY4*xN|dI-I;B24(so_|JU#hI;4`^|}9$MVL=^BXNu8w+qiATgM@b6B(_ z+Kd-knoKtq11fm2ycgw+`&aL?w{x{hD9hh%lUJswRZ%`fF4w}E5I!OVQ(Z|6&wjU{ z)CP1Vf^(Zg727_My>5dq9cD0z6djcULWIdKfb5E;c4IaEV(frja0*Ydy$47Us>E$9 z^gJ2C71nKv4K5cm=;lJX7+!L+SVolH6VZ!iH+uZFBw51fVe$rcR>ry&BBKJ*lizd4 z+UG;jbb<6$mYosi$~Tu%nwu>vM9tGuofoJ$t6zymka>tGY~!3E6k#$MG>&@)B|q4vzw}-6oZ|b6Z>dGGX#?%a$>brlN>XOg)@3%kE;kX4|EiriSLSPYSAEi(Nv z{}(hIk`i7GKq`}ZoHo5~3&m+EQxA3VpUj@6BY^LWT_mIj2qOQrgKINlj-P}ok^@CZ zu435NptY9HJzlVl~OZzpa>*nlEeI6l#Znq>` z^c~#r)(LiQZb)ULYQJZ?ak@x7z=o$LoO#|i@gXuIy_TmSmX#4j!rIZXle;2H;gIQ* zKdc?Y zbRlflZdy|K73GPYB}q%`)xpE?Hp)*qfcE1gl~eJ3^i;(m3wd_fZjEJPcGjtj(qlARf`^L32?Xk%B z^>ymr%1Hz8E+QEbj>=am6c4yaMUbh426GwqcM1({h+d~_fL!Fo9q1+YPKw3Tgxk2r zw8~tOUy_Jic1Gz@m8zmtig?`@y7d@BWF1Cy-yxom`NHK*30}m50=dKop(yj1#43eT zLeP0TobxB~OB7m*!%J{gfKF&ekNsl-TLe_uxjikUz}ol0o>H=3Ns8>cSK*v^G24?f z{$P5OcJ@II&J^Duj@uCRpiZLUAcLs$_~sWGTgt&dnQgNHi-%@~Z-w=he2_9aq{*zh zgyGpYaS0C4Ydj%YI)%7KFV+BT7!3q&3WnyH){H zT+ulJRgqeh8*RkNKj`=soge9lVdXwnYH(?olC&HJgKviVMlNXioa4K@DtNzZYrJv4 z=<$5iy-Kr(F39>YtM$+DJ%bwaWaZp;%;^L!DR;#Eg=DKZ*Cx#afv~}+BVvz9E-u40 z*7}*lsG6XC>sPDC&|N(RNCKgX7IFA($UOX~-TKA;`#pRR+H_j11*MsvGs3DHkFS`Q zWw0CjQ%oA$nE0RF+(qo)aTuxRyvRf6;eko_ICC+)Bx;DZQ@L0PwxO0EjE!5-!)qQX zKvCZ=@hv9)lT{kL+HV^K_9WhwGCt~SOOA}CCs*`!>UIN2XCtSYJTdut9qqQ1n@UFy zG1|>(YVzl%Wb5sfZKj%&sO+o4@7}}J5iX5KMQ?W3tmG{kMu%L%U%W#_1gc@KDL={9 zY2MbB?lOS28~GOSkS>4&uY1GMC#TsjMoYh*gZY9oI1-|kv53!TyDJrgKfy}3rzJJJ zLS_t)LrTd9C7d@JrgdL{sFV{J_R|jP!`HjY9#7zZ31>GIH9oTO-|j+&EWXqb;w!2V z)&`7tKQq3&weUyK^W0z)?Sn0li@}{4mp%eAiy|YjMhL{a7y|Ky%tO|V(OQyI?Vn)a zC4W_y{YU$2s*{8+yZ9DvnXWnm(d&6Gpe zP)bllX+;4QDOr?5mr5gobPSzCgF~F_X2p)R*0c9M=lPuHyg2LKdLvW!egCgte6N|u znfOZ@Cx}bUtwUTQJyfQ)dN$~-E#(k>2?=9i{ta6mbV0O)YDsX}>YlY+!H&m-VUIaY z|5xRU5&H{%QY)}t@1vp#9&6|j7%MZ6Y&$u&r=pqFA!zks=5D(X_wS!+Z#L4w+4S(s z%{S(gAx9p3a!#N(v_I#P@#Vev9JQfDp#53b3cPseb;ijfGuN&$)~eTf=i3#C7nZ&; zO0W(PCc69Ap0P>xGAGY&atRn~tbKx!OV|2R7!B#`GePJau?p(TlQM_rPiSn!YLV|Q zX1FfmAaf!BpHo456)#OssG+|o?5lE$PQ}NYn6SL*dQWPX<@VH(UHpV~FKphM0ls((&P6y%w610|B;4Ee6_Hi?GLwsahve zX}onqysjO2-!Vkb>nuKor}9%x4&q84JDpCQnKY=B`r?(-c?~Igomuyhh>i)oxHqYa zkcUVGh=RexIz6>HKUCaI*N4@W$<>t&aZH7n7p%QS?^mFiJIhqOza92v9G z0|`OLI5p=}trQ(c>1s)&{_Q^Mmv$G3DVmw+E6rjlv|q%Y_!kKFnG8sd%&;1wh;JK? z1YMcjZM{pEYceF;IYiW<#@?GRYwzf*J|ac7$E;s&!YP$R^TwZ7nywzO@H49ZP~l?Z zMd|y{?a)O0LetLgere0}Zkkuu6PH7c2-EAEfvnlH9)C7L@tMlAv?5_s@0cB+?4Y|W zCQul(JtIx?M>8goyv1*fmk;|B^uP9?*$mTNc*!-+{ZDu?eH*F#;2XrVR6~6<<&Ck} z;p-W1@WOG|e1EAw@Ep-vN3~qOg!abp)U}!Ye=p{L&WmZL`=j>)ZXR#iFU{My784Vb zEC@+9(=f1Oo8Bf^V=)3gR!^a8+e1FnPjs6MV$>p~5!Ce!| z3b%@Htdd`v?1&8a_dXeD`qPlp&4*8)pRBe(S_VJYLC8u@d{>tE>OY>Jfttf`3X-+If*5a4W0=V>551Ga z<+@6?eoR3i)Q42ktrX#dYl}QU)0j}ScjP0Ci6AgS_>}4mm!J1a@HrJa!p)HECb0DToG%S$v7---o!wG1e`* z9G-6*tVz5b^8`#aDt z$Y$Fm0dfFB@^j?$IhWOTS-mrHYl*KLSQPQ%h%g#FhWu2u&6LQll2X03SL+|IQ$6A9 zU48T5PNnkmZSr+iTbJ5qq+_x^{CLKn`C^t;Z&By#Xk9ZP3%%z+ESYv~`|bNmC1jv$ zvURn+xuWiw45fW#5{#kF)7iD@fGvYC7{?3pl>pXfDk>!x7V7q@k7Rtc<)RQ-gEto* zZ(GBfSFuarJ2X`ckit(fv10f9W&6gL!9tTML5P=EN=2eSd5q6R zKKQua?BdAbys`Yb)Jt}2rnRjfovgdzQk?lQmFd(^wg%?wG?W8=26khOdOd5R`@)0R zNdX^dZAzLQMP(J0qf`T)n}(S_c~&H|=)2(Pwp9I;uFj*RC@nKHAILfVn?{Myn`-pP zGQIfxglQxI_`0YM4BddQ;u7m`7?4{{GUwKHU1gf0#}*X})#r;WFW+?0+3%&4pUfo^ zp6w}z8s=^?cf<1pp~}QG(G(j9Nb&CU>9wF{)6{rl4cX-s?ALm2%b%lysR!7|5M&SOu!5*@L(GWoCT~}wD?-w5aD91i=MEKqA~n6*IE4vkB>{eB8g&J;Yw>l}Jr2 zr(AtMNB`@2r2e1IBmXzLmo)uvHQqI@pqJuwedA7ctcp*4OgOpPEMNyO`MZ?W8$8#MoQRj zbv65>NkH$T5TSxOwyN<}AA--H1MXXBoLo=J#dtGlIy>(R_C9XQGoTKCveHgZXGR7E ztXOE`3P+X{$?m~u#5Tn2wa#@MECR>*-_CeI>Lm~EWq*hBsBJt%fg-brF)e)lSWw)^ z;z`X7D_0qZ(kqd%xKv#2b*o9~ORfTaNuLfNHyjSTPuz}BC+tqY@gdH8vO7N$Nr*T$ zE7Eqt2?>Kdf8GOiiR`aSjlbewKmJ*ag7`2VXi+Orualbl3}w2pga~+V-C^=u&Im$a z+S-d0>jA-u`2+Cuh>c+))BF!25XKQ;{*@seBEDxxpH90eLH3qQw9u_Mt~IEo&W@G> zV=V)iFG~M5&LamUyMaJ&5qjF@kL6%>Z3T2Xe&I`_IuR*l0z3|0<$rK|>3Hhk@lWTM z#=9!voR*MaDdDgwBVU11YSrrxh44ymX?7)oCN+-H#u2I>Fk8YcE*t5!A-t6x>R@WT zqum2M-;FJJW)zy2^tn!Tn!S5(j6iUadZ=aI+&sA*^5B!+-)`iQ_mvZ#4H?pozM40p zH*N7+I=1(_GiWJ4!u_-GiLnKI`Dz{NxZhgAmIlA12x=h?GWsw>n(8F zvr-Bff{$d9=oJ+<%i99)b0f9K(vriD@>7VE;awjVAdmUFVHA+-^ zF2?p35CW07&^PK*$-IM^_=SgJHEow#jjC7Dc2C;AdHvLmRI>0zApW{D^b30(>824J zo-77HJffC@5zK}=Cm>v|hh2K@SK*h^L*Ys$q207?iye7MgN=Gw-g z_Kwr#<^t7vv+>ws%f_khhl9KQW(E8_cYW?(jIQ3VUH{W9hii8&@S;ols@d<}@#lyDg0mQqj#iHcAdd z&c}g3R37yLE7@|C7n_DdOFAq=lt8+Pt|opng6L?A{{6W6)dD(jxqwRPbzy3hBU$iK zVKAsru_l?XGv>%~umWzvk#JZ=g>K(x(H(zTE6v_FkvgW!|5xmjT_2Df&b+Z};S07c zC`^6;4vj?gI#MC}LLB5!zz71xvGKj`F6=6T1@pihWucYdGo1&G*CJi3%eM!PC8d@^ zhrBaFmGe2{Bg&9@1d;;gzp2h~HuMS<*9I5(;%s~u53FsHPYvZDM3HzDv0w1I!To)4 zs=kZp-VGE6y#VKyIT?;1AI#IazT%I}2fb6N6z_8-tJ(HsFd49$`|ovxAYk!H5kXpRF$WobDaGE{xUoJ8P{w2uTb;%0T$)W&6b zbt4DP-JQneuS1WXH9F5Y;>>GBdy7W2 zS(l$RhxQv6KdBO0$ZLwOJs2{yagcp`1!)MWJkb>`{%O>xUIc8wkUL_D3VD~y#sg-p z8GVy7GL|hjfOq-PgB~kVw)|yxgbt&~KoDJsOSVy^n~xTG6BW(=EHEhdOE zKY&N3;|06~M7e*2p0+=-s*q+#|GgWI@r7ZwJinuWW(pIILj;quP0mrO>J~_r} z@XVK+p4^<$JP*bV@+J;E1O&j*8Pjk3*|%#%myoqrTg@ z!kHyG9+UA2o(HCb2bu5;_wgvDplSHv$h5}cVe?<2WsHb~e(XUsJx!SY$&wB-tjZ`a znT-p9t}HF;#}BqxT8{?8i=^e9>)pUQ=LOPI z{e$!K{cj_Weje~uF&}_jdp`MN~l3NU}&;6|HZUpG*9R+zyo!{l%*`7 zrIMO)$xhTAE8DjpCelHl%evdK%sWm?6WlC^(Z+jRC)`iV|KhcjHhUdXmLM`6G(@-G za_TK;>scRYRHb6NKi5xK#SOiXOJkv;=pJPw^K?$aQZ3DwA95w(FO_2gStO_HycGE9 zmP7fCNSmb9aCP800UhR?VrCE9oetjMKO-v^e>C%+w;p)sv=ma=LBht?(1$5{0#ZsR zQ*r`4GsNLXBU}vmkttVmnsknY5Xb~f=cZoEH zx76MVU-elZ{4P_;UFPnue4K$|a3ByRGc3A*=7afkD~P>>;gd}{7#p_%m8$j5UfF$( zOi6i{;TbsezyUWvy#And*7HB0bB>7Y=b%;zpSPAF3z24oEF1(rXmFL`gH z5183$mA@=n$@si_GC$8wvCHL-5Pi=qAXpPg%A(tsqpj(Pv+`3sg4nR3c()RoXG?}Q zx^Xy~SWbCsMSo2VXTBhFwP3$8o5coi#1G-1B;DB#VhXtSn;Wo}K|c1$iD^xjBJoB| zU778I+MuRCikk}J^RUK2npcPGb)(T9jm4@`mwj2yQ{L29YKLk|eFn(_sFm;rR8`)L zaj{zZM3L_sII0(l18lLGFrYw|5 zY0d~{A_zqwPF`$_isXc_w{ycihwA?0I{$k^B&PFqMn>a)9UDdM zJ5kqmrMx&DcoW7jOP~mUNZc6an~!<_9BC*3GbrOdf;lH#=9a>NF`x98L4EUwolA7{ zD(2AubMJyj$6e5!vmlm#6IPnM{`yW!_-?@*fH8b0SW4IKYdZoQASddlo1aHy5fHAq z5%xeKa*|h73~D_8U$G%`siayy#c4tsjdh(ZA!Vi8_TO> zNyrIc>qkdv7gBUUW}(niUHl0G%44rmsL1{~6Ti>l+RnC-f22hEn=saFJP_^#Vbc;~ z@ekDRegMkJcYw)F$xtJS#%~=Rwf6qwasMHhd}kfWp1-XlxhE&)x@4jaQSDns>wh;{ zW_|n9$ujiA{)gu^o|XcTW;P}hY9OFJzuQUX!~P7f*cmr0Ks9>oE#dy?P+Q@e-sIRt zfWyB-?M2MdsT}#RRRo6Ez3dJ;Tplu2D$9LFz)ArpQJU&SQK=VVN5cX=*7U_1M}EUo zatM?9U{8fk<`O7`j&)>p9}@blu8XHvO4_GWHSgyw>_c{_gy;il3$JX>e>d&0j9kh_ zij1XfU#`AnNLJ}#L}V2(a)qgBXUT_fZ%pnFeU3aP-9W=;k{1v*;=1KKNr7@6Kfc2s zotU(XIJ0&(8WW$0WXn~1p6UDJ|1hAohjwoG1~`8{6rHOfw_f$jj7r^;hD68fypWa! z43mZUz~k+_*mruRp0=j8L0{yivnr?I?CV710P6sDz~8v-HPw%BG!(#QWh1K?LIQx(tnw$Q)*Fkx$!Abs!Y(uv;>g57Z#|u5gwnDgFcSckfG2eMh^bTwMr1oN_*JhpP@;kjk zdtEpP+)}!3n+`cnyazn7=Of>P7_vo}tmv57C6KO8e;L3TbvA}HQ=gw@Z>Y2h%tcimdel0)>| z41t$rdVz*(eex7M?^lJD_k$K7VqA~yAUg>W?}W;V-Kw&aVXv-*%N%p@bvOC2xNhPZ zYgtXauecheD1U!=ZoLl36UHF3y$$7%&bz=H^1==L6^j@5_uDOs*l|OLDZm31s>U?O zsrF++$Ix|#SVq|;P4O@{m_{X%!)=pmU685x^bjI~BZ5N>I@W+7Q6)NYkk@gWi&XQsh^MAFyGcw{CH;ghv&l%(=ohgQvv{t}Ow4^49B zVz^56O+nxzW@PAkuHPAxya7dw-mSCFA0=o)4nycvmpp`9C6ZI37WrmAO07d0ak4SK zUoGt`l7(TA8t2lAHi)O_emOQ>E};kYr$#~!hydT=+A>1o`E25MWvlj#&BM1O=HO)m zn}npwtJZCF1#3lj#^1AqYrEz<3=vohfl zZCmMf6sS`#=2Jy^T=A8I@t&@;f6iG!midU7r)LMay?>}HRD7}t1p2#39Y|;q5sjh- zle1MY&EtfhgsDTMGqmMUBWy&WTSea?|7-#fd^eYnyx@1~RD8Qi0afP&s4u!bNr53r z6^~lGmu=8N$NUxdR3W=I3_vRU_>P%z%6=B=12zFC_ns}A8>|e>zbTx41y4Wth-QfD zl0qg2qvGWd8}~N)%%MZoet|{?&-;qkf85y}M?w5M5QFzKiHyN(q|s&FgsK6!}jZR^_Ul8yxb90`|7jx8%;T#Ai{IJNwYgv}tKq-HI^ z$}@yoRu;{Vmp|-y-9F4IxHOKPSM1_=B9HTFolZ<{9+rK*zxi2?*+cJ`z>B6GdpBNFzMZNjF zcVfwum%O!Q-rueJI9Go+jog4qDSnT(=DBzZUs?d}kk&uqhH_6%?kl7PPelDjxe%y0{4MMU0pLW6ej<@Y=(nBKVr(>2t|m~_rHY&;Y`yJqc=T_R z|55Z42h6bGYSh$@RZyk z)OG0|lnB67247lknR2G)U$ydVe6DahKoVa|F8m;qFXXuFqzD7WrE=0Z`d59xWD9n} z6zDheG@Xedw+y$Ne3B$R88y|JU~<}7u}f>|i{wT4CE9)#{&9#o;qeLVk1}RO4wi+R zsEBLu!exNMWHtTs(a=)f6J>oOlwUu6%2)GzZuq9hK`P8Cr(kM8RJyk0kE1M_IG04v#A`DHq z?lGD$+q=JL2&K$)WIaXB6e4DQGq+4JxPlj=6{bEW39r=IZDx!_5PEcn z$NVDTgGS4AFDAw0$SEE^ERMQ2?~$jjs&Os{6|y*jZuJLWa8#NrM4ilG^l{9iF4OQo`qNw7T~a&dW*Q9UP#C}DLQzt(grgU>Z9+h zuSP-K;!c{l#TqW;%rR@6ff1D4m#0mLa2)wXPoK%F&|)H*}L?m_>!aS;-7(cnY+iG z@xf7(4QlyLuRHDPj>VG?7JK-9s>uAxebweYFGZxt#d=jQ ztd}usBcFS5M8c`LPI=G|XX6)!V<|!wjcjZ>jSfD?sdS35Q`D=TJqY3*gJ)3LX-=(E zm2^Sg#C=i<&N^U{&0d#HUxbeCN2lqD;zhGWmfQBb3LE&KnHKRkIIE~{oj z#Wi+_33AblI=UCgig02rN?4lm$*`wK2`37gnJDnO(bH;fHibsF*LEeX()7TaZj1_3 z98xD7MyGZ}0I+e0n^Bm3Ul#iAe`an(Z?d<;F zGZ?;XU;o_mJA=V^J6}42elI1L|1-8zwtNZpoXK=qbbU`#3BJnr zp{Om^;Z|D=2`y%hW#63T*mrJc-CHS$ecnB|}* zJG&BIJ-+x*J*WI{5Y_vb&R2)2%7}xNmdLVBkgc_{QT)vRa<8g6OU4w;5aw{*K1Yzc zAVJvE%v8fjVJ! zcjqIOZX?*V=6{|0EAyd5c0hfA(82e12wN3g4O%du609k4?uVLXJnx#z!mTK(W+8NlQCr>e9{dVZ>Lj`e+zNr$tc{gV^f-cP2`>G z%E_3U{@Cb#yMFV;vBbQPx|*-+^}q6D^C=l5%1VHr0o8T!KIi1!Wj_`Zx7j%ir;Oqx zY+oWn;7%e><)h|p*PJ*mp$tjITOd2d%)qd0bzZ#CqvMf_~5bh5` zB!Y8h!O*V!TTQV4<2J}B@#@2mWgx3&b=~NfCAd$(3^)CnQb;hs2^TL#3J`P>5Ii03E z+Ovh&h1_N5uf8LQ%(ro}{EPi0{Bmy{p|i?c+Ab$!emZ6N_uFs>8?poG_nE$b`y3j zr6Jn{|EQCI*D;d*i&FBNoj?im%UeMfiz)A@c`}Cic@$Q7Rf1)r+P~>eCcNA zYLu4lL!L1QlGLYZIDrQ_|o*O{&UPjvR+Q!fC z^w{#y2*aj5X;|afov!Mk1FQCCivpKL5z6l^Ch9FIvf2+&i|>M`a>uCU9_7|wi}5z3 zeLPcbugH*Pmge5D%2KDPJbcmNzv#s`>A~Vj(T?w7pqE_ZMPrM=p|9N!J7mQfd07@< z?EMZO>R^ajzE$r~Yekk|K!j~C@>h4FeM_QNOA^3-`RVRinF1e#QQE;-9 zPC7yQn8|xg6Y$iU3dze^ts~8PQV3TyLeQn74C$2|iz9@b3+I%Ln}3zsAQ`AHq6ZP2 zDM${%EC(3^Bj-1XNG|%@H^_xf%kaTszL(uNI%}3%@0O~vL*)YAqE%;GDaWoOyNR{< zZr)^tBN&w~n#~3hKi5IHM8&eHNTfWHZb(h+Xfr8MCWg z5c10TUL4Vk)AzyRApUxok$yer9GUKGjMp*5*U1;w!|7BanB92N$qpR?6`zeTlCsSZ zl59sR>AkofMzGxT77yA?n`acJXd}12old&UfO5E#1pcc=Q?aJmoqB@%^@Jv|2dNBt zoQF!^yDyLgfdjVB7y;NP$hs9F3JRmF2t|&~b1wjMzenRjPx^YR;PPFM==XrEYQCz- zUt{+Ic)mz*p$l@xJu1IP7bH)Yi@J)`09Yi>vS=d3_Ope^cc_yG)*+jg8_frhF8N1y z*u!^^r)>53gd`ziKn7aT6#fbF<#(4NnJBph`-|P2_|AHB9{1gK`kZw~r6^Y(hDHse z4K7%qn0woUz-37%S3I^|lZ(6*(gl5H#aS2UPx+rd5X7uZUVK-EN=>w4{*rjX;Nzk4 z%Iqv;>EUJ$b)yHQCmnV7%<_hp?nhn7&?=D67=yIc$gF&Z{WvQt$Qo>%W%ho7-bKm^ zTgwx1UHaT*J5W*U5jsx~PVsJ6SFGl!(YU)Xy@5g#3ya9& zIiEg>;^}5AB{aL6W0wP>H<1dtP`FPe=3x0*1DVL?UdOOE^<`(1Z@<)P@^ml5ExQ3@=A+RWDv=Y4=P5*K z&}HeR1z{h#pjVvO6&i5w;X{6Jv37zIP+9(4I3ST+k=2J!_YpxU8 zhrT@CCq!c%I3PFreS9Jsm_*Plm^t&n2qlWgUc(iK{n~A&J8}D2r5igja?Q1Z|7$V> zjGg#5UxbbVN)d#&{aM^He1r&NXu^e_6KB$mPa+b5$K2VNWgHA0#2e zTD>QM8F7MVAiH+&w9sd6rT}Ya2P`9XT}uWk;trhIFrO+bwWDhKnGWb%Kl3rYJ=}e; z^nCW!St$-xiLDEn(PZ*9&PQnb6~qz~4hDoI@W~dq(5VD(M)`?r&J;_jW-=Q!uH}jc z#c7O>s#0Z#ut<3p_%EiestMzm2B$vzf$YzGTXimK9U-s;x8KEoWS8*jSwk^f9~`5q zqM++=>6&uj4m!Z^;GBxty1CY|C-_jnAq=`;+h?FQmzxRCtwW37ZF+jI((MOu1T0Vj z5r6dP$t?`qpnG^i;_#;67vO9}$9_-fFG*DSJeZ0|T{QzO@2bm0XX{rFkX3%#aN;3X zs0`*KOCIg*i=vtLVLQMZw?Al9t*DB;L#_tlLJ1zF5aFz6l%%@pb)J` zi+-%RBf^xgos3v;r}XyN(>Z5!L^kG7G z0p6u4Mi;BX$Rg8RS4}jK2wJF75CH@zPqwHGieaVl-5oVQI_y53YBkJZc7c}7&#Rwg zQM4~7AJ09umn&FgQlzS1Uu~E4N`j~d`gr$YktU>F%cUY$;IB9wx=UxC8}(2rF>gfH zYcigo;9XCzJZ1mh<$czptf4r%5`yPhH=`Hq7!}mP3}(Z(vY7L_S1*^EcEr0q;pU%g z)nnn#dH7ptcIvt6k$&~?vGoOaIew=jy7?E*353@TacZbmu89-BNxNzdPI(j<+WHwi z!nq7Hb(mi)Di&yXs%4mvUX2w%(*B zobyPGv-J!)Cgc`!aPhR(iGNCjtWV-9>983)LJX17TH+_hDH(`b&T_c9&f&Yps5|3) zzYo4A4~y8&RRvu7#v3-YFkx<_M(<#`UoI>PkYJ?kUbfb8#)b`aqX3Hkm~=sQv7NT^ zxQ>$?*Irj!i9RId_wCTKT;`l2z&clA#_4 z8on@5PBPujb1DzNem7hbja!-=UAAG6okzBljXS_!-giF>3vcPZcZqV@U_5jed zy`qKt!9XNjBK1qS#Er_GUSy(r052gQf~JYi=Tb`J!!uX?{Hj_}e;B1-Nwh^zM{n|~ zj9;>^|D0_Z!fMp#ZV*0i7qDAo4SuKjdw1xgbNlO$6%aaUdB}vl8FL9!UenPU(u>BF zIP;+^#rC_$)HpDO;k4c}A8;$Ro~)~5@mtNj6)rhm5`aF5JDV&d@zb8#hIxC=xNyu3 z6^bMtWtvXbL5d>|oR6Jj__i9}dEeKi6?`VOUw(_ez{_@DpmQZE$fuA4XT=yoOB2gj z(JXKq{XG2ZfRZYT=z1%MZ>D@CEMm~Zc8;FQj9p8LCcfhnl1^#k6dD_GdunRsB+pLK zrgfabm$4n$t|ru!k6y#AV2_4K7H{O1NrkbT{#0w{ojEpt;GnGI*76KFEqk-diTWct zs=9BI`7WH0wZ-AhYQM{GXZmC=T$+pO!a3CcZ0O}2h1E6i@?7&b_c=>kn`*S>!eUIQ zUbm9QoI4q6^@`_}mFne@_3s?kvSx9=_cV#NOnUJrypLR)VpI7gc+}Pr2^77ocy%~hO zeU{z3DL_zpsSHv9d9f>6l-%V4c35IA literal 0 HcmV?d00001 From 6dc6c60f8de4ce4ba90541ad0e102114d10599b0 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 31 Jan 2024 18:15:23 +0000 Subject: [PATCH 04/19] Further work on ADR --- ...0003-windows-tfms-and-desktop-framework.md | 155 +++++++++++++----- 1 file changed, 114 insertions(+), 41 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index f35c4bdd3..6186ac01c 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -31,82 +31,90 @@ The following sections address all of this before moving onto a [decision](#deci The basic problem is described at the start of this document, but we can characterise it more precisely: -> An application that references `System.Reactive` (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). The [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true` +> An application that references `System.Reactive` (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). The [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true`. +> +> This causes a problem for self-contained deployment (and, by extension, Native AOT) because it means those deployments end up including complete copies of those frameworks. This can add many tens of megabytes to the application in its final deployable form. This is especially frustrating for applications that don't use either WPF or Windows Forms. -That "or transitively" is important but easily overlooked. Some developers have found themselves encountering this problem not because their applications use `System.Reactive` directly, but because they are using some library that depends on it. +That "or transitively" in the first parenthetical is easily overlooked, but is very important. Some developers have found themselves encountering this problem not because their applications use `System.Reactive` directly, but because they are using some library that depends on it. Many simple and initially plausible-looking solutions proposed to the problem this ADR addresses founder in cases where an application acquires a dependency to Rx.NET transitively, especially when it does so through multiple different references, and to different versions. ### The road to the current problem This problem arose from a series of changes that were intended to solve other problems. We need to ensure that we don't reintroduce any of these older problems, so it's important to have a good understanding of the following factors that led to the current design: -1. the long history of confusion in Rx's package structure before Rx 4.0 +1. the long history of confusion in Rx's package structure 2. the subtle problems that could occur when plug-ins use Rx 3. the [_great unification_](https://github.com/dotnet/reactive/issues/199) in Rx 4.0 that solved the first two problems 4. changes in .NET Core 3.0 which, in combination with the _great unification_, caused the problem that this ADR aims to solve #### Rx's history of confusing packaging -The first public previews of Rx appeared before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your installation process. By the time the first supported Rx release shipped, NuGet did exist, but it was early days, so for quite a while Rx had two official distribution channels: NuGet and an installable SDK. +The first public previews of Rx appeared back in 2009 before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your application's installation or deployment process. By the time [the first supported Rx release shipped in June 2011](https://web.archive.org/web/20110810091849/http://www.microsoft.com/download/en/details.aspx?id=26649), NuGet did exist, but it was early days, so for quite a while Rx had [two official distribution channels: NuGet and an installable SDK](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5). -There were several different versions of .NET around at this time. Silverlight and Windows Phone both had their own runtimes, and the latter has a version of Rx preinstalled. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: +There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: * The scheduler support was specialized to work as well as possible on each distinct target * Each platform had a different UI framework (or frameworks) available, so Rx's UI framework integration was different for each target -Some of the differences in the first category were implementation details behind an API common to all versions, but there were some public API differences too. The second category was all about differences in the public API, although at this point the UI-framework-specific code was in separate assemblies. There was a common core to Rx's public API that the same across all platforms. +Some of the differences in the first category were implementation details behind an API common to all versions, but there were some public API differences too. The second category was all about differences in the public API, although at this point in Rx's history, the UI-framework-specific code was in separate assemblies. But there was a common core to Rx's public API that was the same across all platforms. -This meant that it would be possible, in principle, to write a library that depended on Rx, and which could be used on all the same platforms that Rx supported. However, it wasn't entirely straightforward to do this back in 2012. +This meant that it would be possible, in principle, to write a library that depended on Rx, and which could be used on all the same platforms that Rx supported. However, it wasn't entirely straightforward to do this back in 2011. -This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. Understanding which component your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and was something of a barrier to adoption for new users. +This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. Understanding which component your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and presented a barrier to adoption for new users. -An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component, and the original idea behind this was that this would be a stable component that didn't need new releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was correct, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even though nothing changed. The defeated the entire purpose of having a separate component for the core interfaces. +An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea behind this was that this would be a stable component that didn't need frequent releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. The defeated the entire purpose of having a separate component for the core interfaces. + +(In fact things were a little weirder because some of the versions of .NET supported by Rx 1.0 defined the core `IObservable` and `IObserver` interfaces in the runtime class libraries but some did not. These interfaces were not present in .NET 3.5, for example, which Rx 1.0 supported. So Rx had to bring its own definition of these for some platforms. You might expect these to live in `System.Reactive.Interfaces` but they did not, because Microsoft wanted that package to be the same on all platforms. So on platforms where `IObversable/er` were not built in, there was yet another DLL in the mix, further adding to the confusion around exactly what assemblies you needed to ship with your app if you wanted to use Rx.) The other splits were also a bit hard to comprehend—it's not obvious why the LINQ parts of Rx are in their own package. In practice, anyone using Rx is going to use its LINQ features. The 'platform services' part is arguably slightly easier to understand because .NET developers at this time were generally aware that there were lots of flavours of .NET each with slightly different characteristics. Even then, understanding how that worked in practice was tricky, and this was just another bit of complexity that could make Rx harder to use. -With Rx 3.0, things got a little simpler. It settled on NuGet as the distribution mechanism. Rx was still fragmented across multiple components at this point, but Rx 3.0's simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. There were additional packages appropriate for using specific UI frameworks with Rx. However, there was another problem that this did not address. - -#### Plug-in problems +The NuGet distribution of Rx introduced a simplifying concept in v2.2: Rx was still fragmented across multiple components at this point, but the simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. For example, a single reference to `Rx-Main` v2.2.0 would give you everything you needed to use Rx. There were additional metapackages appropriate for using specific UI frameworks with Rx. -Because Rx has always supported many different runtimes, there were several different builds of each component. At one point, there were different copies of Rx for different versions of .NET Framework: there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. +Because Rx has always supported many different runtimes, there were several different builds of each component. For quite a long time, there were different copies of Rx for different versions of .NET Framework. In Rx 2.2.0, there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. -So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (The Rx 3.0 [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively) And then each of those packages contained multiple versions of its assembly. For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders +So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (The Rx 2.0 [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively.) And then each of those packages contained multiple versions of what was, conceptually speaking, the same assembly (but with various technical differences due to differences between the target platforms). For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders ![](./images/0003-Rx-Core-2.2.0-contents.png) -It's the same story for [`Rx-Interfaces`](https://nuget.info/packages/Rx-Interfaces/2.2.0) and [`Rx-Linq`](https://nuget.info/packages/Rx-Linq/2.2.0). And it's almost the same for [`Rx-PlatformServices`](https://nuget.info/packages/Rx-PlatformServices/2.2.0) except for some reason that doesn't have the `portable-windows8+net45+wp8`. Each of these folders contains a version of the assembly for that package. So `Rx-Core` contains 8 copies of `System.Reactive.Core.dll`, `Rx-Interfaces` contains 8 copies of `System.Reactive.Interfaces.dll`, `Rx-Linq` contains 8 copies of `System.Reactive.Linq.dll`, and `Rx-Core` contains 7 copies of `System.Reactive.PlatformServices.dll`. So conceptually we've got 4 assemblies here, but because of all the different builds, there are actually 31 files! +It's the same story for [`Rx-Interfaces`](https://nuget.info/packages/Rx-Interfaces/2.2.0) and [`Rx-Linq`](https://nuget.info/packages/Rx-Linq/2.2.0). And it's almost the same for [`Rx-PlatformServices`](https://nuget.info/packages/Rx-PlatformServices/2.2.0) except for some reason that doesn't have the `portable-windows8+net45+wp8`. -This fragmentation caused a problem with plug-in systems. People ran into this in practice a few times writing extensions for Visual Studio. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assembly in the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. +Each of these subfolders of the NuGet pacakges `lib` folder contains a version of the assembly for that package. So `Rx-Core` contains 8 copies of `System.Reactive.Core.dll`, `Rx-Interfaces` contains 8 copies of `System.Reactive.Interfaces.dll`, `Rx-Linq` contains 8 copies of `System.Reactive.Linq.dll`, and `Rx-Core` contains 7 copies of `System.Reactive.PlatformServices.dll`. So conceptually we've got 4 assemblies here, but because of all the different builds, there are actually 31 files! + +#### Plug-in problems -Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. +This fragmentation caused a problem with plug-in systems. People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it. Any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. -Here's what would happen. Let's say a we have two plug-ins, `PlugOneBuiltFor40` and `PlugInTwoBuildFor45`. Both were built with a reference to `Rx-Main` 2.2.0. That means that if we were to look at how these plug-ins looked on disk we'd see something like this: +If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. + +Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. And that caused a problem. + +Here's what would happen. Let's say a we have two plug-ins, `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45`. Both were built with a reference to `Rx-Main` 2.2.0. That means that if we were to look at how these plug-ins looked on disk once they had been installed in the target application, we'd see something like this: * `PlugInInstallationFolder` - * `PlugOneBuiltFor40` - * `PlugOneBuiltFor40.dll` + * `PlugInOneBuiltFor40` + * `PlugInOneBuiltFor40.dll` * `System.Reactive.Core.dll` v2.2.0 (`net40` build) * `System.Reactive.Interfaces.dll` v2.2.0 (`net40` build) * `System.Reactive.Linq.dll` v2.2.0 (`net40` build) * `System.Reactive.PlatformServices.dll` v2.2.0 (`net40` build) - * `PlugInTwoBuildFor45` - * `PlugInTwoBuildFor45.dll` + * `PlugInTwoBuiltFor45` + * `PlugInTwoBuiltFor45.dll` * `System.Reactive.Core.dll` v2.2.0 (`net45` build) * `System.Reactive.Interfaces.dll` v2.2.0 (`net45` build) * `System.Reactive.Linq.dll` v2.2.0 (`net45` build) * `System.Reactive.PlatformServices.dll` v2.2.0 (`net45` build) -The critical thing to notice here is that for each of the Rx assemblies, we have two copies, one built for .NET 4.0 and one build for .NET 4.5, but crucially _they have the same version number_. They both from NuGet package version 2.2.0, and the assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Core.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. +The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one build for .NET 4.5. Crucially _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are in this case.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. -Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second one plug-in, `PlugInTwoBuildFor45` first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuildFor45` is asking for. Now that was the `net40` version, but the assembly resolver doesn't know that these are different. It assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuildFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` but not the `net40` build. +Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, that will be the `net40` version, but the assembly resolver doesn't know that these are different. It assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` build but not the `net40` build. -This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that a .NET 4.0 component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugOneBuiltFor40` and `PlugInTwoBuildFor45` would end up using the `net45` build of the Rx components. And that would work just fine.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the entire point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. +This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that when a .NET 4.5 application uses a .NET 4.0 component, that component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45` would end up using the `net45` build of the Rx components. And that would work just fine.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the whole point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. -It's worth noting at this point that the problem I've just described doesn't need to affect applications using .NET (as opposed to .NET Framework). Back when it was still called .NET Core, .NET Core added the [`AssemblyLoadContext` type](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext) which makes it possible for different plug-ins each to load their own copies of assemblies, even when they have exactly the same full name as assemblies loaded by other plug-ins. But that didn't exist back in the Rx 2.0 or 3.0 days. +It's worth noting at this point that the problem I've just described doesn't need to affect applications using .NET (as opposed to .NET Framework). Back when the thing we now call ".NET" was still called .NET Core, .NET Core added the [`AssemblyLoadContext` type](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext) which makes it possible for different plug-ins each to load their own copies of assemblies, even when they have exactly the same full name as assemblies loaded by other plug-ins. But that feature didn't exist back in the Rx 2.0 or 3.0 days (and still doesn't exist in .NET Framework even today). -Rx 3.0 attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have thought that this would use the fourth part that .NET assembly versions can have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: +[Rx 3.1](https://github.com/dotnet/reactive/releases/tag/v3.1.0) attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have expected this would use the fourth part that .NET assembly versions have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: ```cs #if NETSTANDARD1_0 || WP8 @@ -128,40 +136,101 @@ Rx 3.0 attempted to solve this by using [slightly different version numbers for #endif ``` -By time time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. +By time time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. -Again, it's worth stepping over into .NET Core/modern .NET at this point to see how things are different there. Those have a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) +Again, it's worth thinking briefly about .NET Core/modern .NET at this point to see how things are different there. This newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems. But if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. -For example, if your application targetted .NET 4.6.2, and you were using two libraries that both depend on Rx 3.1.1, but one of those libraries offers only a `net45` target and the other offers only a `net461` target, they now disagree on the version of Rx they want. The first wants Rx components with version numbers of `3.0.1000.0`, while the second wants components with version numbers of `3.0.4000.0`. This could result in assembly version conflict reports when building the application. You might be able to solve this with assembly binding redirects, and you might even be able to get the build tools to generate those for you. But there were scenarios where that didn't always know what to do, and developers were left trying to understand all the history described to date in order to work out how to unpick the mess. And this also relies on the same "we can resolve it all when we build the application" assumption that is undermined in plug-in scenarios, so this could _still_ cause problems for plug-ins! - -The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different targets. These might be a mutually compatible combination (e.g., if you use components targetting `net40`, `net45`, and `net46`, they can all happily run on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. - +For example, if your application targetted .NET 4.6.2, and you were using two libraries that both depend on Rx 3.1.1, but one of those libraries offers only a `net45` target and the other offers only a `net461` target, they now disagree on the version of Rx they want. The first wants Rx components with version numbers of `3.0.1000.0`, while the second wants components with version numbers of `3.0.4000.0`. This could result in assembly version conflict reports when building the application. You might be able to solve this with assembly binding redirects, and you might even be able to get the build tools to generate those for you. But there were scenarios where the tooling couldn't work out what to do, and developers were left trying to understand all the history described to date in order to work out how to unpick the mess. And this also relies on the same "we can resolve it all when we build the application" assumption that is undermined in plug-in scenarios, so this could _still_ cause problems for plug-ins! +The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different runtimes. These might be a mutually compatible combination (e.g., if you use components targetting `net40`, `net45`, and `net46`, they can all run happily on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. #### Rx 4.0's great unification -Rx 4.0 tried a different approach: have a single Rx package, `System.Reactive`. This removed all of the confusion that had been caused by Rx previously being split into four pieces. +[Rx 4.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.0.0) tried a different approach: have a single Rx package, `System.Reactive`. This was a single package with no dependencies. This removed all of the confusion that had been caused by Rx previously being split into four pieces. -By this time it was also possible to sidestep the plug-in problem because by now, there was no need to ship separate versions for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET 4.0 would be suboptimal on .NET 4.5. But by the time Rx 4.0 came out, .NET Framework 4.0 was old and there was no longer any need to support it. The oldest version of .NET Framework that it made sense to target at this point was .NET 4.6, and it turns out that none of the new features added in later versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targetting different versions of .NET Framework. +Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET 4.0 would be suboptimal on .NET 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it either. The oldest version of .NET Framework that it made sense to target at this point was .NET 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targetting different versions of .NET Framework. -Since there was now just a single .NET Framework target (`net46`), that solved the original plug-in problems. And collapsing it all down to a single assembly, `System.Reactive`, solved all of the newer problems created by the earlier attempt to solve the plug-in problems. +Since there was now just a single .NET Framework target (`net46`), the original plug-in problems could no longer occur. (The only reason they happened in the first place was that Rx used to offer different assemblies targetting different versions of .NET Framework.) Furthermore, collapsing Rx down to a single assembly, `System.Reactive`, solved all of the newer problems created by the Rx 3.1 era attempt to solve the plug-in problems by playing games with .NET assembly version numbers. -It was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. +This simplification was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. -The `System.Reactive` is a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: `net472`, `net6.0`, `net6.0-windows10.0.19041`, `netstandard2.0`, and `uap10.0.18362`. Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. +Although it now targets just one version of .NET Framework, `System.Reactive` is still a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: `net472`, `net6.0`, `net6.0-windows10.0.19041`, `netstandard2.0`, and `uap10.0.18362`. Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. Consequently, the `System.Reactive.dll` in the package's `netstandard2.0` folder does not include the `ControlScheduler`. -This design was introduced in Rx 4.0. Before that, Rx.NET was split across multiple NuGet packages, and this caused a certain amount of confusion. The goal of the _great unification_ that happened with Rx 4.0 was that there would be just one NuGet package for Rx. If you reference that package, you get everything NuGet has to offer on whatever platform you are running on. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. +This illustrates that with this _great unification_, when you add a reference to `System.Reactive`, you get everything NuGet has to offer on whatever platform your application targets. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. +That sounds very convenient, but it turned out to be a simplification too far. #### Problems arising from the great unification -This worked fine until .NET Core 3.1 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.1, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. +The _great unification_ worked fine until .NET Core 3.0 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.0, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. + +But .NET Core 3.0 ended that simple relationship. Consider this table: + +| Framework | Which client-side UI Frameworks are available? | +|--|--| +| .NET Framework (`net462`, `net48` etc.) | Windows Forms and WPF | +| UWP (`uap10.0` etc.) | UWP | +| .NET Core before 3.0 (e.g. `netcoreapp2.1`) | None | +| .NET Core 3.0 (`netcoreapp3.0`) | **It depends...** | + +Before .NET Core 3.0 came out, your choice of target framework would always determine which client-side UI frameworks were available. The _great unification_'s decision to include UI framework support as part of the unification rested on the assumption that this would be the case. But .NET Core 3.0 broke that. Once again, a decision that made sense when it was made was later undermined because one of its assumptions ceased to hold. + +Why is it a problem? Well, what UI framework integration should Rx offer in its various targets? This table attempts to answer that question for all of the targets that [Rx 4.2](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.2.0) (the version that added .NET Core 3.0 support) supported: + +| TFM | Which UI framework should Rx support? | What does it actually support? | +|--|--|--| +| `net46` | Windows Forms and WPF | Windows Forms and WPF | +| `uap10.0` | UWP | UWP | +| `uap10.0.18362` | UWP | UWP | +| `netstandard2.0` | None | None | +| `netcoreapp3.0`| **None, probably** (see below) | **Windows Forms and WPF (!)** | + +Why have I put "None" in the `netcoreapp3.0` row, bearing in mind that .NET .NET Core 3.0 added WPF and Windows Forms support? Well these UI frameworks are only available on Windows. The `netcoreapp3.0` TFM is OS-agnostic. With this target you could find yourself running on macOS or Linux. The Windows-specific underpinnings won't necessarily be there, and that's why I believe the correct answer for that row is "None". + +As part of the [preparation for .NET 5 support](https://github.com/dotnet/reactive/pull/1291), a `net5.0` target was added. This did **not** include Windows Forms and WPF features. That is unarguably correct, because if you were to create a new project targetting `net5.0` and set either `UseWPF` or `UseWindowsForms` (or both) to `true` you'd get a build error telling you that you can only do that when the target platform is Windows. It recommends that you use an OS-specific TFM, such as `net5.0-windows`. + +Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs changed in .NET 5.0](https://github.com/dotnet/designs/blob/main/accepted/2020/net5/net5.md). We didn't have OS-specific TFMs before .NET 5.0. So with .NET 5.0 and later, we can append `-windows` to indicate that we need to run on Windows. Since there was no way to do that before, `netcoreapp3.0` doesn't tell you anything about what the target OS needs to be. + +My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available. That's why I put "None" in the 2nd column for that row. It seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target, and since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. + +I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. + +The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew that with a TFM of `net5.0-windows` you would definitely be running on Windows, although that was no guarantee that .NET 5's Windows Forms and WPF support was actually available. And a TFM of `net5.0` increased the chances of their not being available because you might not even be running on Windows. So let's look at the options again in this new .NET 5.0 world, listing all the TFMs that Rx 5.0 (the first version to support .NET 5.0) offered: + +| TFM | Which UI framework should Rx support? | What does it actually support? | +|--|--|--| +| `net472` | Windows Forms and WPF | Windows Forms and WPF | +| `uap10.0.18362` | UWP | UWP | +| `netstandard2.0` | None | None | +| `netcoreapp3.1`| **None, probably** (see below) | **Windows Forms and WPF (!)** | +| `net5.0` | None | None | +| `net5.0-windows10.0.19401` | **None, probably** (see below) | **Windows Forms and WPF (!)** | + +This repeats the .NET Core 3.0 problem for .NET Core 3.1, but given what Rx 4.2 did, Rx 5.0 pretty much had to do the same thing regardless of whether you think it was right or wrong. + +It does **not** repeat the mistake for `net5.0` but then it can't: the build tools prevent you from trying to use Windows Forms or WPF unless you've specified that your target platform has to be Windows. + +The last row is interesting. Again, I've said it probably shouldn't include Windows Forms and WPF support. But really that's because I think that last row shouldn't even be there. There are good reasons that merely using `-windows` TFM shouldn't automatically turn on WPF and Windows Forms support, but if you agree with that, then there's no longer any reason for Rx to offer a `-windows` TFM at all—there'd be no difference between those two .NET 5.0 TFMs at that point. + +The reason I think Windows Forms and WPF support should not automatically be included with + + + + + +But that's not why I qualified that row with "probably." In fact it's because I think they might not have had a choice: they had already painted themselves into a corner by this time. + +However, that change also added an additional target, `net5.0-windows` + +But the last row is problematic. This is a Windows-specific TFM. If you build an app with that TFM, you can set `UseWPF` or `UseWindowsForms` (or both) to `true`, and you can start using types from those UI frameworks. Does that mean Rx should include its support for Windows Forms and WPF with that target? + +Rx [added .NET Core 3.0 targets](https://github.com/dotnet/reactive/pull/857) in + -But .NET Core 3.1 ended that simple relationship. The answer to the question "Which UI frameworks are available if I run on .NET Core 3.1?" the answer is, unfortunately, "It depends." ### The workaround @@ -341,6 +410,10 @@ Another idea: could we introduce a later Windows-specific TFM, so that use of wi ## Decision +As it says in [the announcement for the first Rx release](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5): + +> it didn't make much sense in a layer map to have those dependencies for something as generic as an event processing library. As a result, we refactored out the UI dependencies in System.Reactive.Windows.Forms.dll (for Windows Forms) and System.Reactive.Windows.Threading.dll (for WPF and Silverlight). + ## Consequences From f96ac1e48abe2e99ea8ddcae3c78a0a0fc867a2c Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 31 Jan 2024 18:41:52 +0000 Subject: [PATCH 05/19] Add WinRT example to make point about -windows TFMs not necessarily having anything to do with WPF --- ...0003-windows-tfms-and-desktop-framework.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 6186ac01c..5387dd766 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -219,6 +219,25 @@ The last row is interesting. Again, I've said it probably shouldn't include Wind The reason I think Windows Forms and WPF support should not automatically be included with +For example, this is a completely legitimate C# console application: + +```cs +using Windows.Devices.Input; + +MouseCapabilities mouseCapabilities = new(); +KeyboardCapabilities keyboardCapabilities = new(); +TouchCapabilities touchCapabilities = new(); + +Console.WriteLine($"Mouse {mouseCapabilities.MousePresent}"); +Console.WriteLine($"Keyboard {keyboardCapabilities.KeyboardPresent}"); +Console.WriteLine($"Touch {touchCapabilities.TouchPresent}"); +``` + +This is using [WinRT-based APIs to discover whether certain forms of input are available on the machine](https://learn.microsoft.com/en-us/windows/apps/design/input/identify-input-devices). These APIs are available if I specify a Windows-specific TFM such as `net6.0-windows10.0.17763.0`. With just `net6.0` that code would fail to compile because these are Windows-only APIs. + + + + From 63b8c2aebf7f352c01f4a4af311555f91fa991ff Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Thu, 1 Feb 2024 10:18:47 +0000 Subject: [PATCH 06/19] Yet more progress on packaging ADR --- ...0003-windows-tfms-and-desktop-framework.md | 108 ++++++++++++++---- 1 file changed, 86 insertions(+), 22 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 5387dd766..866f0c516 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -2,7 +2,7 @@ When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add tens of megabytes to the deployable size of applications. It has caused some projects to abandon Rx entirely. -For example, [Avalonia removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. +For example, after [Avalonia ran into this problem](https://github.com/AvaloniaUI/Avalonia/issues/9549), they [ removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. The view of the Rx .NET maintainers is that projects using Rx should not be forced into this situation. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. @@ -179,7 +179,7 @@ But .NET Core 3.0 ended that simple relationship. Consider this table: Before .NET Core 3.0 came out, your choice of target framework would always determine which client-side UI frameworks were available. The _great unification_'s decision to include UI framework support as part of the unification rested on the assumption that this would be the case. But .NET Core 3.0 broke that. Once again, a decision that made sense when it was made was later undermined because one of its assumptions ceased to hold. -Why is it a problem? Well, what UI framework integration should Rx offer in its various targets? This table attempts to answer that question for all of the targets that [Rx 4.2](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.2.0) (the version that added .NET Core 3.0 support) supported: +Why is it a problem? Well, what UI framework integration should Rx offer in its various targets? This table attempts to answer that question for all of the targets that [Rx 4.2](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.2.0) (the version that added [.NET Core 3.0 support](https://github.com/dotnet/reactive/pull/857)) supported: | TFM | Which UI framework should Rx support? | What does it actually support? | |--|--|--| @@ -212,14 +212,11 @@ The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew t This repeats the .NET Core 3.0 problem for .NET Core 3.1, but given what Rx 4.2 did, Rx 5.0 pretty much had to do the same thing regardless of whether you think it was right or wrong. -It does **not** repeat the mistake for `net5.0` but then it can't: the build tools prevent you from trying to use Windows Forms or WPF unless you've specified that your target platform has to be Windows. +It does **not** repeat the mistake for `net5.0` but then it can't: when targeting .NET 5.0 or later, the build tools prevent you from trying to use Windows Forms or WPF unless you've specified that your target platform has to be Windows. -The last row is interesting. Again, I've said it probably shouldn't include Windows Forms and WPF support. But really that's because I think that last row shouldn't even be there. There are good reasons that merely using `-windows` TFM shouldn't automatically turn on WPF and Windows Forms support, but if you agree with that, then there's no longer any reason for Rx to offer a `-windows` TFM at all—there'd be no difference between those two .NET 5.0 TFMs at that point. +The last row is interesting. Again, I've said it probably shouldn't include Windows Forms and WPF support. But really that's because I think that last row shouldn't even be there. There are good reasons that merely using some `-windows` TFM shouldn't automatically turn on WPF and Windows Forms support, but if you agree with that, then there's no longer any reason for Rx to offer a `-windows` TFM at all—there'd be no difference between those two .NET 5.0 TFMs at that point. -The reason I think Windows Forms and WPF support should not automatically be included with - - -For example, this is a completely legitimate C# console application: +The reason I think Windows Forms and WPF support should not automatically be included just because you've used a `-windows` TFM is that there are many different reasons you might want such a TFM, most of which have nothing to do with either Windows Forms or WPF. For example, this is a completely legitimate C# console application: ```cs using Windows.Devices.Input; @@ -233,28 +230,56 @@ Console.WriteLine($"Keyboard {keyboardCapabilities.KeyboardPresent}"); Console.WriteLine($"Touch {touchCapabilities.TouchPresent}"); ``` -This is using [WinRT-based APIs to discover whether certain forms of input are available on the machine](https://learn.microsoft.com/en-us/windows/apps/design/input/identify-input-devices). These APIs are available if I specify a Windows-specific TFM such as `net6.0-windows10.0.17763.0`. With just `net6.0` that code would fail to compile because these are Windows-only APIs. +This uses [WinRT-based APIs to discover whether certain forms of input are available on the machine](https://learn.microsoft.com/en-us/windows/apps/design/input/identify-input-devices). These APIs are available if I use a suitable Windows-specific TFM. They're only in Windows 10 or later, so I need to use a versioned Windows-specific TFM such as `net8.0-windows10.0.18362.0`. (The APIs I'm using are actually available starting with version 10.0.10240.0, but I've chosen version 10.0.18362 because that's the oldest Windows build number that Visual Studio 2022's installer supports) With just `net8.0` that code would fail to compile because these are Windows-only APIs. + +This illustrates the very specific meaning of OS-specific TFMs: they determine the OS-specific API surface area that your code will attempt to use. Here are some things that OS-specific TFMs **don't** mean: + +* a minimum supported OS version (because code might use a new API when it runs on the latest OS version but be capable of handling its unavailability gracefully) +* an intention to use WPF or Windows Forms (this particular program is a console application) + +If you want to indicate a minimum OS version, you do that with [`SupportedOSPlatformVersion`](https://learn.microsoft.com/en-us/dotnet/standard/frameworks#support-older-os-versions) property in your project file. This is allowed to be lower than the version in your TFM (but you need to detect when you're on an older version and handle the absence of missing APIs gracefully). + +If you want to use WPF, you set the `UseWPF` property to true in your project file. For Windows Forms you set `UseWindowsForms`. It's entirely possible to need to specify a Windows-specific TFM without wanting to use either of these frameworks. The console app shown above is a somewhat unusual example. Another, perhaps more common scenario, is that you want to use a different UI framework. + +But Rx 5.0 takes the position that if an applications targets Windows, Rx should make its WPF and Windows Forms functionality available. (In fact, Rx doesn't support this on versions of Windows older than 10.0.19041, aka Windows 10 2004. So if your TFM specifies an older version, or no version at all (which implicitly means Windows 7 by the way) then Rx's WPF and Windows Forms won't be available.) +And the problem with that is that if you use any self-contained form of deployment (including Native AOT) in which the .NET runtime and its libraries are shipped as part of the application, that means your application will be shipping the WPF and Windows Forms parts of the .NET runtime library. Normally those are optional—the basic .NET runtime does not include them—so this is not a case of "well you'd be doing that anyway." +Let's look at the impact. The first column of the following table shows the size of the deployable output (excluding debug symbols, which get included in the published output by default) for the code shown above. The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. +| Deployment type | Size without Rx | Size with Rx targetting 18362 | Size with Rx targetting 19041 | +|--|--|--|--| +| Framework-dependent | 20.8MB | 22.1MB | 22.5MB | +| Self-contained | 90.8MB | 92.1MB | 182MB | +| Self-contained trimmed | 18.3MB | 18.3MB | 65.7MB | +| Native AOT | 5.9MB | 6.2MB | 17.4MB | +But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though I actually have no use for it. +In the framework-dependent row it makes only a small difference (because the copy of `System.Reactive.dll` we get is a little larger). But that's misleading: the resulting executable will now required host systems to have not just the basic .NET 8.0 runtime installed, but also the optional Windows Desktop components. So unless the target machine already has that installed, I will in fact have a larger install to perform. +The self-contained deployment is the worst. It has roughly doubled in size—it is 90MB larger! And for absolutely no change in behaviour. I compiled exactly the same code for the last two columns, it's just that in the 18362 column I chose a target runtime that would prevent Rx from trying to offer the Windows Forms and WPF support that I'm not using. -But that's not why I qualified that row with "probably." In fact it's because I think they might not have had a choice: they had already painted themselves into a corner by this time. +What's happened here is that because Rx has insisted on providing its Windows Forms and WPF support, the .NET SDK has had to include all of the .NET runtime library components that constitute Windows Forms and WPF, and those are large! That's where that extra 90MB comes from: a complete copy of TWO user interface frameworks, and my application isn't using either of them! -However, that change also added an additional target, `net5.0-windows` +The self-contained trimmed version did a little better. It was able to work out that there was a whole load of code I wasn't using. But there's a limit to that. The trimmer is apparently not able to work out that I wasn't really using Windows Forms or WPF at all, so the deployment is still over 47MB larger. Or to put it another way, the deployment is about 3.5x the size that it needs to be! -But the last row is problematic. This is a Windows-specific TFM. If you build an app with that TFM, you can set `UseWPF` or `UseWindowsForms` (or both) to `true`, and you can start using types from those UI frameworks. Does that mean Rx should include its support for Windows Forms and WPF with that target? +The Native AOT version did better again. Obviously the absolute sizes are all significantly smaller, but the ratio of Rx without Windows Forms and WPF to Rx with Windows Forms and WPF is about 2.8x here. That's a lot better than ordinary trimming. The absolute increase of 11.2MB is relatively modest, and would be a smaller proportion of the whole in a larger application. -Rx [added .NET Core 3.0 targets](https://github.com/dotnet/reactive/pull/857) in +But it's still not great. And there are lots of scenarios in which Native AOT simply isn't an option. There are a fair few in which trimming can't be used either. So that unwanted 90MB in the self-contained deployment is a real problem in many scenarios. +(In case you're wondering why the self-contained deployment is so large, at 20.8MB, most of that is the `Microsoft.Windows.SDK.NET.dll` library. This gets included as a result of using a Windows-specific TFM, and using some of the WinRT-style APIs that it makes available. That library is where the types such as `MouseCapabilities` my example uses come from.) +So this is why, in the earlier table, I said that for the `net5.0-windows10.0.19401` the answer to the question "Which UI framework should Rx support?" should be "None." But why did I qualify it as "probably?" It's because I think they might not have had a choice: they had already painted themselves into a corner by this time. In order to avoid this, they would have had to have designed Rx 4 differently, and that ship had already sailed. + +In my view, the best solution to this whole problem would have been for all of the UI-frameworks-specific pieces of Rx to remain in separate libraries. Although the simplicity of getting all Rx can offer with a single package reference is appealing, we simply wouldn't have the problem we have today if the framework-specific pieces had remained separate. + +This is easy to say with hindsight of course, particularly since there are now many different options for building client-side UI with .NET. In a world where Avalonia, MAUI, Windows Forms, WPF, and WinUI are all possibilities for a .NET application, the idea that `System.Reactive` should do everything looks obviously unsustainable, in a way that it didn't back in the Rx 4.0 days. ### The workaround -If your application has encountered [the problem](#the-problem), you add this to the `csproj`: +If your application has encountered [the problem](#the-problem), you can add this to the `csproj`: ```xml @@ -264,27 +289,66 @@ If your application has encountered [the problem](#the-problem), you add this to This needs to go just the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. The problem afflicts only executables, not DLLs. -Why not just set [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) back to `false`? That might work in a simple single-project setup, but there are cases involving libraries where it does not. For example, consider this dependency chain: +here's an updated version of the table from the previous section. The final two columns are for the same application as last time, `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. + +| Deployment type | Size without Rx | Size with Rx without workaround | Size with Rx using workaround | +|--|--|--|--| +| Framework-dependent | 20.8MB | 22.5MB | 22.5MB +| Self-contained | 90.8MB | 182MB | 92.5MB | +| Self-contained trimmed | 18.3MB | 65.7MB | 18.3MB | +| Native AOT | 5.9MB | 17.4MB | 6.2MB | -* MyApp (with no direct `System.Reactive` dependency) - * SomeThirdPartyLib - * `System.Reactive` +As you can see, this is much more reasonable. In the first two cases, the output grows by the size of the `System.Reactive.dll` file. In the second two cases, the impact is considerably more modest. Rx makes a barely perceptible impact to the trimmed case. It's slightly more noticeable in Native AOT, but it's adding only about 300KB, roughly a 5% increase in size. + +So that seems pretty effective. + +Why not just set [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/-msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) back to `false`? The short answer is: it doesn't work. But why? The problem is that these really only determine whether the code in your project can use WPF or Windows Forms features. Your project might not use them, but that doesn't change the fact that if any of the components you depend on do have a dependency on the .NET runtime Windows Desktop components, your application will automtically pick up that dependency even if you've not turned on the WPF or Windows Forms features for your own build. -If `SomeThirdPartyLib` has target that's subject to this problem (e.g., it targets `net80-windows10.0.19041`), and if it did not also set `UseWPF` and `UseWindowsForms` to `false`, then that library will have a dependency on `Microsoft.WindowsDesktop.App` ### Community input +We started a [public discussion about this topic](https://github.com/dotnet/reactive/discussions/2038) to garner opinions and suggestions. There was plenty of feedback. The following sections attempt to summarise the themes. + +#### Does size matter? -https://github.com/dotnet/reactive/discussions/2038 +[Anais Betts questioned whether this is really a problem](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7610677): -https://github.com/AvaloniaUI/Avalonia/issues/9549 +> Are we really worried about Disk Space here? In 2023? And that's worth nuking our entire ecosystem over? +I would not dismiss Anais's views lightly. She is a long-standing Rx veteran and continues to be a positive force in the Rx community. In particular, I think we need to pay close attention to the concerns she raises over the damage we might do if we overcomplicate Rx's packaging. -Is size that big a deal? Ani Betts wanted to dismiss this whole topic on the grounds that in this day and age, 50MB is really nothing. But the reality is that people voted with their feet. This has been a big deal for some people, and we need to take it seriously. +However, the evidence compels me to disagree on her first point. My answers to her two questions here are: yes, and no, respectively. +Yes, clearly people are worried about this. We know this because people have complained, and some major projects have abandoned Rx purely because of this problem. + +As for the second point no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed in the next section) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. + +The problem is that the answer to the first question is: yes, people are demonstrably worried enough about disk space in 2023 to drop Rx. That means we do actually have to do something. And for what it's worth I'm not convinced it's all about disk space per se. Network bandwidth is an issue. Not everyone has hundreds of megabits of internet connectivity. I live in the UK, which is a reasonably advanced economy, and I can get 1GB internet to my house, but only because I live in a built up area. A significant proportion of the population can't get even 10% of that speed. And that's in a country with unusually high average population density and a fairly high-tech economy. There are plenty of countries where poor connectivity is even more common than here, with no prospect of change any time soon. So adding 90MB unnecessarily to a download is a problem for significant parts of the world. + +Even in a world of gigabit internet, web page download sizes matter. We added trimming support to Rx 6.0 specifically in response to people wanting to use it in Blazor. Bear in mind that in that scenario removed a mere 1MB from the download size, and yet it was a feature people care about. If 1MB can be a problem, I think it's safe to say that there are still scenarios today where 90MB matters. + +But while I disagree with the premise that 90MB is no big deal, I think Anais offers some extremely important analysis. She worries that a solution to this problem could be: + +> worse than the original problem of "Wow there's too many DLLs". Please don't make a mess of the Rx namespaces again, we have already been through this, Rx is already confusing enough, we know from the first time around that having "too many DLLs" caused immense user confusion and (for some reason) accusations of "bloat" because "there were too many references" - now we are proposing making it even more confusing + +What I take from this is that we should keep things as simple as possible, but no simpler. I think the problem we have today is that the _great unification_ was, with hindsight, an oversimplification. So for me the questions are: + +* what's the simplest approach that's not an oversimplification? +* how can we best get from where we are today to that approach? + +I think that second one is the harder question to answer, and we need to keep in mind this request: "Please don't make a mess of the Rx namespaces again." #### The peculiar faith in the power of breaking changes +A "clean break" is probably a misnomer. + + +#### Are we holding Rx to too high a standard? + +Some people think it's wrong of us to try to maintain high levels of backwards compatibility. I think that's debatable, but if you are going to take that position, then it demands the question: how much backwards compatibility is enough? How much can we break. + +As it happens, I made clear from the start that total compatibility was not a goal: I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. + #### Exploiting radical change as an opportunity From b0dd1bee3870cdc4dca4693c946ef2eb5a9c4ed5 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Thu, 1 Feb 2024 16:54:45 +0000 Subject: [PATCH 07/19] Adding community input to the ADR --- ...0003-windows-tfms-and-desktop-framework.md | 175 +++++++++++++++++- 1 file changed, 165 insertions(+), 10 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 866f0c516..e7178ca88 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -20,6 +20,9 @@ Draft @idg10 ([Ian Griffiths](https://endjin.com/who-we-are/our-people/ian-griffiths/)). +NOTE: I forgot I'd done all this in my repo: https://github.com/idg10/reactive/blob/experiment/separate-platforms/Rx.NET/Documentation/adr/0003-multi-platform-packaging.md + +I was sure I'd remembered doing a load of work to collate this information...I was wondering where it all went! ## Context @@ -315,7 +318,7 @@ We started a [public discussion about this topic](https://github.com/dotnet/reac > Are we really worried about Disk Space here? In 2023? And that's worth nuking our entire ecosystem over? -I would not dismiss Anais's views lightly. She is a long-standing Rx veteran and continues to be a positive force in the Rx community. In particular, I think we need to pay close attention to the concerns she raises over the damage we might do if we overcomplicate Rx's packaging. +I would not dismiss anything Anais has to say about Rx lightly. She is a long-standing Rx veteran and continues to be a positive force in the Rx community. In particular, I think we need to pay close attention to the concerns she raises over the damage we might do if we overcomplicate Rx's packaging. However, the evidence compels me to disagree on her first point. My answers to her two questions here are: yes, and no, respectively. @@ -331,7 +334,15 @@ But while I disagree with the premise that 90MB is no big deal, I think Anais of > worse than the original problem of "Wow there's too many DLLs". Please don't make a mess of the Rx namespaces again, we have already been through this, Rx is already confusing enough, we know from the first time around that having "too many DLLs" caused immense user confusion and (for some reason) accusations of "bloat" because "there were too many references" - now we are proposing making it even more confusing -What I take from this is that we should keep things as simple as possible, but no simpler. I think the problem we have today is that the _great unification_ was, with hindsight, an oversimplification. So for me the questions are: +What I take from this is that we should keep things as simple as possible, but no simpler. + +It's worth noting that [glopesdev offered a slightly contrary opinion](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617): + +> I don't buy the argument of counting nodes in a "dependency tree" as a measure of complexity. Humans often care more about names than they care about numbers. My definition of "sanity" is not just that we have one package, it's that the root of the namespace System.Reactive should not be in a package called System.Reactive.Base. + + + +I think the problem we have today is that the _great unification_ was, with hindsight, an oversimplification. So for me the questions are: * what's the simplest approach that's not an oversimplification? * how can we best get from where we are today to that approach? @@ -340,7 +351,27 @@ I think that second one is the harder question to answer, and we need to keep in #### The peculiar faith in the power of breaking changes -A "clean break" is probably a misnomer. +A feature of the community response that surprised me was that a lot of people seem very open to breaking changes. Some go further and are actively enthusiastic about them: + +* Open to breaking changes + * ["I think it's ok to introduce breaking changes"](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7559009) + * ["I also find these "breaking changes" acceptable"](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7603636) +* Actively encouraging breaking changes + * [" I would like to see a splitting"](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7977086) + * [glopesdev's extensive explanation of why he just wants to change `System.Reactive`](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617) + * [A disruptive change is worth it if it solves the current problem and meets future plans, it also helps to ship new features and bug fixes faster.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-8170628) + * [I really would like the team to consider that a major update is a major update and that breaking changes are acceptable](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-8279135) + +About half of these make no attempt to explain why they think a breaking change will help. Simply breaking everything is no guarantee of fixing the problem! + + +#### 'Clean starts' aren't, due to extension method ambiguity + +David Karnok made a [killer argument against so-called clean starts](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557205): + +>As for the clean start, there would be a lot of clashes due to the use of extension methods, right? For example, if old Rx and new Rx needed to be interoperated, whose Select() will be applied on that IObservable? + +For me this pretty much destroys the idea of a 'clean break' to enable breaking changes. The exact meaning of 'clean break' being presumed here is described in the [Option 2: 'clean break' section](#option-2-clean-break-introducing-new-packages-with-no-relationship-with-current-rx) later in this document. That section also explains why the point David raised here is fatal for that design. #### Are we holding Rx to too high a standard? @@ -349,15 +380,34 @@ Some people think it's wrong of us to try to maintain high levels of backwards c As it happens, I made clear from the start that total compatibility was not a goal: I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. +> [I don't understand why given this we are assuming that System.Reactive needs to somehow hold itself to a higher standard than the core framework itself.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617) + +One reason is that we can't actually impose the split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be on one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to + +But as a library author I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects? Or any of the other library types which are nearly identical across both worlds. + +So the question we need to ask is this: is Rx more like a framework or a common library feature? Are we more like ASP.NET Core, or `System.Linq`? I'd say the fact that we offer a `netstandard2.0` target points very much towards the latter. + + + #### Exploiting radical change as an opportunity If radical change is unavoidable (and that's a big "if") there is a view that it presents an opportunity to achieve things that would normally be impossible. This is essentially the 'never let a good crisis go to waste' mindset, a notable example being Chris Pulman's [proposal for a greater separation of concerns in Rx.NET](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7559424). +Chris is one of the main developers on ReactiveUI, an open source Rx-based project. Anais Betts has also been deeply involved in that project in the past. I find it surprising that the two of them have come to almost precisely opposite conclusions on this: whereas Anais recoils at the complexity of the early days of Rx and advocates strongly for staying with the current single-assembly solution, Chris is effectively arguing for something that looks quite a lot like the earliest versions of Rx, but, if anything, slightly more complex! + +I'd quite like to hear the two of them debate this, because they both have formidable experience with Rx. + #### Windows version numbers +#### Use obsolete + +https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7604157 + + ### Constraints Our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. @@ -456,6 +506,34 @@ We don't want to drop UWP support completely, but we are prepared to contemplate This is problematic for all of the reasons just discussed in the preceding section. However, as far as we know UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. +#### Transitive references to different versions + +consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: + +* UiLibA uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` +* NonUiLibB uses Rx 7, and it does not use any UI-framework-specific Rx features but it does use new functionality in Rx 7 + +Imagine for the sake of argument that Rx 7 adds a `RateLimit` operator to deal with the fact that almost nobody likes Throttle. + +So we now have this situation: + +* UiLibA will require `DispatcherScheduler` to be available through `System.Reactive` v5.0 +* NonUiLibB will require `Observable.RateLimit` to be available through whatever reference it's using to get Rx 7 + +This needs to work. This constraint prevents us from simply removing UI-specific types from `System.Reactive` v7.0. + + +#### Limited tolerance for breaking changes + +While we would prefer to have no breaking changes at all, there are some circumstance where the are acceptable: security considerations can trump backwards compatibility for example. (Not that we currently know of any examples of that in Rx.) In considering a breaking change we need to ask which of the following would apply in cases where a change does break an application: + +1. the app authors need to make some changes to get everything working again +2. we put the app authors in a position where it's not possible for them to get everything working again + +Simply removing UI-specific types in `System.Reactive` v7 would be a type 2 change (because of [the need for things to work when we have indirect dependencies on multiple versions](#transitive-references-to-different-versions)). + +We consider type 2 changes to be unacceptable. We consider all breaking changes undesirable, but would be open to type 1 changes if no better alternative exists. + ### The design options @@ -465,21 +543,66 @@ The following sections describe the design choices that have been considered to The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) -#### Option 2: new main Rx package, demoting `System.Reactive` to a facade +#### Option 2: 'clean break' introducing new packages with no relationship with current Rx -#### Option 3: `System.Reactive` remains the primary package and becomes a facade +**Note**: we have already ruled this out. -We could maintain `System.Reactive` as the face of Rx, but turn it into a facade, with all the real bits being elsewhere. This would give people the option to depend on, say, `System.Reactive.Common` or whatever, to be sure of avoiding any UI dependencies. However, this might not help with transitive dependencies. +This idea was one of those to emerge from some some discussion on an early protoype of [option 3](#option-3-new-main-rx-package-demoting-systemreactive-to-a-facade) at https://github.com/dotnet/reactive/pull/2034 +In this design, we effectively introduce a whole new set of Rx packages that have absolutely no connection with any existing packages. It would in effect be a fork of Rx.NET that just happened to be produced by the maintainers of the current Rx.NET. -#### Option 4: UI-framework specific packages, deprecating their +Applications wanting to use the latest Rx would use the new package. (`System.Rereactive`? `System.ReactiveEx`?) Existing code could carry on using `System.Reactive`. The theory is that we would be free to make breaking changes because code with references to the existing Rx would be entirely unaffected. This would free us to move the UI-specific features back out into separate packages. And it would also present a once-in-a-generation opportunity to fix other things, although that line of thinking risks turning this into the vehicle for everyone's pet feature. +That's all moot, though, because we're not going to do it. -### Other options less seriously considered +This option was elegantly torpedoed by David Karnok. The [earlier section about how 'clean breaks' are a myth](#clean-starts-arent-due-to-extension-method-ambiguity) contains relevant quotes and links. I'll now explain why this is a terrible idea. -### Change everything +The defining feature of this idea is that there is no unification between "before" and "after" versions of Rx. Today, if a single application uses several libraries using several versions of Rx, they all end up using the same version—the build tooling basically picks the highest version number referenced, and everyone gets that. This is called _unification_. The effect of introducing completely new packages that are unrelated to the existing ones is that the build tools will consider these totally different libraries. If your app uses some libraries that use Rx 6 and some that use Rx 7, the build tools won't realise that these are references to different versions of the same thing. It would see a reference to `System.Reactive` v6.0.0, and a reference to `System.Newreactive` (or whatever) v7.0.0, and would just treat them as two separate libraries. + +At that point, you've now got both Rx 6 and Rx 7 in your app. So what happens if you want to use Rx in your application code too? When you write this: -We could do something similar to what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretense of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. +```cs +public IObservable FormatNumbers(IObservable xs) => xs.Select(x => x.ToString("G")); +``` + +Which implementation of `Select` is it going to pick? The problem is that you've now got two definitions of `System.Reactive.Linq.Observable` from two different assemblies. Any file that has a `using System.Reactive.Linq` is going to have both the v6 and the v7 definition of `Observable` in scope. Which one is the compiler supposed to pick? + +That call to `Select` is ambiguous. + +It is technically possible to disambiguate identically-named classes with the rarely-used [`extern alias` feature](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/extern-alias). You can associate an alias with a NuGet reference like this: + +```xml + + + RxV6 + + + + RxV7 + + +``` + +With that in place you can then be specific about which one you're using: + +```cs +extern alias RxV6; +extern alias RxV7; + +using RxV7::System; // for the Subscribe extension method +using RxV7::System.Reactive.Linq; // for Select + +IObservable numbers = RxV6::System.Reactive.Linq.Observable.Range(0, 3); +IObservable strings = numbers.Select(x => x.ToString()); + +strings.Subscribe(Console.WriteLine); +``` + +This uses the Rx v6 implementation of `Observable.Range`, but the two places where this uses extension methods (the call to `Select` and the call to the delegate-based `Subscribe` overload) will use the Rx v6 implementation, because that's what we specified in the `using` directives when importing the relevant namespaces. + +So it's possible to make it work, but we consider this to be confusing and painful. The `extern alias` mechanism is really something you should only have to use if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. And it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. + +The only way to avoid this would be to change not just the NuGet package names but also all the namespaces. That somewhat resembles what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretence of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. You could argue that we've already done this. There's a whole new version of Rx at https://github.com/reaqtive/reaqtor that implements functionality not available in `System.Reactive`. (Most notably the ability to persist a subscription. In the ['reaqtive' implementation of Rx](https://reaqtive.net), operators that accumulate state over time, such as [`Aggregate`](https://introtorx.com/chapters/aggregation#aggregate), can migrate across machines, and be checkpointed, enabling reliable, persistent Rx subscriptions to run over the long term, potentially even for years.) The NuGet package names and the namespaces are completely different. There's no attempt to create any continuity here. @@ -487,6 +610,38 @@ An upshot of this is that there is no straightforward way to migrate from `Syste Our view is that we don't want three versions of Rx. The split between `System.Reactive` and [reaqtive Rx](https://reaqtive.net) was essentially a _fait acompli_ by the time the latter was open sourced. And the use cases in which the latter's distinctive features are helpful are sufficiently specialized that in most cases it probably wouldn't make sense to try to migrate from one to the other. But to create yet another public API surface area for Rx in .NET would cause confusion. We don't think it offers enough benefit to offset that. +So for all these reasons, we are rejecting this design option. + +#### Option 3: new main Rx package, demoting `System.Reactive` to a facade + +Discussed at https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557014 with a prototype on https://github.com/dotnet/reactive/pull/2034 + + + +#### Option 4: `System.Reactive` remains the primary package and becomes a facade + +We could maintain `System.Reactive` as the face of Rx, but turn it into a facade, with all the real bits being elsewhere. This would give people the option to depend on, say, `System.Reactive.Common` or whatever, to be sure of avoiding any UI dependencies. However, this might not help with transitive dependencies. + + +#### Option 5: gut the UI-specific types so they are hollowed out shells + +David Karnok [suggested we turn `DispatcherScheduler` into a delegator and move the real implementation elsewhere](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557205). + + +#### Option 6: UI-framework specific packages, deprecating their + + + +#### Um + +https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7628930 + + +### Other options less seriously considered + +### Change everything + + Another idea: could we introduce a later Windows-specific TFM, so that use of windows10.0.19041 becomes a sort of dead end? From 2ea9e713672c95406a33488dd40b945e3862d2d0 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Fri, 2 Feb 2024 08:28:25 +0000 Subject: [PATCH 08/19] Integrate detajls from earlier write up --- ...0003-windows-tfms-and-desktop-framework.md | 163 +++++++++++++++++- 1 file changed, 157 insertions(+), 6 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index e7178ca88..022dd97fe 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -1,12 +1,27 @@ # Windows TFMs and Desktop Framework Dependencies -When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add tens of megabytes to the deployable size of applications. It has caused some projects to abandon Rx entirely. +When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add many tens of megabytes to the deployable size of applications. + +For example, this table shows the output sizes for a simple console app targetting `net8.0-windows10.0.19041` with various deployment models. The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. **Note** this problem only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later. + +| Deployment type | Size without Rx | Size with Rx | +|--|--|--| +| Framework-dependent | 20.8MB | 22.5MB | +| Self-contained | 90.8MB | 182MB | +| Self-contained trimmed | 18.3MB | 65.7MB | +| Native AOT | 5.9MB | 17.4MB | + +This issue has caused some projects to abandon Rx entirely. For example, after [Avalonia ran into this problem](https://github.com/AvaloniaUI/Avalonia/issues/9549), they [ removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. +Recently, a [workaround](#the-workaround) has been discovered, which reduces the sizes to 22.5MB, 92.5MB, 18.3MB, and 6.2MB respectively. (So the impact of adding Rx is reduced to 1.6MB, 1.6MB, unmeasureably small, and 300KB respectively for the four deployment models above.) + The view of the Rx .NET maintainers is that projects using Rx should not be forced into this situation. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. -Since this topic was first raised, we have discovered that there is a [workaround](#the-workaround) to the problem. Back when [endjin](https://endjin.com) took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed. It seems that Rx 6.0 can be used in a way that doesn't encounter these problems, so we now think that a less radical, more gradual longer-term plan is a better bet. We can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. The process of deprecation could begin now, but it would likely be many years before we reach the ends state. +The discovery of a [workaround](#the-workaround) came fairly late in the day, some time after various projects had decided to stop using Rx.NET. That is important context for understanding earlier discussion of this topic. Back when [endjin](https://endjin.com) took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed. It seems that Rx 6.0 can be used in a way that doesn't encounter these problems, so we now think that a less radical, more gradual longer-term plan is a better bet. We can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. + +The process of deprecation could begin now, but it would likely be many years before we reach the ends state. This document explains the root causes of the problem, the current [workaround](#the-workaround), the eventual desired state of Rx .NET, and the path that will get us there. @@ -87,7 +102,7 @@ Each of these subfolders of the NuGet pacakges `lib` folder contains a version o #### Plug-in problems -This fragmentation caused a problem with plug-in systems. People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it. Any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. +This fragmentation caused [a problem with plug-in systems](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it. Any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. @@ -139,11 +154,11 @@ It's worth noting at this point that the problem I've just described doesn't nee #endif ``` -By time time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. +By this time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. Again, it's worth thinking briefly about .NET Core/modern .NET at this point to see how things are different there. This newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) -Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. +Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.2000.0 versions do not). As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems. But if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. @@ -157,6 +172,8 @@ The basic problem here is that when building any single deployable target (eithe Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET 4.0 would be suboptimal on .NET 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it either. The oldest version of .NET Framework that it made sense to target at this point was .NET 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targetting different versions of .NET Framework. +This was a critical change in the landscape. + Since there was now just a single .NET Framework target (`net46`), the original plug-in problems could no longer occur. (The only reason they happened in the first place was that Rx used to offer different assemblies targetting different versions of .NET Framework.) Furthermore, collapsing Rx down to a single assembly, `System.Reactive`, solved all of the newer problems created by the Rx 3.1 era attempt to solve the plug-in problems by playing games with .NET assembly version numbers. This simplification was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. @@ -508,6 +525,7 @@ This is problematic for all of the reasons just discussed in the preceding secti #### Transitive references to different versions + consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: * UiLibA uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` @@ -522,6 +540,44 @@ So we now have this situation: This needs to work. This constraint prevents us from simply removing UI-specific types from `System.Reactive` v7.0. +#### Minimum acceptable path for breaking changes + +When a developer using Rx.NET upgrades to a newer version, what is the smallest version increment for which we are prepared to consider breakage to be acceptable? What's the smallest length of time between an Rx.NET release coming out, and a subsequent release making a breaking change? What are our minimum requirements around giving fair notice with `[Obsolete]` annotations? + +Let's start with some upper and lower bounds: + +* If a project upgrades from Rx.NET 1.0 to 7.0, it is probably reasonable for some things to break +* Components with a dependency on Rx 5.0 or 6.0 running in an application that upgrades that to Rx 7.0 must not be broken by this upgrade + +That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. + +There's another dimension we need to take into account: + +* Applications choosing a higher Rx.NET version +* NuGet packages choosing a higher Rx.NET version + +That second category has the potential to be more insidious, because it can mean that an application gets a 'surprise' upgrade. As an application developer you might upgrade a reference to some 3rd party library without realising that its latest version has a dependency on the latest version of Rx.NET. + +This is particularly troublesome in large complex projects that have been in development for many years. These often end up with large numbers of dependencies of widely varying age. A successful project that is, say, 5 years old may well have taken a dependency early on on some component that used Rx 4.0. And perhaps that dependency has worked perfectly well in all that time, with no need to be upgraded. Now suppose a dependency is added to a new component, and this new component wants Rx.NET 7. + +Is it acceptable for the code to break because that old component is relying on a very old version of Rx.NET? Rx 4.0 would be 4 major versions behind the latest version in this scenario, and it's a version that shipped over half a decade ago. + +Some would argue that this makes it acceptable for the application to break in this scenario. However, there are some reasons to disagree with this: + +* We have not yet made any clear indication (e.g., marking things as [Obsolete]) of plans to remove features from Rx.NET, so this would effectively be a bolt from the blue +* This might create a situation where the application developers are forced to abandon one or other of the components they want to use +* Although the current .NET runtime introduced an exciting world where 3 years counts as "long term support" the .NET Framework still sits in the Windows world in which support typically lasts for 10 years, so it's not entirely unreasonable to expect compatibility with a 5 year old release + +Given that the current LTS policy is for .NET to last 3 years, I think the absolute minimum requirement is that if we want to make a breaking change, we can't do it until at least 3 years after marking the relevant API surface area as `[Obsolete]`. But I also think we need to take into account the times at which people are likely to discover that we've added such annotations. .NET 8.0 came out in November 2023, and many applications will already have upgraded to that and may be planning to keep things that way for at least 2 years. So if we were to release a new version of Rx.NET in March 2024, it's possible that developers for some application might not notice we've done that until 2026 when they start getting ready to upgrade to .NET 10.0. So if we progressed from `[Obsolete]` to removing an API entirely three years from now, that looks like just 1 year of notice to those developers. + +This suggests that for obsolescence we announce today, we can't really implement it any earlier than 2029, 5 years from now. + +I think we might need to be even more conservative than that, because when Rx is used indirectly through transitive references, application developers might be completely unaware that they are dependent on it, and might not see any warnings at all in their code. The first indication they get might be a build failure when they upgrade a component. + +This suggests we need to consider how long it will take for the .NET ecosystem to respond to our changes. It might take three years for a component author to deal with a forthcoming breaking change in Rx, but it might then take another three years for applications using that component to get around to updating. + +This suggest that a 6 year window might be reasonable. Anything we mark as obsolete today in 2024 can't be removed entirely until 2030. + #### Limited tolerance for breaking changes @@ -535,13 +591,108 @@ Simply removing UI-specific types in `System.Reactive` v7 would be a type 2 chan We consider type 2 changes to be unacceptable. We consider all breaking changes undesirable, but would be open to type 1 changes if no better alternative exists. +#### Compatibility with plug-in systems need to work + +We must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. + +It's worth re-iterating that 1 is a non-issue on .NET 6.0+ hosts. Applications hosting plug-ins on those runtimes will use AssemblyLoadContext, enabling each plug-in to make its own decisions about which components it uses. If plug-ins make conflicting decisions, it doesn't matter, because AssemblyLoadContext is designed to allow that. + +So we only need to consider .NET Framework. But there has been one critical change since this problem originally occurred: Rx.NET now defines exactly one .NET Framework target. The original problem occurred only because we had both `net40` and `net45` targets. Now, we have only `net472`. + +However, in design options where we consider splitting Rx.NET across multiple packages (e.g., where we move the UI-framework-specific pieces out into separate UI-framework-specific packages) we also need to be careful that we don't recreate the problems that occurred in Rx 3.0. + +Rx 3.0 solved 1 by using different 'build numbers' (the 3rd part of the assembly version) for different targets. For example, DLLs targeting net45 would have a version of 3.0.1000.0; net451 DLLs used 3.0.2000.0; net46 DLLs used 3.0.3000.0. These assembly version numbers remained the same across multiple NuGet releases, so even in the release that NuGet labels as 3.1.1, the assembly version numbers were all 3.0.xxxx.0. So if a plug-in built for .NET 4.0 used Rx 3.1.1, and was loaded into the same process as another plug-in also using Rx 3.1.1 but built for .NET 4.5, they would be able to load the two different System.Reactive.Linq.dll assemblies because those assemblies had different identities by virtue of having different assembly version numbers. Unfortunately, while this solved problem 1, it caused problem 2: it was relatively easy to confuse NuGet into thinking that you had conflicting dependencies. + +#### Coherent versions across multiple components + +One of the issues that occured with the [Rx 3 era attempt to fix this problem](https://github.com/dotnet/reactive/pull/212) was that people sometimes found that [referencing two Rx.NET components with the same apparent version number could produce version conflicts](https://github.com/dotnet/reactive/issues/305). + +Rx 4.0 solved this (and Rx 6.0 still uses the same approach) by collapsing everything down to a single NuGet package, System.Reactive. To access UI-framework-specific functionality, you no long needed to add a reference to a UI-framework-specific package such as System.Reactive.Windows.Forms. From a developer's perspective, the functionality was right there in System.Reactive. The exact API surface area you saw was determined by your target platform. If you built a UWP application, you would get the lib\uap10.0.16299\System.Reactive.dll which had the UWP-specific dispatcher support built in. If your application was built for net6.0-windows10.0.19041 (or a .NET 6.0 TFM specifying a later Windows SDK) you would get lib\net6.0-windows10.0.19041\System.Reactive.dll which has the Windows Forms and WPF dispatcher and related types build in. If your target was net472 or a later .NET Framework you would get lib\net472\System.Reactive.dll, which also included the Windows Forms and WPF dispatcher support (but built for .NET Framework, instead of .NET 6.0). And if you weren't using any client-side UI framework, then the behaviour depended on whether you were using .NET Framework, or .NET 6.0+. With .NET Framework you'd get the net472 DLL, which would include the WPF and Windows Forms support, but it didn't really matter because those frameworks were present as part of any .NET Framework installation. And if you target .NET 6.0 you'll get the lib\net6.0\System.Reactive.dll, which has no UI framework support, but that's fine because any .NET 6.0+ TFM that doesn't mention 'windows' doesn't offer either WPF or Windows Forms. + +It's worth noting that this problem goes away if you use the modern project system. The problem described in [#305](https://github.com/dotnet/reactive/issues/305) occurs only with the old packages.config system. If you try to reproduce the problem in a project that uses the .NET SDK project system introduced back in MSBuild 15 (Visual Studio 2017), the problem won't occur. Visual Studio 2017 is the oldest version of Visual Studio with mainstream support. + +Although some projects continue to use `packages.config` today, proper package references have been available for a long time now, even for projects that are still stuck on the older style of project file. + +It is explicitly a non-goal to support `packages.config`. Our position is that projects stuck in that world can continue to use Rx 6.0. + + +### Other contextual changes to consider + +A possible fly in the ointment is the netstandard2.0 component, because that could in principle run in a .NET Framework process. We could still end up with that loading first, blocking any attempt to load the net472 version some time later. However, in the plug-ins scenario above, that shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the net472 version over the netstandard2.0 one. (If they target an older version such as net462, then perhaps they would pick netstandard2.0 instead. But then the current Rx 6.0 design fails in that scenario too. So unwinding the earlier design decisions won't make things any worse than they already are.) + +Another consideration is that modern NuGet tooling is better than it was in 2016 when the current design was established. Alternative solutions might be possible now that would not have worked when Rx 4.0 was introduced. + +When it comes to .NET 6.0 and later, these problems should a non-issue because better plug-in architectures exist thanks to AssemblyLoadContext. + +### Types in `System.Reactive` that are partially or completely UI-framework-specific + +Unwinding the consolidation that happened in V4 comes with a challenge: `System.Reactive` now has a slightly different API surface area on different platforms. There are some types available only on specific targets. There is one type that is available on all targets but has a slightly different public API on one target. And there are some types with the same API surface area on all targets but with differences in behavior. This section describes the afflicted API surface area. + +Types available only in specific targets: + +* `net472` + * `RemotingObservable` +* `net6.0-windows` and `net472` + * `DispatcherScheduler` + * `DispatcherObservable` +* `net6.0-windows*` and UWP + * `CoreDispatcherScheduler` + * `CoreDispatcherObservable` + * Windows Foundation Event Handler support: + * `IEventPatternSource` + * `EventPatternSource` + * `WindowsObservable` (see also async operations) + * Adaptation between `IObservable` and WinRT's async actions/operations/progress: + * `AsyncInfoObservableExtensions` + * `AsyncInfoObservable` + * `WindowsObservable` (see also Windows Foundation Event Handler support) + +Types that are in all targets, but which contain UI-frameworks specific public members in relevant targets: + +* UWP + * `ThreadPoolScheduler` has different implementation based on UWP thread pool and provides support for `WorkItemPriority` (a UWPism) + +Behavior that is different across platforms: + +* UWP + * `Scheduler` periodic scheduling has workaround for WinRT not supporting <1ms resolution +* `net6.0-windows*` and UWP + * `IHostLifecycleNotifications` service available from enlightenment provider + * Periodic scheduling will be aware of windows suspend/resume events only if using `System.Reactive` for these targets + + ### The design options The following sections describe the design choices that have been considered to date. #### Option 1: change nothing -The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) +The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) makes this a more attractive option than it had seemed when we first started looking at this problem. + +Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is just to continue doing things in the same way. This is not a good solution. Back when Rx 5.0 was the current version, some people seemed to think that the changes we adopted in Rx 6.0 would be sufficient to solve the problems described in this document. But as will be explained, it doesn't. + +`System.Reactive` 5.0 targeted `netstandard2.0`, `net472`, `netcoreapp3.1`, `net5.0`, `net5.0-windows10.0.19041`, and `uap10.0.16299`. The idea with this design option was to target `netstandard2.0`, `net472`, `net6.0`, `net6.0-windows10.0.19041`, and `uap10.0.16299`. (In other words, drop .NET Core 3.1 and .,NET 5.0, both of which went out of support in 2022, and effectively upgrade the .NET 5.0 target to .NET 6.0.) This is in fact exactly what we did for Rx 6.0, but despite what some people seemed to believe, this was never going to solve the problems described above. + +Let's look at how this gets on with the three challenges: + +**1: Host applications with a plug-in model getting into a state where plug-ins disagree about which `System.Reactive.dll` is loaded** + +Since the plug-in issues are only relevant to .NET Framework, and this doesn't change the .NET Framework packaging in any way, this solves the problem in the same way that Rx 4.0 and 5.0 do. + +**2: Incompatible mixes of version numbers of Rx components** + +Rx 4.0 introduced the unified packaging to solve this problem, and this option retains it, so it will solve the problem in the same way. + +**3. Applications getting WPF and Windows Forms dependencies even though they use neither of these frameworks** + +This design option does not solve this problem. I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that, and I think separating out those parts is the only way to achieve this. + +This design option also doesn't have a good answer for how we provide UI-framework-specific support for other frameworks. (E.g., how would we offer a `DispatcherScheduler` for MAUI's `IScheduler`?) + +So in short, I (@idg10) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from problem 3. + +The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it closely resembles the existing design. + #### Option 2: 'clean break' introducing new packages with no relationship with current Rx From f1b500400ee7c22a61c9b70b63acf2770899b307 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Fri, 2 Feb 2024 16:26:30 +0000 Subject: [PATCH 09/19] ADR feature complete up as far as "Transitive references to different versions" --- ...0003-windows-tfms-and-desktop-framework.md | 219 ++++++++++++------ 1 file changed, 143 insertions(+), 76 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 022dd97fe..3f4a2e9da 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -2,16 +2,16 @@ When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add many tens of megabytes to the deployable size of applications. -For example, this table shows the output sizes for a simple console app targetting `net8.0-windows10.0.19041` with various deployment models. The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. **Note** this problem only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later. +For example, this table shows the output sizes for a simple console app targeting `net8.0-windows10.0.19041` with various deployment models. The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. **Note** this problem only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later. | Deployment type | Size without Rx | Size with Rx | |--|--|--| | Framework-dependent | 20.8MB | 22.5MB | -| Self-contained | 90.8MB | 182MB | +| Self-contained | 90.8MB | **182MB** | | Self-contained trimmed | 18.3MB | 65.7MB | | Native AOT | 5.9MB | 17.4MB | -This issue has caused some projects to abandon Rx entirely. +The worst case, self-contained deployment, is also a widely used case for applications that need to use a `-windows` TFM. It roughly doubles the size of this application, adding over 90MB! This issue has caused some projects to abandon Rx entirely. For example, after [Avalonia ran into this problem](https://github.com/AvaloniaUI/Avalonia/issues/9549), they [ removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. @@ -21,9 +21,9 @@ The view of the Rx .NET maintainers is that projects using Rx should not be forc The discovery of a [workaround](#the-workaround) came fairly late in the day, some time after various projects had decided to stop using Rx.NET. That is important context for understanding earlier discussion of this topic. Back when [endjin](https://endjin.com) took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed. It seems that Rx 6.0 can be used in a way that doesn't encounter these problems, so we now think that a less radical, more gradual longer-term plan is a better bet. We can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. -The process of deprecation could begin now, but it would likely be many years before we reach the ends state. +The process of deprecation could begin now, but it would likely be many years before we reach the end state. -This document explains the root causes of the problem, the current [workaround](#the-workaround), the eventual desired state of Rx .NET, and the path that will get us there. +This document explains the [root causes of the problem](#the-road-to-the-current-problem), the current [workaround](#the-workaround), [the community feedback we've received](#community-input), the [constraints that any solution will have to satisfy](#constraints), and [the eventual desired state of Rx .NET, and the path that will get us there](#decision). ## Status @@ -33,15 +33,12 @@ Draft ## Authors -@idg10 ([Ian Griffiths](https://endjin.com/who-we-are/our-people/ian-griffiths/)). +[@idg10](https://github.com/idg10) ([Ian Griffiths](https://endjin.com/who-we-are/our-people/ian-griffiths/)). -NOTE: I forgot I'd done all this in my repo: https://github.com/idg10/reactive/blob/experiment/separate-platforms/Rx.NET/Documentation/adr/0003-multi-platform-packaging.md - -I was sure I'd remembered doing a load of work to collate this information...I was wondering where it all went! ## Context -To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem). The proposed [workaround](#the-workaround) needs to be understood in detail. We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. +To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem), and the [constraints that any solution must fulfil](#constraints). The proposed [workaround](#the-workaround) needs to be understood in detail. We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. The following sections address all of this before moving onto a [decision](#decision). @@ -49,16 +46,18 @@ The following sections address all of this before moving onto a [decision](#deci The basic problem is described at the start of this document, but we can characterise it more precisely: -> An application that references `System.Reactive` (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). The [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true`. +> An application that references the [`System.Reactive` NuGet package](https://www.nuget.org/packages/System.Reactive) (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` or later will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). +> +> This occurs because in projects that have a direct reference to the package, the [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true`. If the project is an application, that will cause the dependency on the Desktop Runtime. If the project builds a NuGet package, it will cause the package to be built in a way that indicates that the Desktop Runtime is required, which is how indirect references to `System.Reactive` also cause this issue. > -> This causes a problem for self-contained deployment (and, by extension, Native AOT) because it means those deployments end up including complete copies of those frameworks. This can add many tens of megabytes to the application in its final deployable form. This is especially frustrating for applications that don't use either WPF or Windows Forms. +> An unwanted dependency on the .NET Windows Desktop Runtime causes a problem for self-contained deployment (and, by extension, Native AOT) because it means those deployments end up including complete copies of the Windows Forms and WPF frameworks. This can add many tens of megabytes to the application in its final deployable form. This is especially frustrating for applications that don't use either WPF or Windows Forms. That "or transitively" in the first parenthetical is easily overlooked, but is very important. Some developers have found themselves encountering this problem not because their applications use `System.Reactive` directly, but because they are using some library that depends on it. Many simple and initially plausible-looking solutions proposed to the problem this ADR addresses founder in cases where an application acquires a dependency to Rx.NET transitively, especially when it does so through multiple different references, and to different versions. ### The road to the current problem -This problem arose from a series of changes that were intended to solve other problems. We need to ensure that we don't reintroduce any of these older problems, so it's important to have a good understanding of the following factors that led to the current design: +This problem arose from a series of changes made about half a decade ago that were intended to solve other problems. We need to ensure that we don't reintroduce any of these older problems, so it's important to have a good understanding of the following factors that led to the current design: 1. the long history of confusion in Rx's package structure 2. the subtle problems that could occur when plug-ins use Rx @@ -67,42 +66,44 @@ This problem arose from a series of changes that were intended to solve other pr #### Rx's history of confusing packaging -The first public previews of Rx appeared back in 2009 before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary files onto target machines as part of your application's installation or deployment process. By the time [the first supported Rx release shipped in June 2011](https://web.archive.org/web/20110810091849/http://www.microsoft.com/download/en/details.aspx?id=26649), NuGet did exist, but it was early days, so for quite a while Rx had [two official distribution channels: NuGet and an installable SDK](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5). +The first public previews of Rx appeared back in 2009 before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary redistributable files onto target machines as part of your application's installation or deployment process. By the time [the first supported Rx release shipped in June 2011](https://web.archive.org/web/20110810091849/http://www.microsoft.com/download/en/details.aspx?id=26649), NuGet did exist, but it was early days, so for quite a while Rx had [two official distribution channels: NuGet and an installable SDK](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5). There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: * The scheduler support was specialized to work as well as possible on each distinct target * Each platform had a different UI framework (or frameworks) available, so Rx's UI framework integration was different for each target -Some of the differences in the first category were implementation details behind an API common to all versions, but there were some public API differences too. The second category was all about differences in the public API, although at this point in Rx's history, the UI-framework-specific code was in separate assemblies. But there was a common core to Rx's public API that was the same across all platforms. +Some of the differences in the first category were implementation details behind an API common to all versions, but there were some public API differences too. (You can still see an echo of this today. Rx's `ThreadPoolScheduler` class is available on all platforms, but on UWP, which is effectively the successor to the old Windows 8 API, this class has two extra properties, `Priority` and `Options`, for controlling UWP-specific thread pool behaviour.) The second category necessarily involves differences in the public API, although at this point in Rx's history, all of its UI-framework-specific code was in separate assemblies, so those differences were isolated. There was a common core to Rx's public API that was the same across all platforms. This meant that it would be possible, in principle, to write a library that depended on Rx, and which could be used on all the same platforms that Rx supported. However, it wasn't entirely straightforward to do this back in 2011. -This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. Understanding which component your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and presented a barrier to adoption for new users. +This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. -An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea behind this was that this would be a stable component that didn't need frequent releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. The defeated the entire purpose of having a separate component for the core interfaces. +An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea behind this was that this would be a stable component that didn't need frequent releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. This defeated the entire purpose of having a separate component for the core interfaces. (In fact things were a little weirder because some of the versions of .NET supported by Rx 1.0 defined the core `IObservable` and `IObserver` interfaces in the runtime class libraries but some did not. These interfaces were not present in .NET 3.5, for example, which Rx 1.0 supported. So Rx had to bring its own definition of these for some platforms. You might expect these to live in `System.Reactive.Interfaces` but they did not, because Microsoft wanted that package to be the same on all platforms. So on platforms where `IObversable/er` were not built in, there was yet another DLL in the mix, further adding to the confusion around exactly what assemblies you needed to ship with your app if you wanted to use Rx.) -The other splits were also a bit hard to comprehend—it's not obvious why the LINQ parts of Rx are in their own package. In practice, anyone using Rx is going to use its LINQ features. +The other splits were also hard to comprehend—it's not obvious why the LINQ parts of Rx are in their own package. In practice, anyone using Rx is going to use its LINQ features. The role of the 'providers' component that existed in these early days is also not obvious. The 'platform services' part is arguably slightly easier to understand because .NET developers at this time were generally aware that there were lots of flavours of .NET each with slightly different characteristics. Even then, understanding how that worked in practice was tricky, and this was just another bit of complexity that could make Rx harder to use. -The NuGet distribution of Rx introduced a simplifying concept in v2.2: Rx was still fragmented across multiple components at this point, but the simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. For example, a single reference to `Rx-Main` v2.2.0 would give you everything you needed to use Rx. There were additional metapackages appropriate for using specific UI frameworks with Rx. +In summary, you couldn't simply add a reference and start using Rx. Understanding which components your applications or libraries should reference in order to use Rx, and understanding which particular DLLs needed to be deployed was not easy, and presented a barrier to adoption for new users. + +The NuGet distribution of Rx introduced a simplifying concept in v2.2: Rx was still fragmented across multiple components at this point, but the simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. For example, a single reference to [`Rx-Main` v2.2.0](https://www.nuget.org/packages/Rx-Main/2.2.0) would give you everything you needed to use Rx. There were additional metapackages appropriate for using specific UI frameworks with Rx. For the first time, now you could just add a single reference and immediately start using Rx. -Because Rx has always supported many different runtimes, there were several different builds of each component. For quite a long time, there were different copies of Rx for different versions of .NET Framework. In Rx 2.2.0, there was one targetting .NET Framework 4.0, and another targetting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out which one to use based on the runtime you target. +Because Rx has always supported many different runtimes, each Rx.NET NuGet package contained several different builds of its component. For quite a long time, there were different copies of Rx for different versions of .NET Framework. In Rx 2.2.0, there was one targeting .NET Framework 4.0, and another targeting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out at build time which one to use based on the runtime your application targets. -So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (The Rx 2.0 [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively.) And then each of those packages contained multiple versions of what was, conceptually speaking, the same assembly (but with various technical differences due to differences between the target platforms). For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders +So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (Rx 2.2's [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively.) And then each of those packages contained multiple versions of what was, conceptually speaking, the same assembly (but with various technical differences due to differences between the target platforms). For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders -![](./images/0003-Rx-Core-2.2.0-contents.png) +![A folder view showing a 'lib' folder, with 8 subfolders: net40, net45, portable-net40+sl5+win8+wp8, portalble-windows8+net45+wp8, sl5, windows8, windowsphone71, and windowsphone8](./images/0003-Rx-Core-2.2.0-contents.png) -It's the same story for [`Rx-Interfaces`](https://nuget.info/packages/Rx-Interfaces/2.2.0) and [`Rx-Linq`](https://nuget.info/packages/Rx-Linq/2.2.0). And it's almost the same for [`Rx-PlatformServices`](https://nuget.info/packages/Rx-PlatformServices/2.2.0) except for some reason that doesn't have the `portable-windows8+net45+wp8`. +It's the same story for [`Rx-Interfaces`](https://nuget.info/packages/Rx-Interfaces/2.2.0) and [`Rx-Linq`](https://nuget.info/packages/Rx-Linq/2.2.0). And it's almost the same for [`Rx-PlatformServices`](https://nuget.info/packages/Rx-PlatformServices/2.2.0) except for some reason that doesn't have a `portable-windows8+net45+wp8` folder. -Each of these subfolders of the NuGet pacakges `lib` folder contains a version of the assembly for that package. So `Rx-Core` contains 8 copies of `System.Reactive.Core.dll`, `Rx-Interfaces` contains 8 copies of `System.Reactive.Interfaces.dll`, `Rx-Linq` contains 8 copies of `System.Reactive.Linq.dll`, and `Rx-Core` contains 7 copies of `System.Reactive.PlatformServices.dll`. So conceptually we've got 4 assemblies here, but because of all the different builds, there are actually 31 files! +Each of these subfolders of each NuGet package's `lib` folder contains a version of the assembly for that package. So `Rx-Core` contains 8 copies of `System.Reactive.Core.dll`, `Rx-Interfaces` contains 8 copies of `System.Reactive.Interfaces.dll`, `Rx-Linq` contains 8 copies of `System.Reactive.Linq.dll`, and `Rx-Core` contains 7 copies of `System.Reactive.PlatformServices.dll`. So conceptually we've got 4 assemblies here, but because of all the different builds, there are actually 31 files! #### Plug-in problems -This fragmentation caused [a problem with plug-in systems](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it. Any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. +This fragmentation caused [a problem with plug-in systems](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it, but any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. @@ -124,15 +125,19 @@ Here's what would happen. Let's say a we have two plug-ins, `PlugInOneBuiltFor40 * `System.Reactive.Linq.dll` v2.2.0 (`net45` build) * `System.Reactive.PlatformServices.dll` v2.2.0 (`net45` build) -The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one build for .NET 4.5. Crucially _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are in this case.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. +(Visual Studio uses a more complex folder layout in reality, but that's not significant. _Any_ plug-in host will have the same issue.) -Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, that will be the `net40` version, but the assembly resolver doesn't know that these are different. It assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` build but not the `net40` build. +The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one built for .NET 4.5. Crucially, _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are the same in this case.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. + +Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, this already-loaded copy will be the `net40` version, but the assembly resolver doesn't know that it's different from what `PlugInTwoBuildFor45` wants. + +The .NET Framework assembly resolver assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` build but not the `net40` build. This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that when a .NET 4.5 application uses a .NET 4.0 component, that component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45` would end up using the `net45` build of the Rx components. And that would work just fine.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the whole point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. It's worth noting at this point that the problem I've just described doesn't need to affect applications using .NET (as opposed to .NET Framework). Back when the thing we now call ".NET" was still called .NET Core, .NET Core added the [`AssemblyLoadContext` type](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext) which makes it possible for different plug-ins each to load their own copies of assemblies, even when they have exactly the same full name as assemblies loaded by other plug-ins. But that feature didn't exist back in the Rx 2.0 or 3.0 days (and still doesn't exist in .NET Framework even today). -[Rx 3.1](https://github.com/dotnet/reactive/releases/tag/v3.1.0) attempted to solve this by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have expected this would use the fourth part that .NET assembly versions have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: +[Rx 3.1](https://github.com/dotnet/reactive/releases/tag/v3.1.0) attempted to solve the plug-in problem by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have expected this would use the fourth part that .NET assembly versions have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: ```cs #if NETSTANDARD1_0 || WP8 @@ -154,31 +159,39 @@ It's worth noting at this point that the problem I've just described doesn't nee #endif ``` -By this time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated vesion of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. +By this time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated version of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. Again, it's worth thinking briefly about .NET Core/modern .NET at this point to see how things are different there. This newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) -Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.2000.0 versions do not). +Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.1000.0 versions do not). -As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems. But if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. +As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems, but if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. For example, if your application targetted .NET 4.6.2, and you were using two libraries that both depend on Rx 3.1.1, but one of those libraries offers only a `net45` target and the other offers only a `net461` target, they now disagree on the version of Rx they want. The first wants Rx components with version numbers of `3.0.1000.0`, while the second wants components with version numbers of `3.0.4000.0`. This could result in assembly version conflict reports when building the application. You might be able to solve this with assembly binding redirects, and you might even be able to get the build tools to generate those for you. But there were scenarios where the tooling couldn't work out what to do, and developers were left trying to understand all the history described to date in order to work out how to unpick the mess. And this also relies on the same "we can resolve it all when we build the application" assumption that is undermined in plug-in scenarios, so this could _still_ cause problems for plug-ins! -The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different runtimes. These might be a mutually compatible combination (e.g., if you use components targetting `net40`, `net45`, and `net46`, they can all run happily on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. +The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different runtimes. These might be a mutually compatible combination (e.g., if you use components targeting `net40`, `net45`, and `net46`, they can all run happily on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. #### Rx 4.0's great unification -[Rx 4.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.0.0) tried a different approach: have a single Rx package, `System.Reactive`. This was a single package with no dependencies. This removed all of the confusion that had been caused by Rx previously being split into four pieces. +[Rx 4.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.0.0) tried a different approach: have just one Rx package, `System.Reactive`. This was a single package with no dependencies. This removed all of the confusion that had been caused by Rx previously being split into four pieces. -Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET 4.0 would be suboptimal on .NET 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it either. The oldest version of .NET Framework that it made sense to target at this point was .NET 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targetting different versions of .NET Framework. +Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET Framework 4.0 would be suboptimal on .NET Framework 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it. In fact, the oldest version of .NET Framework that it made sense to target at this point was 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targeting different versions of .NET Framework. -This was a critical change in the landscape. +This was a critical change in the landscape, because it created an opportunity for Rx.NET. -Since there was now just a single .NET Framework target (`net46`), the original plug-in problems could no longer occur. (The only reason they happened in the first place was that Rx used to offer different assemblies targetting different versions of .NET Framework.) Furthermore, collapsing Rx down to a single assembly, `System.Reactive`, solved all of the newer problems created by the Rx 3.1 era attempt to solve the plug-in problems by playing games with .NET assembly version numbers. +Since there was now just a single .NET Framework target (`net46`), the original plug-in problems could no longer occur. (The only reason they happened in the first place was that Rx used to offer different assemblies targeting different versions of .NET Framework.) Furthermore, collapsing Rx down to a single assembly, `System.Reactive`, solved all of the newer problems created by the Rx 3.1 era attempt to solve the plug-in problems by playing games with .NET assembly version numbers. This simplification was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. -Although it now targets just one version of .NET Framework, `System.Reactive` is still a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: `net472`, `net6.0`, `net6.0-windows10.0.19041`, `netstandard2.0`, and `uap10.0.18362`. Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. Consequently, the `System.Reactive.dll` in the package's `netstandard2.0` folder does not include the `ControlScheduler`. +Although it now targets just one version of .NET Framework, `System.Reactive` is still a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: + +* `net472` (.NET Framework) +* `net6.0` +* `net6.0-windows10.0.19041` +* `netstandard2.0` +* `uap10.0.18362` (UWP) + +Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. Consequently, the `System.Reactive.dll` in the package's `netstandard2.0` folder does not include the `ControlScheduler`. This illustrates that with this _great unification_, when you add a reference to `System.Reactive`, you get everything NuGet has to offer on whatever platform your application targets. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. @@ -195,9 +208,9 @@ But .NET Core 3.0 ended that simple relationship. Consider this table: | .NET Framework (`net462`, `net48` etc.) | Windows Forms and WPF | | UWP (`uap10.0` etc.) | UWP | | .NET Core before 3.0 (e.g. `netcoreapp2.1`) | None | -| .NET Core 3.0 (`netcoreapp3.0`) | **It depends...** | +| .NET Core 3.0 and later .NET (`netcoreapp3.0`, `net6.0`, `net8.0` etc.) | **It depends...** | -Before .NET Core 3.0 came out, your choice of target framework would always determine which client-side UI frameworks were available. The _great unification_'s decision to include UI framework support as part of the unification rested on the assumption that this would be the case. But .NET Core 3.0 broke that. Once again, a decision that made sense when it was made was later undermined because one of its assumptions ceased to hold. +.NET Core 3.0 broke the assumption that your choice of target framework would fully determine which client-side UI frameworks were available. Why is it a problem? Well, what UI framework integration should Rx offer in its various targets? This table attempts to answer that question for all of the targets that [Rx 4.2](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.2.0) (the version that added [.NET Core 3.0 support](https://github.com/dotnet/reactive/pull/857)) supported: @@ -211,15 +224,15 @@ Why is it a problem? Well, what UI framework integration should Rx offer in its Why have I put "None" in the `netcoreapp3.0` row, bearing in mind that .NET .NET Core 3.0 added WPF and Windows Forms support? Well these UI frameworks are only available on Windows. The `netcoreapp3.0` TFM is OS-agnostic. With this target you could find yourself running on macOS or Linux. The Windows-specific underpinnings won't necessarily be there, and that's why I believe the correct answer for that row is "None". -As part of the [preparation for .NET 5 support](https://github.com/dotnet/reactive/pull/1291), a `net5.0` target was added. This did **not** include Windows Forms and WPF features. That is unarguably correct, because if you were to create a new project targetting `net5.0` and set either `UseWPF` or `UseWindowsForms` (or both) to `true` you'd get a build error telling you that you can only do that when the target platform is Windows. It recommends that you use an OS-specific TFM, such as `net5.0-windows`. +As part of Rx.NET's [preparation for .NET 5 support](https://github.com/dotnet/reactive/pull/1291), a `net5.0` target was added. This did **not** include Windows Forms and WPF features. That is unarguably correct, because if you were to create a new project targeting `net5.0` and set either `UseWPF` or `UseWindowsForms` (or both) to `true` you'd get a build error telling you that you can only do that when the target platform is Windows. It recommends that you use an OS-specific TFM, such as `net5.0-windows`. Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs changed in .NET 5.0](https://github.com/dotnet/designs/blob/main/accepted/2020/net5/net5.md). We didn't have OS-specific TFMs before .NET 5.0. So with .NET 5.0 and later, we can append `-windows` to indicate that we need to run on Windows. Since there was no way to do that before, `netcoreapp3.0` doesn't tell you anything about what the target OS needs to be. -My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available. That's why I put "None" in the 2nd column for that row. It seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target, and since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. +My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available (unless that component is specifically designed to be used only in environments where these frameworks will be available). That's why I put "None" in the 2nd column for that row. However, it seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target. Since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. -The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew that with a TFM of `net5.0-windows` you would definitely be running on Windows, although that was no guarantee that .NET 5's Windows Forms and WPF support was actually available. And a TFM of `net5.0` increased the chances of their not being available because you might not even be running on Windows. So let's look at the options again in this new .NET 5.0 world, listing all the TFMs that Rx 5.0 (the first version to support .NET 5.0) offered: +The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew that with a TFM of `net5.0-windows` you would definitely be running on Windows, although that was no guarantee that .NET 5's Windows Forms and WPF support was actually available. (On Windows, you can install just the [.NET 5.0 Runtime](https://dotnet.microsoft.com/en-us/download/dotnet/5.0) without including the .NET Desktop Runtime if you want.) And a TFM of `net5.0` increased the chances of their not being available because you might not even be running on Windows. So let's look at the options again in this new .NET 5.0 world, listing all the TFMs that [Rx 5.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v5.0.0) (the first version to support .NET 5.0) offered: | TFM | Which UI framework should Rx support? | What does it actually support? | |--|--|--| @@ -236,7 +249,7 @@ It does **not** repeat the mistake for `net5.0` but then it can't: when targetin The last row is interesting. Again, I've said it probably shouldn't include Windows Forms and WPF support. But really that's because I think that last row shouldn't even be there. There are good reasons that merely using some `-windows` TFM shouldn't automatically turn on WPF and Windows Forms support, but if you agree with that, then there's no longer any reason for Rx to offer a `-windows` TFM at all—there'd be no difference between those two .NET 5.0 TFMs at that point. -The reason I think Windows Forms and WPF support should not automatically be included just because you've used a `-windows` TFM is that there are many different reasons you might want such a TFM, most of which have nothing to do with either Windows Forms or WPF. For example, this is a completely legitimate C# console application: +The reason I think Windows Forms and WPF support should not automatically be included just because you've used a `-windows` TFM is that there are many different reasons you might want such a TFM, many of which have nothing to do with either Windows Forms or WPF. For example, this is a completely legitimate C# console application: ```cs using Windows.Devices.Input; @@ -250,9 +263,9 @@ Console.WriteLine($"Keyboard {keyboardCapabilities.KeyboardPresent}"); Console.WriteLine($"Touch {touchCapabilities.TouchPresent}"); ``` -This uses [WinRT-based APIs to discover whether certain forms of input are available on the machine](https://learn.microsoft.com/en-us/windows/apps/design/input/identify-input-devices). These APIs are available if I use a suitable Windows-specific TFM. They're only in Windows 10 or later, so I need to use a versioned Windows-specific TFM such as `net8.0-windows10.0.18362.0`. (The APIs I'm using are actually available starting with version 10.0.10240.0, but I've chosen version 10.0.18362 because that's the oldest Windows build number that Visual Studio 2022's installer supports) With just `net8.0` that code would fail to compile because these are Windows-only APIs. +This uses [WinRT-based APIs to discover whether certain forms of input are available on the machine](https://learn.microsoft.com/en-us/windows/apps/design/input/identify-input-devices). These APIs are available if I use a suitable Windows-specific TFM. They're only in Windows 10 or later, so I need to use a versioned Windows-specific TFM such as `net8.0-windows10.0.18362.0`. (The APIs I'm using are actually available starting with version 10.0.10240.0, but I've chosen version 10.0.18362 because that's the oldest Windows build number that the current .NET tooling supports.) With just `net8.0` that code would fail to compile because these are Windows-only APIs. -This illustrates the very specific meaning of OS-specific TFMs: they determine the OS-specific API surface area that your code will attempt to use. Here are some things that OS-specific TFMs **don't** mean: +This illustrates the very specific meaning of OS-specific TFMs: they determine what the OS-specific API surface area your code can attempt to use. Here are some things that OS-specific TFMs **don't** mean: * a minimum supported OS version (because code might use a new API when it runs on the latest OS version but be capable of handling its unavailability gracefully) * an intention to use WPF or Windows Forms (this particular program is a console application) @@ -263,18 +276,18 @@ If you want to use WPF, you set the `UseWPF` property to true in your project fi But Rx 5.0 takes the position that if an applications targets Windows, Rx should make its WPF and Windows Forms functionality available. (In fact, Rx doesn't support this on versions of Windows older than 10.0.19041, aka Windows 10 2004. So if your TFM specifies an older version, or no version at all (which implicitly means Windows 7 by the way) then Rx's WPF and Windows Forms won't be available.) -And the problem with that is that if you use any self-contained form of deployment (including Native AOT) in which the .NET runtime and its libraries are shipped as part of the application, that means your application will be shipping the WPF and Windows Forms parts of the .NET runtime library. Normally those are optional—the basic .NET runtime does not include them—so this is not a case of "well you'd be doing that anyway." +The problem with that is that if you use any self-contained form of deployment (including Native AOT) in which the .NET runtime and its libraries are shipped as part of the application, that means your application will be shipping the WPF and Windows Forms parts of the .NET runtime library. Normally those are optional—the basic .NET runtime does not include them—so this is not a case of "well you'd be doing that anyway." Let's look at the impact. The first column of the following table shows the size of the deployable output (excluding debug symbols, which get included in the published output by default) for the code shown above. The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. -| Deployment type | Size without Rx | Size with Rx targetting 18362 | Size with Rx targetting 19041 | +| Deployment type | Size without Rx | Size with Rx targeting 18362 | Size with Rx targeting 19041 | |--|--|--|--| | Framework-dependent | 20.8MB | 22.1MB | 22.5MB | | Self-contained | 90.8MB | 92.1MB | 182MB | | Self-contained trimmed | 18.3MB | 18.3MB | 65.7MB | | Native AOT | 5.9MB | 6.2MB | 17.4MB | -But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though I actually have no use for it. +But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`, the oldest Windows version for which Rx offers support on .NET 6.0 and later. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though I actually have no use for it. In the framework-dependent row it makes only a small difference (because the copy of `System.Reactive.dll` we get is a little larger). But that's misleading: the resulting executable will now required host systems to have not just the basic .NET 8.0 runtime installed, but also the optional Windows Desktop components. So unless the target machine already has that installed, I will in fact have a larger install to perform. @@ -282,13 +295,13 @@ The self-contained deployment is the worst. It has roughly doubled in size—it What's happened here is that because Rx has insisted on providing its Windows Forms and WPF support, the .NET SDK has had to include all of the .NET runtime library components that constitute Windows Forms and WPF, and those are large! That's where that extra 90MB comes from: a complete copy of TWO user interface frameworks, and my application isn't using either of them! -The self-contained trimmed version did a little better. It was able to work out that there was a whole load of code I wasn't using. But there's a limit to that. The trimmer is apparently not able to work out that I wasn't really using Windows Forms or WPF at all, so the deployment is still over 47MB larger. Or to put it another way, the deployment is about 3.5x the size that it needs to be! +The self-contained trimmed version did a little better. It was able to work out that there was a whole load of code I wasn't using. But there's a limit to its abilities. The trimmer is apparently not able to work out that I wasn't using Windows Forms or WPF at all, so the deployment is still over 47MB larger. Or to put it another way, comparing it with the other two columns in that row we can see that the deployment is about 3.5x the size that it needs to be! -The Native AOT version did better again. Obviously the absolute sizes are all significantly smaller, but the ratio of Rx without Windows Forms and WPF to Rx with Windows Forms and WPF is about 2.8x here. That's a lot better than ordinary trimming. The absolute increase of 11.2MB is relatively modest, and would be a smaller proportion of the whole in a larger application. +The Native AOT version did better again. Obviously the absolute sizes are all significantly smaller, but the ratio of Rx without Windows Forms and WPF to Rx with Windows Forms and WPF is also better, at about 2.8x here. That's a lot better than ordinary trimming. The absolute increase of 11.2MB is relatively modest, and would be a smaller proportion of the whole in a larger application. But it's still not great. And there are lots of scenarios in which Native AOT simply isn't an option. There are a fair few in which trimming can't be used either. So that unwanted 90MB in the self-contained deployment is a real problem in many scenarios. -(In case you're wondering why the self-contained deployment is so large, at 20.8MB, most of that is the `Microsoft.Windows.SDK.NET.dll` library. This gets included as a result of using a Windows-specific TFM, and using some of the WinRT-style APIs that it makes available. That library is where the types such as `MouseCapabilities` my example uses come from.) +(In case you're wondering why the framework-dependent deployment is so large, 20.8MB, most of that is the `Microsoft.Windows.SDK.NET.dll` library. This gets included as a result of using a Windows-specific TFM, and using some of the WinRT-style APIs that it makes available. That library is where the types such as `MouseCapabilities` my example uses come from.) So this is why, in the earlier table, I said that for the `net5.0-windows10.0.19401` the answer to the question "Which UI framework should Rx support?" should be "None." But why did I qualify it as "probably?" It's because I think they might not have had a choice: they had already painted themselves into a corner by this time. In order to avoid this, they would have had to have designed Rx 4 differently, and that ship had already sailed. @@ -307,18 +320,18 @@ If your application has encountered [the problem](#the-problem), you can add thi ``` -This needs to go just the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. The problem afflicts only executables, not DLLs. +This only needs to go in the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. The problem afflicts only executables, not DLLs. -here's an updated version of the table from the previous section. The final two columns are for the same application as last time, `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. +here's an updated version of the table from the previous section. The final two columns are for the same application as last time, targeting `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. -| Deployment type | Size without Rx | Size with Rx without workaround | Size with Rx using workaround | +| Deployment type | Size without Rx | Size with Rx, no workaround | Size with Rx and workaround | |--|--|--|--| | Framework-dependent | 20.8MB | 22.5MB | 22.5MB | Self-contained | 90.8MB | 182MB | 92.5MB | | Self-contained trimmed | 18.3MB | 65.7MB | 18.3MB | | Native AOT | 5.9MB | 17.4MB | 6.2MB | -As you can see, this is much more reasonable. In the first two cases, the output grows by the size of the `System.Reactive.dll` file. In the second two cases, the impact is considerably more modest. Rx makes a barely perceptible impact to the trimmed case. It's slightly more noticeable in Native AOT, but it's adding only about 300KB, roughly a 5% increase in size. +As you can see, this is much more reasonable. In the first two cases, using Rx.NET grows the output by the size of the `System.Reactive.dll` file. In the second two cases, the impact is considerably more modest. Rx makes a barely perceptible impact to the trimmed case. It's slightly more noticeable in Native AOT, but it's adding only about 300KB, roughly a 5% increase in size. So that seems pretty effective. @@ -337,27 +350,27 @@ We started a [public discussion about this topic](https://github.com/dotnet/reac I would not dismiss anything Anais has to say about Rx lightly. She is a long-standing Rx veteran and continues to be a positive force in the Rx community. In particular, I think we need to pay close attention to the concerns she raises over the damage we might do if we overcomplicate Rx's packaging. -However, the evidence compels me to disagree on her first point. My answers to her two questions here are: yes, and no, respectively. - -Yes, clearly people are worried about this. We know this because people have complained, and some major projects have abandoned Rx purely because of this problem. +However, the evidence compels me to disagree with the implication of her rhetorical question. My answers to her two questions here are: yes, and no, respectively. -As for the second point no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed in the next section) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. +Yes, clearly people are worried about this. We know this because people have complained, and some major projects have abandoned Rx purely because of this problem. They did not do that lightly. This demonstrates that yes, people really are worried. -The problem is that the answer to the first question is: yes, people are demonstrably worried enough about disk space in 2023 to drop Rx. That means we do actually have to do something. And for what it's worth I'm not convinced it's all about disk space per se. Network bandwidth is an issue. Not everyone has hundreds of megabits of internet connectivity. I live in the UK, which is a reasonably advanced economy, and I can get 1GB internet to my house, but only because I live in a built up area. A significant proportion of the population can't get even 10% of that speed. And that's in a country with unusually high average population density and a fairly high-tech economy. There are plenty of countries where poor connectivity is even more common than here, with no prospect of change any time soon. So adding 90MB unnecessarily to a download is a problem for significant parts of the world. +For what it's worth I'm not convinced it's all about disk space per se. Network bandwidth is an issue. Not everyone has hundreds of megabits of internet connectivity. I live in the UK, which is a reasonably advanced economy, and I can get 1GB internet to my house, but only because I live in a built up area. A significant proportion of the population can't get even 10% of that speed. And that's in a country with unusually high average population density and a fairly high-tech economy. There are plenty of countries where poor connectivity is even more common than here, with no prospect of change any time soon. So adding 90MB unnecessarily to a download is a problem for significant parts of the world. Even in a world of gigabit internet, web page download sizes matter. We added trimming support to Rx 6.0 specifically in response to people wanting to use it in Blazor. Bear in mind that in that scenario removed a mere 1MB from the download size, and yet it was a feature people care about. If 1MB can be a problem, I think it's safe to say that there are still scenarios today where 90MB matters. -But while I disagree with the premise that 90MB is no big deal, I think Anais offers some extremely important analysis. She worries that a solution to this problem could be: +But regardless of what the reasons might be, people are demonstrably worried enough about disk space in 2023 to drop Rx. And that means we do actually have to do something. -> worse than the original problem of "Wow there's too many DLLs". Please don't make a mess of the Rx namespaces again, we have already been through this, Rx is already confusing enough, we know from the first time around that having "too many DLLs" caused immense user confusion and (for some reason) accusations of "bloat" because "there were too many references" - now we are proposing making it even more confusing +As for Anais's second point, no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed in the next section) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. -What I take from this is that we should keep things as simple as possible, but no simpler. +And while I disagree with the premise that 90MB is no big deal, I think Anais offers some extremely important analysis. She worries that a solution to this problem could be: -It's worth noting that [glopesdev offered a slightly contrary opinion](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617): +> worse than the original problem of "Wow there's too many DLLs". Please don't make a mess of the Rx namespaces again, we have already been through this, Rx is already confusing enough, we know from the first time around that having "too many DLLs" caused immense user confusion and (for some reason) accusations of "bloat" because "there were too many references" - now we are proposing making it even more confusing -> I don't buy the argument of counting nodes in a "dependency tree" as a measure of complexity. Humans often care more about names than they care about numbers. My definition of "sanity" is not just that we have one package, it's that the root of the namespace System.Reactive should not be in a package called System.Reactive.Base. +My conclusion is that we should keep things as simple as possible, but no simpler. And that "no simpler" is the tricky bit. +By the way, it's worth noting that [glopesdev offered a slightly view, but one still focused on how easy it is to understand the packagin](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617): +> I don't buy the argument of counting nodes in a "dependency tree" as a measure of complexity. Humans often care more about names than they care about numbers. My definition of "sanity" is not just that we have one package, it's that the root of the namespace System.Reactive should not be in a package called System.Reactive.Base. I think the problem we have today is that the _great unification_ was, with hindsight, an oversimplification. So for me the questions are: @@ -381,6 +394,32 @@ A feature of the community response that surprised me was that a lot of people s About half of these make no attempt to explain why they think a breaking change will help. Simply breaking everything is no guarantee of fixing the problem! +`System.Reactive` seems to get about half a million downloads a month, so it seems reasonable to conclude that any change has the potential to affect a lot of people. You may find breaking changes acceptable, but there might be tens of thousands who take a different view. Moreover, I've helped enough teams out with dependency messes caused by ill-thought-out breaking changes to want to be extremely cautious about introducing breaking changes in a widely used package. + +It's also worth pointing out that not all breaking changes are created equal. Here are some examples: + +1. Recent versions of .NET have optimized the performance of `FileStream` +2. CLR serialization is most of the way through a very long process of deprecation +3. We could (hypothetically) remove the UI-specific feature from `System.Reactive` in v7 + +These are all, technically, binary breaking changes. For example, `FileStream` has become much less aggressive about synchronizing some of its internal state with the operating system. This has dramatically improved performance in certain scenarios, but it does mean any code written for .NET Core 3.1 that made certain assumptions about exactly how `FileStream` was implemented might break on .NET 8.0. Microsoft categorises this as a binary breaking change, but the reality is that the overwhelming majority of users of `FileStream` will be unaffected. Their code will run a little faster but nothing else will change. Microsoft went to great lengths to minimize any practical incompatibilities, and also, for a few releases, they provided a way to revert to the old behaviour. This was carefully thought out, there was a multi-release plan for how the new behaviour would be phased in, with safeguards in place in case the negative impact was worse than expected, and with escape hatches for anyone adversely affected. + +The second case, the gradual removal of CLR serialization, is quite different. In this case, functionality is being removed. Anyone dependent on that functionality will be out of luck once it goes entirely. This is being done because CLR serialization is a security liability, and it's a feature nobody should really be using. But lots of people have used it in the past, which is why it was brought over from .NET Framework to .NET. So although this is a more brutal kind of breaking change than 1) above, it is being handled in a way designed to give developers using it a very long runway for finally breaking free of it. There is a [published plan](https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md) for how it will be phased out. There has been a phased approach in which there were initially just warnings, and then a change where it was disabled by default but could easily be re-enabled. It will eventually vanish completely, but we had years of notice that it was on the way out. Even though CLR serialization was re-introduced in .NET Core explicitly as a stopgap measure, very clearly signposted as something intended only to support porting of code from .NET Framework, and not something to be used in new development, the deprecation was done over about half a decade, and 5 releases of .NET. + +Now compare that with 3), the idea that we should relax about backwards compatibility and just remove the problematic APIs from Rx. That very different from 2) (which in turn is very different from 1). This would be a sudden shock for existing users of Rx. This is absolutely guaranteed to cause problems for anyone who was using Rx in a way that it was very much designed to be used. That's a very different sort of breaking change from one that only affects people who knowingly used a doomed API, or one that doesn't affect anyone using an API normally. + +I should clarify that we're not totally opposed to breaking changes, we just want to ensure the following: + +* they actually have the intended effect +* they aren't worse than necessary +* people have sufficient notice to be able to deal with the change + +We think that in the long run we do need to make a breaking change to deal with this issue properly: we need to get the UI-framework-specific pieces out of `System.Reactive`. So this can never be as gentle a breaking change as 1) above: we are intending to remove something from the API. + +It won't be quite as gentle as 2) either. The thing about CLR serialization is that anyone using it in .NET Core knew its days were numbered from the start. But in Rx we're talking about a change that had not been envisaged back in 2018 when the Rx API adopted its current form. + +But I don't want to do anything as brutal as 3. And the difference between 2 and 3 is essentially the combination of fair warning and time. + #### 'Clean starts' aren't, due to extension method ambiguity @@ -388,15 +427,17 @@ David Karnok made a [killer argument against so-called clean starts](https://git >As for the clean start, there would be a lot of clashes due to the use of extension methods, right? For example, if old Rx and new Rx needed to be interoperated, whose Select() will be applied on that IObservable? -For me this pretty much destroys the idea of a 'clean break' to enable breaking changes. The exact meaning of 'clean break' being presumed here is described in the [Option 2: 'clean break' section](#option-2-clean-break-introducing-new-packages-with-no-relationship-with-current-rx) later in this document. That section also explains why the point David raised here is fatal for that design. +For me this pretty much destroys the idea of a 'clean break' to enable breaking changes. The exact meaning of 'clean break' being presumed here is described in the [Option 2: 'clean break' section](#option-2-clean-break-introducing-new-packages-with-no-relationship-with-current-rx) later in this document. That section also explains why the point David raised here is fatal for that design. (I just wanted to mention it in this part of the doc, because this is a summary of all the community feedback we received.) #### Are we holding Rx to too high a standard? -Some people think it's wrong of us to try to maintain high levels of backwards compatibility. I think that's debatable, but if you are going to take that position, then it demands the question: how much backwards compatibility is enough? How much can we break. +Some people think it's wrong of us to try to maintain high levels of backwards compatibility. I think that's debatable, but if you are going to take that position, then it demands the question: how much backwards compatibility is enough? How much can we break? As it happens, I made clear from the start that total compatibility was not a goal: I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. +TODO: this bit needs to be more coherent. I'm replying to things I'm not quoting... + > [I don't understand why given this we are assuming that System.Reactive needs to somehow hold itself to a higher standard than the core framework itself.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617) One reason is that we can't actually impose the split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be on one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to @@ -419,19 +460,36 @@ I'd quite like to hear the two of them debate this, because they both have formi #### Windows version numbers +One request we had (via a backchannel, so I can't post a link) was to support older OS versions in whichever Rx.NET components still have a Windows-specific TFM. + +This refers to the fact that to get access to Rx.NET's Windows Forms or WPF functionality, it's not enough for an application to target `net8.0-windows`. You have to specify `net8.0-windows10.0.19041` or later. + +I believe the main reason behind this request is simply that the status quo really confuses people. As far as I know, this change wouldn't enable any new scenarios. I think it has more to do with the fact that version-specific TFMs are frequently misunderstood. + +For example, it does not appear to be widely understood that all OS-specific TFMs are also version-specific. If you don't specify a version, the SDK picks one for you. (The documentation doesn't appear to tell you _which_ version it picks, but if you look at the build variables with a tool such as the MS Build structured log viewer, you can work it out. Or you can dig around in the .NET SDK installation folder and look at what the various `.props` and `.targets` files in there do..) On the .NET 8.0 SDK, if you leave out the version number on a Windows TFM, it implies Windows 7. + +Also, it does not appear to be that well understood that specifying a version doesn't necessarily imply a minimum required version. It only determines whether you're able to attempt to use an API; the attempt will fail if you're on a version of Windows that doesn't offer the API you want but that might be fine—maybe your application can gracefully downgrade its operation. + +So in general, specifying a Windows 10.0.19041 TFM doesn't really cause any problems. The main thing that would be achieved by changing Rx to target `net8.0-windows` would be that fewer people would be tripped up by this. + +That said there's one possible benefit: specifying a Windows 10 version-specific TFM does result in an interop DLL that's roughly 20MB in size being included in your application. Arguably the ability to remove that would be a benefit. + +Currently, I don't know if this is possible. When [endjin](https://endjin.com) took over maintenance of Rx.NET, we did try to find out why the Windows-specific Rx.NET targets chose 10.0.19041. I never got an answer. I think we can't go back any further than 10.0.17763 because I _think_ [C#/WinRT](https://learn.microsoft.com/en-us/windows/apps/develop/platform/csharp-winrt/) requires that version or later, and in practice 10.0.18362 might be the lowest we can use in practice because that's the oldest SDK that Visual Studio 2022 supports. #### Use obsolete +One of the comments was a recommendation to deal with this problem using `[Obsolete]`: + https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7604157 +The reason we hadn't originally proposed this is that we thought we needed to take action quickly to resolve the problem. `[Obsolete]` is really only suitable when you form a multi-year to remove an API, and people were having real problems right now. -### Constraints +However, now that a [workaround](#the-workaround) seems to be available, using `[Obsolete]` does in fact look likely to be the best option. -Our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. -These expectations are not unreasonable because Rx.NET has been positioned as a core, flexible piece of technology. Its widespread use has been strongly encouraged, and as its new(ish) maintainers, we at [endjin](https://endjin.com) continue to encourage this. By doing so we are effectively setting expectations around backwards compatibility similar to those that could reasonably apply to types in `Microsoft.Extensions.*` namespaces, and perhaps even to types in the .NET runtime libraries themselves. +### Constraints -This goal creates some constraints. +There are a few constraints that we need to impose on any possible solution to this problem. The following sections describe these constraints, and the reasons for imposing them. #### Can't remove types until a long Obsolete period @@ -515,18 +573,25 @@ And that gets us back to square one: if taking a dependency on `System.Reactive` It would be technically possible to meddle with the build system's normal behaviour in order to produce a `System.Reactive` assembly with a suitable type forwarder, but for the resulting NuGet package not to have the corresponding dependency. However, this is unsupported, and is likely to cause a lot of confusion for people who actually do want the WPF functionality, because adding a reference to just `System.Reactive` (which has been all that has been required for Rx v4 through v6) would still enable code using WPF features to compile when upgrading to this hypothetical form of v7, but it would result in runtime errors due to the `System.Reactive.Wpf` assembly not being found. So this is not an acceptable workaround. +In short, if we were to introduce a breaking change, we don't just create a situation where applications need to fix a few things when they upgrade. We create a situation where using certain combinations of other libraries become unusable. The only ways to resolve this are either to hope all your libraries get upgraded soon, or to decide not to use all of the libraries you want to. + +As is often the case, it's the dependency scenarios that cause trouble. For this reason, our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. + +These expectations are not unreasonable because Rx.NET has been positioned as a core, flexible piece of technology. Its widespread use has been strongly encouraged, and as its new(ish) maintainers, we at [endjin](https://endjin.com) continue to encourage this. By doing so we are effectively setting expectations around backwards compatibility similar to those that could reasonably apply to types in `Microsoft.Extensions.*` namespaces, and perhaps even to types in the .NET runtime libraries themselves. + +And whatever people's expectations are, the fact that Rx.NET is widely used means we can cause people some real problems if we introduce breaking changes suddenly. So that's why we want a long `[Obsolete]` period before removing anything. + #### ...except for UWP We are considering making an exception to the constraint just discussed for UWP. The presence of UWP code causes considerable headaches because UWP is not a properly supported target. The modern .NET SDK build system doesn't fully recognize it, and we end up using the [`MSBuild.Sdk.Extras`](https://github.com/novotnyllc/MSBuildSdkExtras) package to work around this. That repository hasn't had an update since 2021, and it was originally written in the hope of being a stopgap while Microsoft got proper UWP support in place. Proper UWP support never arrived, mainly because UWP is a technology Microsoft has long been telling people not to use. -We don't want to drop UWP support completely, but we are prepared to contemplate removing the UWP-specific target (`uap10.0.16299`). UWP has long supported .NET Standard 2.0, so Rx.NET would still be available. However, the UWP-specific types would no longer be in `System.Reactive`. (We would move them into a separate NuGet package.) +We don't want to drop UWP support completely, but we are prepared to contemplate removing the UWP-specific target (`uap10.0.16299`) much earlier than any of the other targets, possibly right away. UWP has long supported .NET Standard 2.0, so Rx.NET would still be available if we did this. The only change would be that the UWP-specific types would no longer be in `System.Reactive`. (We would move them into a separate NuGet package.) This is problematic for all of the reasons just discussed in the preceding section. However, as far as we know UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. #### Transitive references to different versions - -consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: +Consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: * UiLibA uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` * NonUiLibB uses Rx 7, and it does not use any UI-framework-specific Rx features but it does use new functionality in Rx 7 @@ -615,8 +680,10 @@ Although some projects continue to use `packages.config` today, proper package r It is explicitly a non-goal to support `packages.config`. Our position is that projects stuck in that world can continue to use Rx 6.0. +#### Minimize confusion as far as possible + -### Other contextual changes to consider +#### .NET Standard mustn't break things A possible fly in the ointment is the netstandard2.0 component, because that could in principle run in a .NET Framework process. We could still end up with that loading first, blocking any attempt to load the net472 version some time later. However, in the plug-ins scenario above, that shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the net472 version over the netstandard2.0 one. (If they target an older version such as net462, then perhaps they would pick netstandard2.0 instead. But then the current Rx 6.0 design fails in that scenario too. So unwinding the earlier design decisions won't make things any worse than they already are.) @@ -689,7 +756,7 @@ This design option does not solve this problem. I think this is a fundamental pr This design option also doesn't have a good answer for how we provide UI-framework-specific support for other frameworks. (E.g., how would we offer a `DispatcherScheduler` for MAUI's `IScheduler`?) -So in short, I (@idg10) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from problem 3. +So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from problem 3. The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it closely resembles the existing design. From 2287b1838f5be43d6be5a2919f2498d4c27a15ee Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Mon, 5 Feb 2024 08:04:25 +0000 Subject: [PATCH 10/19] ADR feature complete up to design options section --- ...0003-windows-tfms-and-desktop-framework.md | 73 +++++++++++++------ 1 file changed, 50 insertions(+), 23 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 3f4a2e9da..366bd013a 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -589,6 +589,15 @@ We don't want to drop UWP support completely, but we are prepared to contemplate This is problematic for all of the reasons just discussed in the preceding section. However, as far as we know UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. +There are a couple of options here: + +* Remove UWP support in Rx v7 +* Obsolete UWP support in Rx v7, then remove it in v8 + +As much as I want to be shot of UWP as soon as possible, I think the second option is probably the better one. + +In either case, we would retain an emergency fallback position: if it turns out we underestimated the popularity of UWP, and we receive a significant number of complaints when we remove UWP, we could reinstate it. So we could ship, say, 8.0.0 without UWP and if that turns out to be a horrible mistake, we could release an 8.0.1 with it re-enabled. (This means we'd actually need to leave all the UWP code in place for yet another release cycle, just in case we needed to do that sudden U turn. If a year passed with nobody complaining that our moving the UWP support out into a separate package was unworkable for them, then we would look at removing UWP properly (except for the one UWP-specific package) to remove all the build problems associated with keeping it around.) + #### Transitive references to different versions Consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: @@ -596,25 +605,25 @@ Consider an application MyApp that has two dependencies (which might be indirect * UiLibA uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` * NonUiLibB uses Rx 7, and it does not use any UI-framework-specific Rx features but it does use new functionality in Rx 7 -Imagine for the sake of argument that Rx 7 adds a `RateLimit` operator to deal with the fact that almost nobody likes Throttle. +Imagine for the sake of argument that Rx 7 adds a `RateLimit` operator to deal with the fact that almost nobody likes `Throttle`. So we now have this situation: * UiLibA will require `DispatcherScheduler` to be available through `System.Reactive` v5.0 * NonUiLibB will require `Observable.RateLimit` to be available through whatever reference it's using to get Rx 7 -This needs to work. This constraint prevents us from simply removing UI-specific types from `System.Reactive` v7.0. +This needs to work. This constraint prevents us from simply removing UI-specific types from `System.Reactive` in v7.0. #### Minimum acceptable path for breaking changes When a developer using Rx.NET upgrades to a newer version, what is the smallest version increment for which we are prepared to consider breakage to be acceptable? What's the smallest length of time between an Rx.NET release coming out, and a subsequent release making a breaking change? What are our minimum requirements around giving fair notice with `[Obsolete]` annotations? -Let's start with some upper and lower bounds: +Let's start by picking some upper and lower bounds as a starting point. These aren't meant to be the final answer. The idea here is to pick timespans where we can agree that the upper bound isn't long enough, or that the lower bound is too short: * If a project upgrades from Rx.NET 1.0 to 7.0, it is probably reasonable for some things to break * Components with a dependency on Rx 5.0 or 6.0 running in an application that upgrades that to Rx 7.0 must not be broken by this upgrade -That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. +That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. (That said, some people seem to be [positively enthusiastic about breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), and would actually disgree with the lower bound I've put here. But our view is that the version of Rx that was still the very latest version as recently as 18th May 2023. Less than a year has passed, which seems like not nearly enough time to be removing API features that even in the current version are not marked as obsolete. So although we recognize that there are some people who think it would be OK to break things more quickly than the lower bound stated above suggests, we aren't going to do that, meaning that this is a lower bound in practice.) There's another dimension we need to take into account: @@ -625,11 +634,11 @@ That second category has the potential to be more insidious, because it can mean This is particularly troublesome in large complex projects that have been in development for many years. These often end up with large numbers of dependencies of widely varying age. A successful project that is, say, 5 years old may well have taken a dependency early on on some component that used Rx 4.0. And perhaps that dependency has worked perfectly well in all that time, with no need to be upgraded. Now suppose a dependency is added to a new component, and this new component wants Rx.NET 7. -Is it acceptable for the code to break because that old component is relying on a very old version of Rx.NET? Rx 4.0 would be 4 major versions behind the latest version in this scenario, and it's a version that shipped over half a decade ago. +Is it acceptable for the code to break because that old component is relying on a very old version of Rx.NET? Rx 4.0 would be 3 major versions behind the latest version in this scenario, and it's a version that shipped over half a decade ago. -Some would argue that this makes it acceptable for the application to break in this scenario. However, there are some reasons to disagree with this: +Some would argue that this makes it acceptable for the application to break in this scenario. However, it's not entirely clear cut. Here are some reasons to disagree: -* We have not yet made any clear indication (e.g., marking things as [Obsolete]) of plans to remove features from Rx.NET, so this would effectively be a bolt from the blue +* We have not yet made any clear indication (e.g., marking things as [Obsolete]) of plans to remove any of the features under discussion here from Rx.NET, so this would effectively be a bolt from the blue—Rx 4.0 may be half a decade old but we'd still be giving no notice of change whatesoever * This might create a situation where the application developers are forced to abandon one or other of the components they want to use * Although the current .NET runtime introduced an exciting world where 3 years counts as "long term support" the .NET Framework still sits in the Windows world in which support typically lasts for 10 years, so it's not entirely unreasonable to expect compatibility with a 5 year old release @@ -641,59 +650,77 @@ I think we might need to be even more conservative than that, because when Rx is This suggests we need to consider how long it will take for the .NET ecosystem to respond to our changes. It might take three years for a component author to deal with a forthcoming breaking change in Rx, but it might then take another three years for applications using that component to get around to updating. -This suggest that a 6 year window might be reasonable. Anything we mark as obsolete today in 2024 can't be removed entirely until 2030. +This suggest that a 6 year window might be reasonable. Anything we mark as obsolete today in 2024 can't be removed entirely until 2030 (apart from [UWP, as discussed earlier](#except-for-uwp)). #### Limited tolerance for breaking changes -While we would prefer to have no breaking changes at all, there are some circumstance where the are acceptable: security considerations can trump backwards compatibility for example. (Not that we currently know of any examples of that in Rx.) In considering a breaking change we need to ask which of the following would apply in cases where a change does break an application: +While we would prefer to have no breaking changes at all, there are some circumstance where they are acceptable: security considerations can trump backwards compatibility for example. (Not that we currently know of any examples of that in Rx.) In considering a breaking change we need to ask which of the following would apply in cases where a change does break an application: 1. the app authors need to make some changes to get everything working again 2. we put the app authors in a position where it's not possible for them to get everything working again Simply removing UI-specific types in `System.Reactive` v7 would be a type 2 change (because of [the need for things to work when we have indirect dependencies on multiple versions](#transitive-references-to-different-versions)). -We consider type 2 changes to be unacceptable. We consider all breaking changes undesirable, but would be open to type 1 changes if no better alternative exists. +We consider type 2 changes to be unacceptable unless [sufficient time has passed](#minimum-acceptable-path-for-breaking-changes). We consider all breaking changes undesirable, but would be open to type 1 changes on a shorter (e.g. 1-4 year) timescale if no better alternative exists. -#### Compatibility with plug-in systems need to work +#### Plug-in systems must work We must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. -It's worth re-iterating that 1 is a non-issue on .NET 6.0+ hosts. Applications hosting plug-ins on those runtimes will use AssemblyLoadContext, enabling each plug-in to make its own decisions about which components it uses. If plug-ins make conflicting decisions, it doesn't matter, because AssemblyLoadContext is designed to allow that. +It's worth re-iterating that this is a non-issue on .NET 6.0+ hosts. Applications hosting plug-ins on those runtimes will use [`AssemblyLoadContext`](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext), enabling each plug-in to make its own decisions about which components it uses. If plug-ins make conflicting decisions, it doesn't matter, because `AssemblyLoadContext` is designed to allow that. + +So we only need to consider .NET Framework. But there has been one critical change since Rx.NET first started running into the problems that led to the current situation: Rx.NET now defines exactly one .NET Framework target. The original problem occurred only because we had both `net40` and `net45` targets. Now, we have only `net472`. + +However, in design options where we consider splitting Rx.NET across multiple packages (e.g., where we move the UI-framework-specific pieces out into separate UI-framework-specific packages) we also need to be careful that we don't recreate the problems that occurred in Rx 3.0. We need to be certain that plug-ins that use two Rx.NET assemblies (e.g., the main Rx.NET assembly and the one containing WPF integration) can't end up with a mismatched pair because some other plug-in already forced the loading of an older version of just one of those assemblies. + +Since we change Rx.NET assembly versions with each major release, we shouldn't get a situation where `PlugInOne` loads, say, `System.Reactive` 7.0.0, and `PlugInTwo` wants v8.0.0 of `System.Reactive` and (say) `System.Reactive.Wpf` but ends up with v7 of `System.Reactive` (because that's what `PlugInOne` already loaded) and v8 of `System.Reactive`. These two different versions of `System.Reactive` would have different strong names, so .NET Framework will load both versions in this scenario. + +But there is a more subtle scenario. `PlugInOne` and `PlugInTwo` might both depend on slightly different versions of Rx v8. Assuming we continue to keep .NET assembly version numbers fixed across major releases of Rx, this means that assemblies from Rx 8.0.0 and Rx 8.0.1 would have the same strong names. So it would be possible for `PlugInTwo` to end up with a mixture of 8.0.0 and 8.0.1 assemblies. -So we only need to consider .NET Framework. But there has been one critical change since this problem originally occurred: Rx.NET now defines exactly one .NET Framework target. The original problem occurred only because we had both `net40` and `net45` targets. Now, we have only `net472`. +The effect of this is to impose the following constraint: within a single major release, there must be both backward and forward compatibility between `System.Reactive` and any UI framework support Rx components. E.g., `System.Reactive` v8.0.X and `System.Reactive.Wpf` v8.0.Y must be able to coexist for any combination of X and Y. -However, in design options where we consider splitting Rx.NET across multiple packages (e.g., where we move the UI-framework-specific pieces out into separate UI-framework-specific packages) we also need to be careful that we don't recreate the problems that occurred in Rx 3.0. +Note that we aren't going to try and fix the problem where `PlugInOne` loading and older version of some Rx.NET assembly prevents `PlugInTwo` from loading a later version of the same assembly when they have the same major version number. That problem exists today, and is fundamentally unavoidable with a .NET Framework plug-in system. -Rx 3.0 solved 1 by using different 'build numbers' (the 3rd part of the assembly version) for different targets. For example, DLLs targeting net45 would have a version of 3.0.1000.0; net451 DLLs used 3.0.2000.0; net46 DLLs used 3.0.3000.0. These assembly version numbers remained the same across multiple NuGet releases, so even in the release that NuGet labels as 3.1.1, the assembly version numbers were all 3.0.xxxx.0. So if a plug-in built for .NET 4.0 used Rx 3.1.1, and was loaded into the same process as another plug-in also using Rx 3.1.1 but built for .NET 4.5, they would be able to load the two different System.Reactive.Linq.dll assemblies because those assemblies had different identities by virtue of having different assembly version numbers. Unfortunately, while this solved problem 1, it caused problem 2: it was relatively easy to confuse NuGet into thinking that you had conflicting dependencies. #### Coherent versions across multiple components -One of the issues that occured with the [Rx 3 era attempt to fix this problem](https://github.com/dotnet/reactive/pull/212) was that people sometimes found that [referencing two Rx.NET components with the same apparent version number could produce version conflicts](https://github.com/dotnet/reactive/issues/305). +One of the issues that occured with the [Rx 3 era attempt to fix this problem](https://github.com/dotnet/reactive/pull/212) was that people sometimes found that [referencing two Rx.NET components with the same apparent version number could produce version conflicts](https://github.com/dotnet/reactive/issues/305). (This is essentially the root cause of the plug-in problem. But I'm keeping the requirements for [plug-in systems to work](#plug-in-systems-must-work) as a separate constraint because that's a key scenario. It may seem redundant to have a separate section addressing the root cause, but it's possible that this problem causes other issues we don't know about.) -Rx 4.0 solved this (and Rx 6.0 still uses the same approach) by collapsing everything down to a single NuGet package, System.Reactive. To access UI-framework-specific functionality, you no long needed to add a reference to a UI-framework-specific package such as System.Reactive.Windows.Forms. From a developer's perspective, the functionality was right there in System.Reactive. The exact API surface area you saw was determined by your target platform. If you built a UWP application, you would get the lib\uap10.0.16299\System.Reactive.dll which had the UWP-specific dispatcher support built in. If your application was built for net6.0-windows10.0.19041 (or a .NET 6.0 TFM specifying a later Windows SDK) you would get lib\net6.0-windows10.0.19041\System.Reactive.dll which has the Windows Forms and WPF dispatcher and related types build in. If your target was net472 or a later .NET Framework you would get lib\net472\System.Reactive.dll, which also included the Windows Forms and WPF dispatcher support (but built for .NET Framework, instead of .NET 6.0). And if you weren't using any client-side UI framework, then the behaviour depended on whether you were using .NET Framework, or .NET 6.0+. With .NET Framework you'd get the net472 DLL, which would include the WPF and Windows Forms support, but it didn't really matter because those frameworks were present as part of any .NET Framework installation. And if you target .NET 6.0 you'll get the lib\net6.0\System.Reactive.dll, which has no UI framework support, but that's fine because any .NET 6.0+ TFM that doesn't mention 'windows' doesn't offer either WPF or Windows Forms. +Rx 4.0 solved this (and Rx 6.0 still uses the same approach) by collapsing everything down to a single NuGet package, `System.Reactive`. To access UI-framework-specific functionality, you no long needed to add a reference to a UI-framework-specific package such as `System.Reactive.Windows.Forms`. From a developer's perspective, the functionality was right there in `System.Reactive`. The exact API surface area you saw was determined by your target platform. If you built a UWP application, you would get `lib\uap10.0.16299\System.Reactive.dll` which had the UWP-specific dispatcher support built in. If your application was built for `net6.0-windows10.0.19041` (or a .NET 6.0 TFM specifying a later Windows SDK) you would get `lib\net6.0-windows10.0.19041\System.Reactive.dll` which has the Windows Forms and WPF dispatcher and related types build in. If your target was `net472` or a later .NET Framework you would get `lib\net472\System.Reactive.dll`, which also included the Windows Forms and WPF dispatcher support (but built for .NET Framework, instead of .NET 6.0). And if you weren't using any client-side UI framework, then the behaviour depended on whether you were using .NET Framework, or .NET 6.0+. With .NET Framework you'd get the `net472` DLL, which would include the WPF and Windows Forms support, but it didn't really matter because those frameworks were present as part of any .NET Framework installation. And if you target .NET 6.0 you'll get the `lib\net6.0\System.Reactive.dll`, which has no UI framework support, but that's fine because any .NET 6.0+ TFM that doesn't mention 'windows' doesn't offer either WPF or Windows Forms. -It's worth noting that this problem goes away if you use the modern project system. The problem described in [#305](https://github.com/dotnet/reactive/issues/305) occurs only with the old packages.config system. If you try to reproduce the problem in a project that uses the .NET SDK project system introduced back in MSBuild 15 (Visual Studio 2017), the problem won't occur. Visual Studio 2017 is the oldest version of Visual Studio with mainstream support. +It's worth noting that even if we do go back to a multi-package approach, this problem goes away if you use the modern project system. The problem described in [#305](https://github.com/dotnet/reactive/issues/305) occurs only with the old `packages.config` system. If you try to reproduce the problem in a project that uses the .NET SDK project system introduced back in MSBuild 15 (Visual Studio 2017), the problem won't occur. Visual Studio 2017 is the oldest version of Visual Studio with mainstream support. Although some projects continue to use `packages.config` today, proper package references have been available for a long time now, even for projects that are still stuck on the older style of project file. -It is explicitly a non-goal to support `packages.config`. Our position is that projects stuck in that world can continue to use Rx 6.0. +It is explicitly a non-goal to support `packages.config`. Our position is that projects stuck in that world can continue to use Rx 6.0. So our constraint is only that we don't somehow create a situation in which references to multiple Rx.NET packages can result in an incoherent set of versions even when using package references. + #### Minimize confusion as far as possible +[Anais Betts has implored us](#does-size-matter) not to "make a mess of" things again. This is a subjective requirement, but an important one nonetheless. The basic goal here is that it should be easy to start using Rx.NET. In normal use, this means you should need only a single NuGet package reference, and it shouldn't be difficult to work out which one you need. + +We are not interpreting this to mean that the current position, in which there is only one package (`System.Reactive`) and it serves all purposes must necessarily be preserved. We are currently of the view that it is acceptable for UI-framework-specific support to be in separate components. That would necessarily be the case for frameworks Rx.NET does not provide built-in support for such as Avalonia. And there are now so many UI frameworks for .NET that we think it's a self-evident truth that Rx.NET can't provide built-in support for all of them. And once you accept that some UI-framework-specific support is to be found in separate NuGet packages, it is arguably less confusing if they all work that way. + +People have demonstrably been confused by the current (v4 through v6) setup. For all that a single-package approach is simple, people still get tripped up by the highly non-obvious requirement to change their TFM from `net6.0-windows` to `net6.0-windows10.0.19041`. (And over a year after taking over maintenance of this project, I've still not yet managed to find a satisfactory answer as to why it's that particular version. So I don't know how this is meant to be simple for anyone just getting started with Rx.NET.) + +So we are open to multiple packages. But we don't want to let that mean that we open the floodgates to a much more granular approach. + #### .NET Standard mustn't break things -A possible fly in the ointment is the netstandard2.0 component, because that could in principle run in a .NET Framework process. We could still end up with that loading first, blocking any attempt to load the net472 version some time later. However, in the plug-ins scenario above, that shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the net472 version over the netstandard2.0 one. (If they target an older version such as net462, then perhaps they would pick netstandard2.0 instead. But then the current Rx 6.0 design fails in that scenario too. So unwinding the earlier design decisions won't make things any worse than they already are.) +A possible fly in the ointment is the `netstandard2.0` component, because that could in principle run in a .NET Framework process. We could still end up some plug-in loading that one first, blocking any attempt by subsequent plug-ins to load the `net472` version. + +I've put this section in here not because I expect it to happen, but just because we need to double check that it doesn't. In the plug-in scenario described above, it shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the `net472` version over the `netstandard2.0` one. (If they target an older version such as `net462`, then perhaps they would pick `netstandard2.0` instead. But if that's the case, then the current Rx 6.0 design fails in that scenario too, so unwinding the earlier design decisions won't make things any worse than they already are.) Another consideration is that modern NuGet tooling is better than it was in 2016 when the current design was established. Alternative solutions might be possible now that would not have worked when Rx 4.0 was introduced. -When it comes to .NET 6.0 and later, these problems should a non-issue because better plug-in architectures exist thanks to AssemblyLoadContext. +When it comes to .NET 6.0 and later, these problems should a non-issue because better plug-in architectures exist thanks to `AssemblyLoadContext`. ### Types in `System.Reactive` that are partially or completely UI-framework-specific -Unwinding the consolidation that happened in V4 comes with a challenge: `System.Reactive` now has a slightly different API surface area on different platforms. There are some types available only on specific targets. There is one type that is available on all targets but has a slightly different public API on one target. And there are some types with the same API surface area on all targets but with differences in behavior. This section describes the afflicted API surface area. +Before we go on to consider various design options, it's useful to understand exactly which bits of the current Rx.NET API are likely to need to change. Thanks to the _great unification_, `System.Reactive` now has a slightly different API surface area on different platforms. There are some types available only on specific targets. There is one type that is available on all targets but has a slightly different public API on one target. And there are some types with the same API surface area on all targets but with differences in behavior. This section describes the afflicted API surface area. Types available only in specific targets: @@ -723,7 +750,7 @@ Behavior that is different across platforms: * UWP * `Scheduler` periodic scheduling has workaround for WinRT not supporting <1ms resolution -* `net6.0-windows*` and UWP +* `net6.0-windows10.*` and UWP * `IHostLifecycleNotifications` service available from enlightenment provider * Periodic scheduling will be aware of windows suspend/resume events only if using `System.Reactive` for these targets From 73cb63fd143b4f5b305c1afd4d85f83ea704df9d Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Mon, 5 Feb 2024 08:30:55 +0000 Subject: [PATCH 11/19] Done up to option 2 --- ...0003-windows-tfms-and-desktop-framework.md | 22 +++++-------------- 1 file changed, 6 insertions(+), 16 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 366bd013a..5e90c93df 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -763,29 +763,19 @@ The following sections describe the design choices that have been considered to The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) makes this a more attractive option than it had seemed when we first started looking at this problem. -Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is just to continue doing things in the same way. This is not a good solution. Back when Rx 5.0 was the current version, some people seemed to think that the changes we adopted in Rx 6.0 would be sufficient to solve the problems described in this document. But as will be explained, it doesn't. +Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is just to continue doing things in the same way. This is not a good solution. Back when Rx 5.0 was the current version, some people seemed to think that the changes we adopted in Rx 6.0 would be sufficient to solve the problems described in this document, but as is now clear, they don't. -`System.Reactive` 5.0 targeted `netstandard2.0`, `net472`, `netcoreapp3.1`, `net5.0`, `net5.0-windows10.0.19041`, and `uap10.0.16299`. The idea with this design option was to target `netstandard2.0`, `net472`, `net6.0`, `net6.0-windows10.0.19041`, and `uap10.0.16299`. (In other words, drop .NET Core 3.1 and .,NET 5.0, both of which went out of support in 2022, and effectively upgrade the .NET 5.0 target to .NET 6.0.) This is in fact exactly what we did for Rx 6.0, but despite what some people seemed to believe, this was never going to solve the problems described above. +`System.Reactive` 5.0 targeted `netstandard2.0`, `net472`, `netcoreapp3.1`, `net5.0`, `net5.0-windows10.0.19041`, and `uap10.0.16299`. Rx 6.0 targets `netstandard2.0`, `net472`, `net6.0`, `net6.0-windows10.0.19041`, and `uap10.0.16299`. (In other words, it dropped .NET Core 3.1 and .,NET 5.0, both of which went out of support in 2022, and effectively upgraded the .NET 5.0 target to .NET 6.0.) This is in fact exactly what we did for Rx 6.0, but despite what some people seemed to believe, this was never going to solve the problems this ADR discusses. -Let's look at how this gets on with the three challenges: +This meets all of the [constraints](#constraints), for the simple reason that those are all concerned with not making things worse than they already are. This _is_ where we already are, so it can't possibly be any worse than where we are. But the big problem is the one stated at the start of this ADR: the fact that self-contained deployments are vastly bloated by unwanted dependencies on Windows Forms and WPF. -**1: Host applications with a plug-in model getting into a state where plug-ins disagree about which `System.Reactive.dll` is loaded** - -Since the plug-in issues are only relevant to .NET Framework, and this doesn't change the .NET Framework packaging in any way, this solves the problem in the same way that Rx 4.0 and 5.0 do. - -**2: Incompatible mixes of version numbers of Rx components** - -Rx 4.0 introduced the unified packaging to solve this problem, and this option retains it, so it will solve the problem in the same way. - -**3. Applications getting WPF and Windows Forms dependencies even though they use neither of these frameworks** - -This design option does not solve this problem. I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that, and I think separating out those parts is the only way to achieve this. +I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that. The [workaround](#the-workaround) provides a way to undo this problem, but workaround usually have their own problems, and we don't want people to have to discover and then apply a fix just to be able to use Rx.NET. I think separating out these parts is the only way to achieve this. This design option also doesn't have a good answer for how we provide UI-framework-specific support for other frameworks. (E.g., how would we offer a `DispatcherScheduler` for MAUI's `IScheduler`?) -So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from problem 3. +So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from primary problem this ADR discusses. -The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it closely resembles the existing design. +The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it _is_ the existing design. #### Option 2: 'clean break' introducing new packages with no relationship with current Rx From 567613cdb4d0fb292b0580a3f49b78e9e8870e5c Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Mon, 5 Feb 2024 14:41:19 +0000 Subject: [PATCH 12/19] A few more ADR tweaks --- ...0003-windows-tfms-and-desktop-framework.md | 143 +++++++++++++++--- 1 file changed, 120 insertions(+), 23 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 5e90c93df..77900b43c 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -2,6 +2,26 @@ When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add many tens of megabytes to the deployable size of applications. +## In brief + +This is an extremely long document, because there is a lot of context. The primary purpose of this ADR is to fully describe all of the context, and to enable us to properly evaluate the design choice in the light of all relevant information. That makes it very long. If you aren't doing one of the following: + +* proposing a solution +* reviewing a proposed solution +* implementing a solution + +then this will be far more detail than is necessary. + +If you want to understand the problem, the preferred solution, and its consequences, you do not need to read everything. It is sufficient to read the following: + +* the [Overview](#overview) section +* the [preferred design option](#option-6-ui-framework-specific-packages-deprecating-the-systemreactive-versions) +* the [Decision](#decision) section +* the [Consequences](#consequences) section + + +## Overview + For example, this table shows the output sizes for a simple console app targeting `net8.0-windows10.0.19041` with various deployment models. The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. **Note** this problem only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later. | Deployment type | Size without Rx | Size with Rx | @@ -54,6 +74,17 @@ The basic problem is described at the start of this document, but we can charact That "or transitively" in the first parenthetical is easily overlooked, but is very important. Some developers have found themselves encountering this problem not because their applications use `System.Reactive` directly, but because they are using some library that depends on it. Many simple and initially plausible-looking solutions proposed to the problem this ADR addresses founder in cases where an application acquires a dependency to Rx.NET transitively, especially when it does so through multiple different references, and to different versions. +### A note on Target Framework Monikers + +In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These are often misunderstood. The problem this ADR describes can partly be blamed on a misuse of TFMs. + +TFMs can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). + +A version-specific TFM does not necessarily imply a minimum OS version requirement. A component must specify 10.0.19041 if it wants to attempt to use APIs introduced in that version of Windows, but the operative word here is "attempt". It's possible to detect failures and fall back to alternative behaviour when the API turns out to be unavailable. Thus, a component with a `net6.0-windows10.0.19041` TFM may well be able to run on Windows 10.0.18362. + +Although the OS version part of a TFM does not imply a minimum requirement, the .NET version number does. `net472` components can't run on .NET Framework 4.6. `net6.0` components can't run on `net5.0`. + +The final popular misunderstanding of TFMs is that when a .NET version goes out of support, its corresponding TFM remains supported. Often when people see a library with a TFM of, say, `net5.0`, they think the library is defunct because .NET 5.0 went out of support a long time ago. That's not necessarily true, because .NET 8.0, the latest version of .NET as I write this, supp.rts running components with a TFM of `net5.0`. In fact it supports TFMs all the way back to `netcoreapp1.0`! ### The road to the current problem @@ -434,19 +465,25 @@ For me this pretty much destroys the idea of a 'clean break' to enable breaking Some people think it's wrong of us to try to maintain high levels of backwards compatibility. I think that's debatable, but if you are going to take that position, then it demands the question: how much backwards compatibility is enough? How much can we break? -As it happens, I made clear from the start that total compatibility was not a goal: I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. +As it happens, I made clear from the start that total compatibility was not a goal. For example. I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. -TODO: this bit needs to be more coherent. I'm replying to things I'm not quoting... +Here's a discussion where this was raised: > [I don't understand why given this we are assuming that System.Reactive needs to somehow hold itself to a higher standard than the core framework itself.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617) -One reason is that we can't actually impose the split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be on one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to +This talks about how .NET itself has had some emphatically non-backwards-compatible moments. The transition from .NET Framework to .NET Core/.NET 5.0+ is one example, and the change from ASP.NET to ASP.NET Core is another. The argument is essentially: Microsoft is prepared to make earth-shattering non-backwards compatible changes, so why should Rx.NET try so hard? + +I think this argument misses that frameworks are fundamentally different from libraries. It's easier for a framework to break completely with the past than it is for a widely used general purpose library. -But as a library author I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects? Or any of the other library types which are nearly identical across both worlds. +I don't believe Rx.NET can impose the kind of split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be on one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to choose your framework carefully. Upgrading to newer versions of frameworks is a far more momentous decision than upgrading a library precisely because we expect frameworks to make breaking changes, particularly when they are essentially brand new things that make no pretence of being simply an update to their predecessors. + +As a developer writing a library, I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects. Or any of the other library types which are nearly identical across both worlds. So the question we need to ask is this: is Rx more like a framework or a common library feature? Are we more like ASP.NET Core, or `System.Linq`? I'd say the fact that we offer a `netstandard2.0` target points very much towards the latter. +As already discussed in [the section on the appetite for breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), when you look at breaking changes in .NET itself, although framework changes often are disruptive, changes to general-purpose library features typically try to preserve compatibility as much as is feasible even when the change is officially categorised as a binary breaking change. And it's like this because these kinds of library features are very often used quietly by other libraries as an implementation detail. They get imposed on an application from the bottom up, giving application authors much less control over what versions they use. So standards of compatibility need to be higher for this kind of library. +We're not actually holding Rx.NET to as a high a level as the .NET runtime class libraries in this regard. But we do need to be careful not to make changes that are absolutely certain to cause problems for large classes of users. #### Exploiting radical change as an opportunity @@ -786,23 +823,23 @@ This idea was one of those to emerge from some some discussion on an early proto In this design, we effectively introduce a whole new set of Rx packages that have absolutely no connection with any existing packages. It would in effect be a fork of Rx.NET that just happened to be produced by the maintainers of the current Rx.NET. -Applications wanting to use the latest Rx would use the new package. (`System.Rereactive`? `System.ReactiveEx`?) Existing code could carry on using `System.Reactive`. The theory is that we would be free to make breaking changes because code with references to the existing Rx would be entirely unaffected. This would free us to move the UI-specific features back out into separate packages. And it would also present a once-in-a-generation opportunity to fix other things, although that line of thinking risks turning this into the vehicle for everyone's pet feature. +Applications wanting to use the latest Rx would use the new package. (`System.Rereactive`? `System.ReactiveEx`?) Existing code could carry on using `System.Reactive`. The theory is that we would be free to make breaking changes because code with references to the existing Rx would be entirely unaffected. This would free us to move the UI-specific features back out into separate packages. It would also present a once-in-a-generation opportunity to fix other things, although that line of thinking risks turning this into the vehicle for everyone's pet feature. That's all moot, though, because we're not going to do it. This option was elegantly torpedoed by David Karnok. The [earlier section about how 'clean breaks' are a myth](#clean-starts-arent-due-to-extension-method-ambiguity) contains relevant quotes and links. I'll now explain why this is a terrible idea. -The defining feature of this idea is that there is no unification between "before" and "after" versions of Rx. Today, if a single application uses several libraries using several versions of Rx, they all end up using the same version—the build tooling basically picks the highest version number referenced, and everyone gets that. This is called _unification_. The effect of introducing completely new packages that are unrelated to the existing ones is that the build tools will consider these totally different libraries. If your app uses some libraries that use Rx 6 and some that use Rx 7, the build tools won't realise that these are references to different versions of the same thing. It would see a reference to `System.Reactive` v6.0.0, and a reference to `System.Newreactive` (or whatever) v7.0.0, and would just treat them as two separate libraries. +The defining feature of this idea is that there is no unification between "before" and "after" versions of Rx. Today, if a single application uses several libraries using several versions of Rx, they all end up using the same version—the build tooling basically picks the highest version number referenced, and everyone gets that. This is called _unification_. (For plug-ins, this unification happens at the scope of the plug-in, not the application.) The effect of introducing completely new packages that are unrelated to the existing ones is that the build tools will consider these totally different libraries. If your app uses some libraries that use Rx 6 and some that use RxNew 7, the build tools won't realise that these are references to different versions of what is logically the same thing. It would see a reference to `System.Reactive` v6.0.0, and a reference to `System.Newreactive` (or whatever) v7.0.0, and would just treat them as two separate libraries. -At that point, you've now got both Rx 6 and Rx 7 in your app. So what happens if you want to use Rx in your application code too? When you write this: +At that point, you've now got both Rx 6 and Rx 7 in your app. So what happens if you want to use Rx in your application code too? Suppose you were to write this: ```cs public IObservable FormatNumbers(IObservable xs) => xs.Select(x => x.ToString("G")); ``` -Which implementation of `Select` is it going to pick? The problem is that you've now got two definitions of `System.Reactive.Linq.Observable` from two different assemblies. Any file that has a `using System.Reactive.Linq` is going to have both the v6 and the v7 definition of `Observable` in scope. Which one is the compiler supposed to pick? +Which implementation of `Select` is the compiler going to pick? The problem is that you've now got two definitions of `System.Reactive.Linq.Observable` from two different assemblies. Any file that has a `using System.Reactive.Linq;` directive is going to have both the v6 and the v7 definition of `Observable` in scope. Which one is the compiler supposed to pick? -That call to `Select` is ambiguous. +That call to `Select` is ambiguous. That's the problem you create if you go down this so-called "clean break" path. It is technically possible to disambiguate identically-named classes with the rarely-used [`extern alias` feature](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/extern-alias). You can associate an alias with a NuGet reference like this: @@ -824,22 +861,40 @@ With that in place you can then be specific about which one you're using: extern alias RxV6; extern alias RxV7; -using RxV7::System; // for the Subscribe extension method -using RxV7::System.Reactive.Linq; // for Select +// This brings extension methods defined by Rx v7 in the System namespace into +// scope. That means that when we call the Subscribe extension method below, we +// get the Rx v7 version. +using RxV7::System; + +// This brings extension methods defined by Rx v7 in the System.Reactive.Linq +// namespace into scope. That means that when we call Select below, we get the +// Rx v7 version. +using RxV7::System.Reactive.Linq; +// Here we're indicating that we want the Rx v6 version of Observable. IObservable numbers = RxV6::System.Reactive.Linq.Observable.Range(0, 3); + +// This plugs the Range observable from Rx v6 into the Select operator from Rx +// v7. This is allowed, because all versions of Rx respect the IObservable +// contract. It might be suboptimal, because Rx operators recognized one +// another when you compose them, and can perform certain optimizations, and +// that won't happen here because the v7 library isn't going to recognize this +// Range operator as one of its own. But it will still work correctly. IObservable strings = numbers.Select(x => x.ToString()); +// This applies the Rx 7 version of the callback-drive Subscribe extension +// method because we specified the RxV7 prefix with the using directive +// above. strings.Subscribe(Console.WriteLine); ``` This uses the Rx v6 implementation of `Observable.Range`, but the two places where this uses extension methods (the call to `Select` and the call to the delegate-based `Subscribe` overload) will use the Rx v6 implementation, because that's what we specified in the `using` directives when importing the relevant namespaces. -So it's possible to make it work, but we consider this to be confusing and painful. The `extern alias` mechanism is really something you should only have to use if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. And it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. +So it's possible to make it work, but we consider this to be confusing and painful. You should only really need to use the `extern alias` mechanism if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. And it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. -The only way to avoid this would be to change not just the NuGet package names but also all the namespaces. That somewhat resembles what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretence of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. +The only way to avoid this would be to change not just the NuGet package names but also all the namespaces. (Even this still has the potential to cause confusion if any file ends up needing to us namespaces from both worlds.) That somewhat resembles what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretence of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. -You could argue that we've already done this. There's a whole new version of Rx at https://github.com/reaqtive/reaqtor that implements functionality not available in `System.Reactive`. (Most notably the ability to persist a subscription. In the ['reaqtive' implementation of Rx](https://reaqtive.net), operators that accumulate state over time, such as [`Aggregate`](https://introtorx.com/chapters/aggregation#aggregate), can migrate across machines, and be checkpointed, enabling reliable, persistent Rx subscriptions to run over the long term, potentially even for years.) The NuGet package names and the namespaces are completely different. There's no attempt to create any continuity here. +You could argue that we've already done this. There's a whole new version of Rx at https://github.com/reaqtive/reaqtor that implements functionality not available in `System.Reactive`. (Most notably the ability to persist a subscription. In this, the ['reaqtive' implementation of Rx](https://reaqtive.net), operators that accumulate state over time, such as [`Aggregate`](https://introtorx.com/chapters/aggregation#aggregate), can migrate across machines, and be checkpointed, enabling reliable, persistent Rx subscriptions to run over the long term, potentially even for years.) The NuGet package names and the namespaces are completely different. There's no attempt to create any continuity here. An upshot of this is that there is no straightforward way to migrate from `System.Reactive` to the [reaqtive Rx](https://reaqtive.net). (The Azure SDK revamp has the same characteristic. You can't just change your NuGet package references: you need to change your code to use the newer libraries, because lots of things are just different.) @@ -849,48 +904,90 @@ So for all these reasons, we are rejecting this design option. #### Option 3: new main Rx package, demoting `System.Reactive` to a facade -Discussed at https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557014 with a prototype on https://github.com/dotnet/reactive/pull/2034 +In this design, we introduce a new NuGet package that supersedes `System.Reactive` (much as `System.Reactive` superseded `Rx-Main` back in Rx v4.0). This new package would essentially be `System.Reactive` with all the UI-specific functionality removed. We would add new per-UI-framework packages and the UI-specific functionality would move into these. We would continue to build and publish `System.Reactive`, which would become a facade, containing nothing but type forwarders. `System.Reactive` would continue to support the same targets. Its use would be deprecated, but we would likely continue to support it for a very long time—we still build the type facades that were added to enable pre-great-unification code to continue to use the older package structure. + +The main appeal of this option was that it offered a fairly quick fix for the main problem in this ADR. We could immediately publish a package that made Rx's core functionality available in a way that would definitely not add any unwanted UI framework references. Before we discovered the [workaround](#the-workaround), we needed a solution urgently, because there didn't seem to be any way to work around the problems that this ADR addresses. This seemed like the fastest way to get to that point. However, now that we have a workaround, this removes the time pressure. +The biggest downside of this approach is that it's yet another change in how to use Rx.NET, so this conflicts with the constraint of [minimizing confusion](#minimize-confusion-as-far-as-possible). If it had been the only way to unblock projects that wanted to use Rx.NET on `net6.0-windows.10.*` targets without bringing in WPF and Windows Forms, this might have been an acceptable downside. But that availability of a workaround effectively removes this option's unique upside, at which point the downside looks less acceptable. + +This was discussed at https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557014 and we built a prototype on https://github.com/dotnet/reactive/pull/2034 (back before we knew about the workaround). #### Option 4: `System.Reactive` remains the primary package and becomes a facade -We could maintain `System.Reactive` as the face of Rx, but turn it into a facade, with all the real bits being elsewhere. This would give people the option to depend on, say, `System.Reactive.Common` or whatever, to be sure of avoiding any UI dependencies. However, this might not help with transitive dependencies. +This is almost exactly the same as [option 3](#option-3-new-main-rx-package-demoting-systemreactive-to-a-facade), but with one difference: `System.Reactive` continues to be the face of Rx. + +Just like with option 3, `System.Reactive` would be a facade, with all the real bits being elsewhere. It's just that we don't have to tell people that sorry, we've changed the main Rx package. And we would most likely want to deprecate the UI-framework-specific types in this model so that in the long run they could be removed, and we could end up with a long-term end state in which these these types are finally removed from `System.Reactive`. + +This might also have been able to provide a quick fix just like option 3, because there would be some `System.Reactive.Common` (or whatever) containing the code with no UI dependencies, and applications that want to can use that to ensure they don't acquire unwanted frameworks dependencies. However, this might not help with transitive dependencies. + +One unresolved issue with this option (and which is not unique to this option) is how to deal with the fact that Rx v6 provides UWP-specific thread pool handling by defining extra methods on `ThreadPoolScheduler`. What should the UWP version `System.Reactive` forward the `ThreadPoolScheduler` to? If we eventually want to reach a state where `System.Reactive` has no UI-framework-specific surface area, what do we do with `ThreadPoolScheduler`? We can't just obsolete it and remove it, because the core non-UI-specific Rx.NET flavours also have a `ThreadPoolScheduler` and we don't want to remove it. #### Option 5: gut the UI-specific types so they are hollowed out shells David Karnok [suggested we turn `DispatcherScheduler` into a delegator and move the real implementation elsewhere](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557205). +Unless I've missed something, I don't think this helps us at all because all of these types have UI-specific types as part of their public API. So even if you remove all implementation details, you still end up with dependencies on the relevant frameworks. -#### Option 6: UI-framework specific packages, deprecating their +#### Option 6: UI-framework specific packages, deprecating the `System.Reactive` versions +In this option, `System.Reactive` would continue to be the package through which core Rx functionality is used, but we would add new NuGet packages for each UI framework we wish to support. All of the UI-framework-specific functionality that's in `System.Reactive` would be copied into these new packages. The types would remain in the relevant targets of `System.Reactive` too, but would be marked as `[Obsolete]`, with the long term plan being to remove then [as soon as is acceptable](#minimum-acceptable-path-for-breaking-changes). +It's not clear yet whether we would need to define brand new types in these new frameworks, or use type forwarders. I'm not sure if it's possible to deprecate use of a type through a type forwarder, but not have it be deprecated if you use it directly. I _think_ you can't but need to check. -#### Um +This also has a similar unresolved question around `ThreadPoolScheduler` was was discussed in [option 4](#option-4-systemreactive-remains-the-primary-package-and-becomes-a-facade). -https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7628930 ### Other options less seriously considered -### Change everything +There were a few other ideas that we rejected fairly quickly. + +#### Build system hackery + +As discussed in https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7628930: + +>It would be technically possible for us to build a NuGet package for System.Reactive which contains a net8.0 target and which did NOT contain a net8.0-windows... target, but where the DLL in the net8.0 folder contained a DLL that did in fact have references to WPF and Windows Forms assemblies. And if you do this, then as long as you never attempt to use any of the WPF/WinForms features, you'd never know those references are there. The attraction of this is that if you do happen to be running in an application that has specified, say, true, then you will be able to use the WPF-specific features in this System.Reactive. + + +While this might work, employing hacks that deliberately subvert the way the build system works are exactly the sort of thing that causes trouble a few years down the line. So we don't want to go down that path. +#### Make `windows10.0.19041` a dead end by adding a later target -Another idea: could we introduce a later Windows-specific TFM, so that use of windows10.0.19041 becomes a sort of dead end? +It was suggested that we could introduce a later Windows-specific TFM, so that use of windows10.0.19041 becomes a sort of dead end. The idea was that the 19041 version would continue to be available for those that need it but the default would be that you don't get WPF and Windows Forms. + +The biggest problem with this is that any projects already using Rx.NET that have a later Windows-specific TFM will be broken by this, and it's not clear how they'd unbreak themselves. ## Decision +We will go with [option 6](#option-6-ui-framework-specific-packages-deprecating-the-systemreactive-versions): + +* `System.Reactive` will remain as the main package for using Rx.NET +* we will add a new NuGet packages for each UI framework, and put UI-framework-specific support in these +* the UI-frameworks-specific types in `System.Reactive` will be marked as `[Obsolete]` with the [very long-term](#minimum-acceptable-path-for-breaking-changes) intention of removing them entirely, likely not before 2030 + + As it says in [the announcement for the first Rx release](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5): > it didn't make much sense in a layer map to have those dependencies for something as generic as an event processing library. As a result, we refactored out the UI dependencies in System.Reactive.Windows.Forms.dll (for Windows Forms) and System.Reactive.Windows.Threading.dll (for WPF and Silverlight). +The benefit of hindsight makes the wisdom of this evident. There were some good reasons for the _great unification_ but many of the factors that made it look like the best option at the time no longer apply, and changes in the .NET world that occurred after the _great unification_ have turned it into a problematic decision. + +So it is time to de-unify the UI-framework-specific parts. + + ## Consequences +The main relevant consequences are: + +* This minimizes disruption: `System.Reactive` continues to be the main way to use Rx.NET +* Applications encountering the problem described in this ADR will need to use [the workaround](#the-workaround) for the foreseeable future +* The eventual end state (ca. 2030) is that `System.Reactive` will be free from UI-framework-specific types +* We'll be able to take UWP out of the majority of the source tree in 2-3 years +* All UI-frameworks-specific support will be on an equal footing, enabling MAUI, Blazor, and Avalonia to be supported in exactly the same way as Windows Forms and WPF (i.e., through separate NuGet packages) -Spare: -In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). From 1fc3e4f11cf910f490a38e3a52973d4c45182b92 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 7 Feb 2024 07:13:33 +0000 Subject: [PATCH 13/19] Update type forwarder details and package names After some experiments, we can rule out certain type forwarder based approaches --- .../0003-windows-tfms-and-desktop-framework.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 77900b43c..fcb84db9b 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -602,11 +602,11 @@ The JIT compiler will then inspect the `System.Reactive` assembly and discover t And that's why we can't just remove types from `System.Reactive`. -What we can do is replace them with a type forwarder. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, we can do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." +Type forwarders seem like they might offer a solution, but unfortunately not. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, in theory we do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." -Doesn't a type forwarder solve the problem? Not really, because if the `System.Reactive` assembly contains type forwarders to some proposed `System.Reactive.Wpf` assembly, the .NET SDK will require the resulting `System.Reactive` NuGet package to have a dependency on `System.Reactive.Wpf`. +Doesn't that solve the problem? Not really, because if the `System.Reactive` assembly contains type forwarders to some proposed `System.Reactive.Wpf` assembly, the .NET SDK will require the resulting `System.Reactive` NuGet package to have a dependency on `System.Reactive.Wpf`. -And that gets us back to square one: if taking a dependency on `System.Reactive` causes you to acquire a transitive dependency on `System.Reactive.Wpf`, that means using Rx automatically opts you into use WPF whether you want it or not. +And that gets us back to square one: if taking a dependency on `System.Reactive` causes you to acquire a transitive dependency on `System.Reactive.Wpf`, that means using Rx automatically opts you into use WPF whether you want it or not. (Also, since the new component containing WPF-specific Rx.NET features would need to depend on `System.Reactive` for its `IScheduler` definition, attempting to use a type forwarder in this way would create a circular reference. So in practice it wouldn't be possible without going back to the older design in which such things were defined in `System.Reactive.Interfaces`, and we don't want to do that.) It would be technically possible to meddle with the build system's normal behaviour in order to produce a `System.Reactive` assembly with a suitable type forwarder, but for the resulting NuGet package not to have the corresponding dependency. However, this is unsupported, and is likely to cause a lot of confusion for people who actually do want the WPF functionality, because adding a reference to just `System.Reactive` (which has been all that has been required for Rx v4 through v6) would still enable code using WPF features to compile when upgrading to this hypothetical form of v7, but it would result in runtime errors due to the `System.Reactive.Wpf` assembly not being found. So this is not an acceptable workaround. @@ -934,7 +934,7 @@ Unless I've missed something, I don't think this helps us at all because all of In this option, `System.Reactive` would continue to be the package through which core Rx functionality is used, but we would add new NuGet packages for each UI framework we wish to support. All of the UI-framework-specific functionality that's in `System.Reactive` would be copied into these new packages. The types would remain in the relevant targets of `System.Reactive` too, but would be marked as `[Obsolete]`, with the long term plan being to remove then [as soon as is acceptable](#minimum-acceptable-path-for-breaking-changes). -It's not clear yet whether we would need to define brand new types in these new frameworks, or use type forwarders. I'm not sure if it's possible to deprecate use of a type through a type forwarder, but not have it be deprecated if you use it directly. I _think_ you can't but need to check. +We would need to define brand new types in these new frameworks to replace the ones that will ultimately be removed. We can't use type forwarders because it's not possible to deprecate use of a type through a type forwarder, but not have it be deprecated if you use it directly. This also has a similar unresolved question around `ThreadPoolScheduler` was was discussed in [option 4](#option-4-systemreactive-remains-the-primary-package-and-becomes-a-facade). @@ -986,8 +986,17 @@ The main relevant consequences are: * This minimizes disruption: `System.Reactive` continues to be the main way to use Rx.NET * Applications encountering the problem described in this ADR will need to use [the workaround](#the-workaround) for the foreseeable future * The eventual end state (ca. 2030) is that `System.Reactive` will be free from UI-framework-specific types +* New assemblies, one for each UI framework we support (Windows Forms, WPF, and UWP) will be added containing replacements for the UI-framework-specific types in `System.Reactive` +* We will deprecate the existing UI-framework-specific types so that developers will be notified that they are on their way out +* We won't be able to add type forwarders from the old types in `System.Reactive` to their replacements because you can't deprecate a type forwarder * We'll be able to take UWP out of the majority of the source tree in 2-3 years * All UI-frameworks-specific support will be on an equal footing, enabling MAUI, Blazor, and Avalonia to be supported in exactly the same way as Windows Forms and WPF (i.e., through separate NuGet packages) +Although there are some existing old packages that used to contain some UI-framework-specific types, they are now backwards-compatibility facades. However, these don't separate out all the frameworks completely: Windows Forms has its own one, `System.Reactive.Windows.Forms`, but the WPF support lives in a more confusingly named `System.Reactive.Windows.Threading`, and that same component also defines the UWP integration. The WPF support is available only in the `net472` build, and the UWP support is only in the `uap10.0.18362` build, so this one package contains completely different types in its two target frameworks! And just to confuse matters, there's also a `System.Reactive.WindowsRuntime` package that targets only `uap10.0.18362`, and which contains some different UWP types. +Our current plan is to leave these facade components untouched, since the split of types is a bit confusing, the naming is a bit inconsistent, and their purpose for years has been to provide backwards compatibility for Rx v3 era apps. We will introduce new components each with a clear purpose: + +* `System.Reactive.Integration.WindowsForms` +* `System.Reactive.Integration.WPF` +* `System.Reactive.Integration.UWP` From 1ddaac41c8d97c93b9a9ade8661aa15177d8ccb0 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Tue, 20 Feb 2024 10:30:31 +0000 Subject: [PATCH 14/19] Update ADR after re-read after prototype Got as far the great unification section --- ...0003-windows-tfms-and-desktop-framework.md | 75 ++++++++++++------- 1 file changed, 50 insertions(+), 25 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index fcb84db9b..e5cc4d6a0 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -1,6 +1,6 @@ # Windows TFMs and Desktop Framework Dependencies -When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add many tens of megabytes to the deployable size of applications. +When a .NET project that targets Windows takes a dependency on `System.Reactive`, there are circumstances in which this causes unwanted dependencies on the WPF and Windows Forms frameworks. This can add many tens of megabytes to the deployable size of applications. This document describes the workaround available for Rx 6.0, and how Rx 7.0 and future versions will fix this problem in the long term. ## In brief @@ -22,7 +22,7 @@ If you want to understand the problem, the preferred solution, and its consequen ## Overview -For example, this table shows the output sizes for a simple console app targeting `net8.0-windows10.0.19041` with various deployment models. The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. **Note** this problem only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later. +This table shows the output sizes for a simple console app targeting `net8.0-windows10.0.19041` with various deployment models. (The problem this ADR addresses only afflicts applications with a Windows-specific TFM, and only those specifying a version of `10.0.19041` or later.) The console app calls some WinRT APIs, hence the need for the `-windows` TFM, but does not use any UI framework. The final column shows the impact of adding a reference to Rx and a single line of code using Rx. | Deployment type | Size without Rx | Size with Rx | |--|--|--| @@ -31,19 +31,28 @@ For example, this table shows the output sizes for a simple console app targetin | Self-contained trimmed | 18.3MB | 65.7MB | | Native AOT | 5.9MB | 17.4MB | -The worst case, self-contained deployment, is also a widely used case for applications that need to use a `-windows` TFM. It roughly doubles the size of this application, adding over 90MB! This issue has caused some projects to abandon Rx entirely. +The worst case, self-contained deployment, is widely used by applications that need a `-windows` TFM. It roughly doubles the size of this application, adding over 90MB! With trimming, the absolute impact is smaller (Rx adds 47.4MB) but the relative increase is even larger, at a factor of 3.6. This issue has caused some projects to abandon Rx entirely. -For example, after [Avalonia ran into this problem](https://github.com/AvaloniaUI/Avalonia/issues/9549), they [ removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of https://github.com/dotnet/reactive/issues/1461 you'll see some people talking about not being able to use Rx because of this problem. +After [Avalonia ran into this problem](https://github.com/AvaloniaUI/Avalonia/issues/9549), they [ removed all use of Rx.NET](https://github.com/AvaloniaUI/Avalonia/pull/9749) in January 2023. In the discussion of [#1461](https://github.com/dotnet/reactive/issues/1461) you'll see some people talking about not being able to use Rx because of this problem. -Recently, a [workaround](#the-workaround) has been discovered, which reduces the sizes to 22.5MB, 92.5MB, 18.3MB, and 6.2MB respectively. (So the impact of adding Rx is reduced to 1.6MB, 1.6MB, unmeasureably small, and 300KB respectively for the four deployment models above.) +Recently, a [workaround](#the-workaround) has been discovered, which reduces the sizes to the values shown in this table: -The view of the Rx .NET maintainers is that projects using Rx should not be forced into this situation. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. +| Deployment type | Size without Rx | Size with Rx using workaround | +|--|--|--| +| Framework-dependent | 20.8MB | 22.5MB | +| Self-contained | 90.8MB | **92MB** | +| Self-contained trimmed | 18.3MB | 18.3MB | +| Native AOT | 5.9MB | 6.2MB | + +So the workaround reduces the impact of adding Rx to 1.6MB, 1.6MB, unmeasureably small, and 300KB respectively for the four deployment models shown. + +The view of the Rx .NET maintainers is that projects using Rx should not be forced into a situation where their deployments become unreasonably large. There are a lot of subtleties and technical complexity here, but the bottom line is that we want Rx to be an attractive choice. The discovery of a [workaround](#the-workaround) came fairly late in the day, some time after various projects had decided to stop using Rx.NET. That is important context for understanding earlier discussion of this topic. Back when [endjin](https://endjin.com) took over maintenance and development of Rx .NET at the start of 2023, it was believed that there was no workaround, so our plan was that Rx 7.0 would address this problem, possibly making radical changes (e.g., introducing a new 'main' Rx package, with `System.Reactive` being the sad casualty of an unfortunate technical decision made half a decade ago). But now that a workaround has been identified, the pressure to make changes soon has been removed. It seems that Rx 6.0 can be used in a way that doesn't encounter these problems, so we now think that a less radical, more gradual longer-term plan is a better bet. We can deprecate the parts of the library that caused this problem and introduce replacements in other components, with a long term plan of eventually removing them from `System.Reactive`, at which point the workaround would no longer be required. The process of deprecation could begin now, but it would likely be many years before we reach the end state. -This document explains the [root causes of the problem](#the-road-to-the-current-problem), the current [workaround](#the-workaround), [the community feedback we've received](#community-input), the [constraints that any solution will have to satisfy](#constraints), and [the eventual desired state of Rx .NET, and the path that will get us there](#decision). +This document explains the [root causes of the problem](#the-road-to-the-current-problem), the current [workaround](#the-workaround), [the community feedback we've received](#community-input), and the [constraints that any solution will have to satisfy](#constraints). It then describes [the eventual desired state of Rx .NET, and the path that will get us there](#decision). ## Status @@ -58,7 +67,9 @@ Draft ## Context -To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem), and the [constraints that any solution must fulfil](#constraints). The proposed [workaround](#the-workaround) needs to be understood in detail. We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. +To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem), and the [constraints that any solution must fulfil](#constraints). The [workaround](#the-workaround) needs to be understood in detail because it will be the interim solution for many years. + +We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. The following sections address all of this before moving onto a [decision](#decision). @@ -68,7 +79,7 @@ The basic problem is described at the start of this document, but we can charact > An application that references the [`System.Reactive` NuGet package](https://www.nuget.org/packages/System.Reactive) (directly or transitively) and which has a Windows-specific target specifying a version of `10.0.19041` or later will acquire a dependency on the [.NET Windows Desktop Runtime](https://github.com/dotnet/windowsdesktop). > -> This occurs because in projects that have a direct reference to the package, the [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) properties will have been set to `true`. If the project is an application, that will cause the dependency on the Desktop Runtime. If the project builds a NuGet package, it will cause the package to be built in a way that indicates that the Desktop Runtime is required, which is how indirect references to `System.Reactive` also cause this issue. +> This occurs because the `System.Reactive` package's `nuspec` file's `frameworkReferences` section states that this package's `net6.0-windows10.0.19041` target has a `frameworkReference` to `Microsoft.WindowsDesktop.App`. Framework references are transitive, which is why it's not just direct references to `System.Reactive` that cause this issue. > > An unwanted dependency on the .NET Windows Desktop Runtime causes a problem for self-contained deployment (and, by extension, Native AOT) because it means those deployments end up including complete copies of the Windows Forms and WPF frameworks. This can add many tens of megabytes to the application in its final deployable form. This is especially frustrating for applications that don't use either WPF or Windows Forms. @@ -78,13 +89,15 @@ That "or transitively" in the first parenthetical is easily overlooked, but is v In .NET, components and applications indicate the environments they can run on with a Target Framework Moniker (TFM). These are often misunderstood. The problem this ADR describes can partly be blamed on a misuse of TFMs. -TFMs can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, and it has indicated a particular Windows API surface area that it was built for. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). +TFMs can be very broad. A component with a TFM of `netstandard2.0` can run on any .NET runtime that supports .NET Standard 2.0 (e.g., .NET 8.0, or .NET Framework 4.7.2), and does not care which operating system it runs on. But TFMs can be a good deal more specific. If a component has a TFM of `net6.0-windows10.0.19041`, it requires .NET 6.0 or later (so it won't run on any version of .NET Framework) and will run only on Windows. Moreover, it has indicated that it was built for a particular Windows API surface area. That `10.0.19041` is an SDK version number but it corresponds to the May 2020 update to Windows 10 (also known as version 2004, or 20H1). A version-specific TFM does not necessarily imply a minimum OS version requirement. A component must specify 10.0.19041 if it wants to attempt to use APIs introduced in that version of Windows, but the operative word here is "attempt". It's possible to detect failures and fall back to alternative behaviour when the API turns out to be unavailable. Thus, a component with a `net6.0-windows10.0.19041` TFM may well be able to run on Windows 10.0.18362. +All OS-specific TFMs are also version-specific. If you don't specify the version, the .NET SDK picks one for you. For example, in all .NET SDKs published to date since 5.0 (the latest being 8.0.200 at the time of writing this), `net6.0-windows` is equivalent to `net6.0-windows7`. + Although the OS version part of a TFM does not imply a minimum requirement, the .NET version number does. `net472` components can't run on .NET Framework 4.6. `net6.0` components can't run on `net5.0`. -The final popular misunderstanding of TFMs is that when a .NET version goes out of support, its corresponding TFM remains supported. Often when people see a library with a TFM of, say, `net5.0`, they think the library is defunct because .NET 5.0 went out of support a long time ago. That's not necessarily true, because .NET 8.0, the latest version of .NET as I write this, supp.rts running components with a TFM of `net5.0`. In fact it supports TFMs all the way back to `netcoreapp1.0`! +The final popularly misunderstood feature of TFMs is that when a .NET version goes out of support, its corresponding TFM remains supported. Often when people see a library with a TFM of, say, `net5.0`, they think the library is defunct because .NET 5.0 went out of support a long time ago. That's not necessarily true, because .NET 8.0, the latest version of .NET as I write this, supports running components with a TFM of `net5.0`. In fact it supports TFMs all the way back to `netcoreapp1.0`! ### The road to the current problem @@ -94,12 +107,13 @@ This problem arose from a series of changes made about half a decade ago that we 2. the subtle problems that could occur when plug-ins use Rx 3. the [_great unification_](https://github.com/dotnet/reactive/issues/199) in Rx 4.0 that solved the first two problems 4. changes in .NET Core 3.0 which, in combination with the _great unification_, caused the problem that this ADR aims to solve +5. the regression that re-introduced the plug-in problem in Rx 5.0, and why it might not matter as much as it used to #### Rx's history of confusing packaging The first public previews of Rx appeared back in 2009 before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary redistributable files onto target machines as part of your application's installation or deployment process. By the time [the first supported Rx release shipped in June 2011](https://web.archive.org/web/20110810091849/http://www.microsoft.com/download/en/details.aspx?id=26649), NuGet did exist, but it was early days, so for quite a while Rx had [two official distribution channels: NuGet and an installable SDK](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5). -There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: +There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled as part of the OS. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: * The scheduler support was specialized to work as well as possible on each distinct target * Each platform had a different UI framework (or frameworks) available, so Rx's UI framework integration was different for each target @@ -110,7 +124,7 @@ This meant that it would be possible, in principle, to write a library that depe This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. -An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea behind this was that this would be a stable component that didn't need frequent releases because the expectation was that the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. This defeated the entire purpose of having a separate component for the core interfaces. +An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea was that this would be a stable component that didn't need frequent releases, because the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. This defeated the entire purpose of having a separate component for the core interfaces. (In fact things were a little weirder because some of the versions of .NET supported by Rx 1.0 defined the core `IObservable` and `IObserver` interfaces in the runtime class libraries but some did not. These interfaces were not present in .NET 3.5, for example, which Rx 1.0 supported. So Rx had to bring its own definition of these for some platforms. You might expect these to live in `System.Reactive.Interfaces` but they did not, because Microsoft wanted that package to be the same on all platforms. So on platforms where `IObversable/er` were not built in, there was yet another DLL in the mix, further adding to the confusion around exactly what assemblies you needed to ship with your app if you wanted to use Rx.) @@ -122,9 +136,9 @@ In summary, you couldn't simply add a reference and start using Rx. Understandin The NuGet distribution of Rx introduced a simplifying concept in v2.2: Rx was still fragmented across multiple components at this point, but the simplifying move was to define NuGet metapackages enabling you to use just a single package reference for basic Rx usage. For example, a single reference to [`Rx-Main` v2.2.0](https://www.nuget.org/packages/Rx-Main/2.2.0) would give you everything you needed to use Rx. There were additional metapackages appropriate for using specific UI frameworks with Rx. For the first time, now you could just add a single reference and immediately start using Rx. -Because Rx has always supported many different runtimes, each Rx.NET NuGet package contained several different builds of its component. For quite a long time, there were different copies of Rx for different versions of .NET Framework. In Rx 2.2.0, there was one targeting .NET Framework 4.0, and another targeting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. And the idea is that the .NET SDK will work out at build time which one to use based on the runtime your application targets. +Because Rx has always supported many different runtimes, each Rx.NET NuGet package contained several different builds of its component. For quite a long time, there were different copies of Rx for different versions of .NET Framework. For example, Rx 2.2.0 had one targeting .NET Framework 4.0, and another targeting .NET Framework 4.5. NuGet can cope with this—you just end up with `net40` and `net45` subfolders under `lib`. The .NET SDK works out at build time which one to use based on the runtime your application targets. -So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (Rx 2.2's [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively.) And then each of those packages contained multiple versions of what was, conceptually speaking, the same assembly (but with various technical differences due to differences between the target platforms). For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders +So there were effectively two dimensions of fragmentation. First, behind each metapackage there were multiple NuGet packages. (Rx 2.2's [`Rx-Main` metapackage](https://www.nuget.org/packages/Rx-Main/2.2.0#dependencies-body-tab) depends on [`Rx-Core`](https://www.nuget.org/packages/Rx-Core/2.2.0), [`Rx-Interfaces`](https://www.nuget.org/packages/Rx-Interfaces/2.2.0), [`Rx-Linq`](https://www.nuget.org/packages/Rx-Linq/2.2.0) and [`Rx-PlatformServices`](https://www.nuget.org/packages/Rx-PlatformServices/2.2.0), for example. And just to add to the confusion, the package names aren't the same as the names of the assemblies they contain. These four packages provide `System.Reactive.Core.dll`, `System.Reactive.Interfaces.dll`, `System.Reactive.Linq.dll`, and `System.Reactive.PlatformServices.dll` respectively.) And then each of those packages contained multiple versions of what was, conceptually speaking, the same assembly (but with various technical differences due to differences between the target platforms). For example, if you [look inside `Rx-Core` 2.2.0](https://nuget.info/packages/Rx-Core/2.2.0) you'll see its `lib` folder contains 8 folders, each of which contains a slightly different build of `System.Reactive.Core.dll`. ![A folder view showing a 'lib' folder, with 8 subfolders: net40, net45, portable-net40+sl5+win8+wp8, portalble-windows8+net45+wp8, sl5, windows8, windowsphone71, and windowsphone8](./images/0003-Rx-Core-2.2.0-contents.png) @@ -134,11 +148,11 @@ Each of these subfolders of each NuGet package's `lib` folder contains a version #### Plug-in problems -This fragmentation caused [a problem with plug-in systems](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it, but any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. +This fragmentation caused [a problem with plug-in systems (#97)](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it, and it was common for any one user to use a lot of plug-ins, but any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. -If one plug-in was written to use Rx.NET and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. +If one plug-in was written to use Rx.NET 2.2.0 and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. -Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. And that caused a problem. +Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. That caused a problem. Here's what would happen. Let's say a we have two plug-ins, `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45`. Both were built with a reference to `Rx-Main` 2.2.0. That means that if we were to look at how these plug-ins looked on disk once they had been installed in the target application, we'd see something like this: @@ -158,17 +172,19 @@ Here's what would happen. Let's say a we have two plug-ins, `PlugInOneBuiltFor40 (Visual Studio uses a more complex folder layout in reality, but that's not significant. _Any_ plug-in host will have the same issue.) -The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one built for .NET 4.5. Crucially, _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are the same in this case.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. +The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one built for .NET 4.5. Crucially, _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are for Rx 2.2.0.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, this already-loaded copy will be the `net40` version, but the assembly resolver doesn't know that it's different from what `PlugInTwoBuildFor45` wants. -The .NET Framework assembly resolver assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. This would result in `MissingMethodException` failures if that second component tried to use features that were present in the `net45` build but not the `net40` build. +The .NET Framework assembly resolver assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. As it happens, these have the same public API surface area, so in this particular case we wouldn't get `TypeLoadException` or `MissingMethodException` failures. But there is a behavioural difference. It's quite an obscure one, relating to whether an [`OperationCanceledException`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception) reports the correct [`CancellationToken`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception.cancellationtoken) when you use Rx's `ToTask` or `ForEachAsync`. (As far as I can tell, this is the only respect in which the `net40` and `net45` versions of Rx were different at that time.) If `PlugInTwoBuiltFor45` depended on the correct behavior here, that would be a problem because it would end up using the `net40` version, and it was not possible to implement this correctly on .NET Framework 4.0. -This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that when a .NET 4.5 application uses a .NET 4.0 component, that component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45` would end up using the `net45` build of the Rx components. And that would work just fine.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the whole point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. +Although this was an extremely specific problem, the bigger problem was that if future versions of Rx ended up with greater divergences on different .NET Framework versions, plug-ins wanting newer versions could well end up encountering `TypeLoadException` or `MissingMethodException` failures as a result of not getting the version they require. + +This only afflicts plug-in systems because those defeat an assumption that is normally valid. Normally we can assume that for any single application, the build process for that application will have an opportunity to look at all of the components that make up the application, including all transitive dependencies, and to detect situations like this. In some cases, it might be possible to use rules to resolve it automatically. (You might have a rule saying that when a .NET 4.5 application uses a .NET 4.0 component, that component can be given the .NET 4.5 version of one of its dependencies. In this case it would mean both `PlugInOneBuiltFor40` and `PlugInTwoBuiltFor45` would end up using the `net45` build of the Rx components. And that would work just fine, assuming `PlugInOneBuiltFor40` didn't actually depend on the slightly deficient `CancellationToken` handling in the `net40` build.) Or it might detect a conflict that cannot be safely resolved automatically. But the problem with plug-in systems is that the exact set of .NET components in use does not become apparent until runtime, and will change each time you add a new plug-in. It's not possible to know what the entire application looks like when you build the application because the whole point of a plug-in system is that it makes it possible to add new components to the application long after the application has shipped. It's worth noting at this point that the problem I've just described doesn't need to affect applications using .NET (as opposed to .NET Framework). Back when the thing we now call ".NET" was still called .NET Core, .NET Core added the [`AssemblyLoadContext` type](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext) which makes it possible for different plug-ins each to load their own copies of assemblies, even when they have exactly the same full name as assemblies loaded by other plug-ins. But that feature didn't exist back in the Rx 2.0 or 3.0 days (and still doesn't exist in .NET Framework even today). -[Rx 3.1](https://github.com/dotnet/reactive/releases/tag/v3.1.0) attempted to solve the plug-in problem by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have expected this would use the fourth part that .NET assembly versions have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: +[Rx 3.1](https://github.com/dotnet/reactive/releases/tag/v3.1.0) attempted to solve the plug-in problem by using [slightly different version numbers for the same 'logical' component on each supported target](https://github.com/dotnet/reactive/pull/212). You might have expected this to use the fourth part that .NET assembly versions have, with the first 3 matching the 3 parts that NuGet packages have, but in fact they chose to use the 3rd part, leaving the 4th part as 0. You can see the [code that sets the version number differently based on the target in GitHub](https://github.com/dotnet/reactive/blob/e0b6af3e204feb8aa13841a8a873d78ae6c43467/Rx.NET/Source/GlobalAssemblyVersion.cs) but I've reproduced it here: ```cs #if NETSTANDARD1_0 || WP8 @@ -192,15 +208,15 @@ It's worth noting at this point that the problem I've just described doesn't nee By this time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated version of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. -Again, it's worth thinking briefly about .NET Core/modern .NET at this point to see how things are different there. This newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. It typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue, but it's helpful to bear in mind that a basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) +This change predates .NET Core/modern .NET, and this newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. A basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) Fortunately, it typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue on the newer runtimes that have this different versioning behavior. -Unfortunately, this change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.1000.0 versions do not). +Unfortunately, Rx 3.1's change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.1000.0 versions do not). As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems, but if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. For example, if your application targetted .NET 4.6.2, and you were using two libraries that both depend on Rx 3.1.1, but one of those libraries offers only a `net45` target and the other offers only a `net461` target, they now disagree on the version of Rx they want. The first wants Rx components with version numbers of `3.0.1000.0`, while the second wants components with version numbers of `3.0.4000.0`. This could result in assembly version conflict reports when building the application. You might be able to solve this with assembly binding redirects, and you might even be able to get the build tools to generate those for you. But there were scenarios where the tooling couldn't work out what to do, and developers were left trying to understand all the history described to date in order to work out how to unpick the mess. And this also relies on the same "we can resolve it all when we build the application" assumption that is undermined in plug-in scenarios, so this could _still_ cause problems for plug-ins! -The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different runtimes. These might be a mutually compatible combination (e.g., if you use components targeting `net40`, `net45`, and `net46`, they can all run happily on .NET 4.6.2) but if any of them used Rx you now have a problem because they all want different versions of Rx. +The basic problem here is that when building any single deployable target (either an application or a plug-in) you might be using a mixture of components that target several different runtimes. These might be a mutually compatible combination (e.g., if you use components targeting `net40`, `net45`, and `net46`, they can all run happily on .NET 4.6.2) but if any of them used Rx you might now have a problem because they could all want different versions of Rx. #### Rx 4.0's great unification @@ -341,6 +357,10 @@ In my view, the best solution to this whole problem would have been for all of t This is easy to say with hindsight of course, particularly since there are now many different options for building client-side UI with .NET. In a world where Avalonia, MAUI, Windows Forms, WPF, and WinUI are all possibilities for a .NET application, the idea that `System.Reactive` should do everything looks obviously unsustainable, in a way that it didn't back in the Rx 4.0 days. +### The return of the plug-in problems in Rx 5.0 + +TBD. + ### The workaround If your application has encountered [the problem](#the-problem), you can add this to the `csproj`: @@ -755,6 +775,11 @@ Another consideration is that modern NuGet tooling is better than it was in 2016 When it comes to .NET 6.0 and later, these problems should a non-issue because better plug-in architectures exist thanks to `AssemblyLoadContext`. + +#### Threading constraints are back thanks to WASM + +TBD. + ### Types in `System.Reactive` that are partially or completely UI-framework-specific Before we go on to consider various design options, it's useful to understand exactly which bits of the current Rx.NET API are likely to need to change. Thanks to the _great unification_, `System.Reactive` now has a slightly different API surface area on different platforms. There are some types available only on specific targets. There is one type that is available on all targets but has a slightly different public API on one target. And there are some types with the same API surface area on all targets but with differences in behavior. This section describes the afflicted API surface area. From 11111f0c9c063fb7e0af7ff36c0cdad0b3637c65 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 21 Feb 2024 12:07:44 +0000 Subject: [PATCH 15/19] Update package ADR more in light of prototype Updated as far as "The workaround" section --- ...0003-windows-tfms-and-desktop-framework.md | 64 +++++++++++++------ 1 file changed, 46 insertions(+), 18 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index e5cc4d6a0..50b7002cd 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -220,9 +220,9 @@ The basic problem here is that when building any single deployable target (eithe #### Rx 4.0's great unification -[Rx 4.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.0.0) tried a different approach: have just one Rx package, `System.Reactive`. This was a single package with no dependencies. This removed all of the confusion that had been caused by Rx previously being split into four pieces. +[Rx 4.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v4.0.0) tried a different approach: have just one Rx package, `System.Reactive`, which has no dependencies. This removed all of the confusion that had been caused by Rx previously being split into four pieces. -Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool, meaning that a version of Rx that worked on .NET Framework 4.0 would be suboptimal on .NET Framework 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it. In fact, the oldest version of .NET Framework that it made sense to target at this point was 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targeting different versions of .NET Framework. +Rx 4.0 was able to sidestep the plug-in problem because by now, there was no need to ship separate Rx builds for multiple versions of .NET Framework. That had been necessary on older versions because different .NET Framework releases had different capabilities relating to the thread pool or other task-related features, meaning that a version of Rx that worked on .NET Framework 4.0 would be suboptimal on .NET Framework 4.5. But by the time Rx 4.0 came out (May 2018) Microsoft had already ended support for .NET Framework 4.0, so Rx didn't need to support it. In fact, the oldest version of .NET Framework that it made sense to target at this point was 4.6, and it turns out that none of the new features added in subsequent versions of .NET Framework were of particular use to Rx.NET, so there was no longer any value in building multiple versions of Rx.NET targeting different versions of .NET Framework. This was a critical change in the landscape, because it created an opportunity for Rx.NET. @@ -230,15 +230,14 @@ Since there was now just a single .NET Framework target (`net46`), the original This simplification was an ingenious master stroke, and it worked brilliantly. Until it didn't. But we'll get to that. -Although it now targets just one version of .NET Framework, `System.Reactive` is still a multi-target NuGet package. If you download the v6.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: +Although it now targets just one version of .NET Framework, `System.Reactive` is still a multi-target NuGet package. If you download the v4.0 package and unzip it (`.nupkg` files are just ZIP files) you will find the `lib` folder contains subfolders for 5 different TFMs: -* `net472` (.NET Framework) -* `net6.0` -* `net6.0-windows10.0.19041` +* `net46` (.NET Framework 4.0) * `netstandard2.0` +* `uap10` (UWP) * `uap10.0.18362` (UWP) -Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net472` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net472` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. Consequently, the `System.Reactive.dll` in the package's `netstandard2.0` folder does not include the `ControlScheduler`. +Each contains a `System.Reactive.dll` file, and each is slightly different. The `netstandard2.0` one is effectively a lowest common denominator, and it is missing some types you will find in the more specialized versions. For example, the version in `net46` includes `ControlScheduler`, a type that provides integration between Rx and the Windows Forms desktop client framework. Windows Forms is built into .NET Framework—it's not possible to install .NET Framework without Windows Forms—and so it's possible for the `net46` version of Rx to include that type. But `netstandard2.0` does not include Windows Forms—that version of Rx may find itself running on Linux, where Windows Forms definitely won't be available. Consequently, the `System.Reactive.dll` in the package's `netstandard2.0` folder does not include the `ControlScheduler`. This illustrates that with this _great unification_, when you add a reference to `System.Reactive`, you get everything NuGet has to offer on whatever platform your application targets. So if you're using .NET Framework, you get Rx's WPF and Windows Forms features because WPF and Windows Forms are built into .NET Framework. If you're writing a UWP application and you add a reference to `System.Reactive`, you get the UWP features of Rx. @@ -269,7 +268,7 @@ Why is it a problem? Well, what UI framework integration should Rx offer in its | `netstandard2.0` | None | None | | `netcoreapp3.0`| **None, probably** (see below) | **Windows Forms and WPF (!)** | -Why have I put "None" in the `netcoreapp3.0` row, bearing in mind that .NET .NET Core 3.0 added WPF and Windows Forms support? Well these UI frameworks are only available on Windows. The `netcoreapp3.0` TFM is OS-agnostic. With this target you could find yourself running on macOS or Linux. The Windows-specific underpinnings won't necessarily be there, and that's why I believe the correct answer for that row is "None". +Why have I put "None" in the middle column of the `netcoreapp3.0` row, bearing in mind that .NET .NET Core 3.0 added WPF and Windows Forms support? Well these UI frameworks are only available on Windows. The `netcoreapp3.0` TFM is OS-agnostic. With this target you could find yourself running on macOS or Linux. The Windows-specific underpinnings won't necessarily be there, and that's why I believe the correct answer for that row is "None". As part of Rx.NET's [preparation for .NET 5 support](https://github.com/dotnet/reactive/pull/1291), a `net5.0` target was added. This did **not** include Windows Forms and WPF features. That is unarguably correct, because if you were to create a new project targeting `net5.0` and set either `UseWPF` or `UseWindowsForms` (or both) to `true` you'd get a build error telling you that you can only do that when the target platform is Windows. It recommends that you use an OS-specific TFM, such as `net5.0-windows`. @@ -277,7 +276,7 @@ Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs chan My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available (unless that component is specifically designed to be used only in environments where these frameworks will be available). That's why I put "None" in the 2nd column for that row. However, it seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target. Since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. -I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. +I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. It's moot because .NET Core 3.0 is no out of support, but unfortunately, the decision made in the .NET 3.0 Core timeframe remains with us. The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew that with a TFM of `net5.0-windows` you would definitely be running on Windows, although that was no guarantee that .NET 5's Windows Forms and WPF support was actually available. (On Windows, you can install just the [.NET 5.0 Runtime](https://dotnet.microsoft.com/en-us/download/dotnet/5.0) without including the .NET Desktop Runtime if you want.) And a TFM of `net5.0` increased the chances of their not being available because you might not even be running on Windows. So let's look at the options again in this new .NET 5.0 world, listing all the TFMs that [Rx 5.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v5.0.0) (the first version to support .NET 5.0) offered: @@ -292,11 +291,13 @@ The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew t This repeats the .NET Core 3.0 problem for .NET Core 3.1, but given what Rx 4.2 did, Rx 5.0 pretty much had to do the same thing regardless of whether you think it was right or wrong. -It does **not** repeat the mistake for `net5.0` but then it can't: when targeting .NET 5.0 or later, the build tools prevent you from trying to use Windows Forms or WPF unless you've specified that your target platform has to be Windows. +It does **not** repeat the mistake with the `net5.0` TFM but then it can't: when targeting .NET 5.0 or later, the build tools prevent you from trying to use Windows Forms or WPF unless you've specified that your target platform has to be Windows. The last row is interesting. Again, I've said it probably shouldn't include Windows Forms and WPF support. But really that's because I think that last row shouldn't even be there. There are good reasons that merely using some `-windows` TFM shouldn't automatically turn on WPF and Windows Forms support, but if you agree with that, then there's no longer any reason for Rx to offer a `-windows` TFM at all—there'd be no difference between those two .NET 5.0 TFMs at that point. -The reason I think Windows Forms and WPF support should not automatically be included just because you've used a `-windows` TFM is that there are many different reasons you might want such a TFM, many of which have nothing to do with either Windows Forms or WPF. For example, this is a completely legitimate C# console application: +(Actually, there is a third point of view: Rx could have provided a `.net5.0-windows10.0.x` TFM that included the features that support Windows Runtime types such as integration between `IAsyncOperation` and `IObservable`, but not include WPF or Windows Forms support.) + +The reason I think Windows Forms and WPF support should not automatically be included just because you've used a `-windows` TFM is that there are many different reasons an application might specify such a TFM, many of which have nothing to do with either Windows Forms or WPF. For example, this is a completely legitimate C# console application: ```cs using Windows.Devices.Input; @@ -317,15 +318,15 @@ This illustrates the very specific meaning of OS-specific TFMs: they determine w * a minimum supported OS version (because code might use a new API when it runs on the latest OS version but be capable of handling its unavailability gracefully) * an intention to use WPF or Windows Forms (this particular program is a console application) -If you want to indicate a minimum OS version, you do that with [`SupportedOSPlatformVersion`](https://learn.microsoft.com/en-us/dotnet/standard/frameworks#support-older-os-versions) property in your project file. This is allowed to be lower than the version in your TFM (but you need to detect when you're on an older version and handle the absence of missing APIs gracefully). +If you want to indicate a minimum OS version, you do that with [`SupportedOSPlatformVersion`](https://learn.microsoft.com/en-us/dotnet/standard/frameworks#support-older-os-versions) property in your project file. This is allowed to be lower than the version in your TFM (but you would then need to detect when you're on an older version and handle the absence of missing APIs gracefully). -If you want to use WPF, you set the `UseWPF` property to true in your project file. For Windows Forms you set `UseWindowsForms`. It's entirely possible to need to specify a Windows-specific TFM without wanting to use either of these frameworks. The console app shown above is a somewhat unusual example. Another, perhaps more common scenario, is that you want to use a different UI framework. +If you are writing an application that wants to use WPF, you set the `UseWPF` property to true in your project file. For Windows Forms you set `UseWindowsForms`. It's entirely possible to need to specify a Windows-specific TFM without wanting to use either of these frameworks. The console app shown above is a somewhat unusual example. Another, perhaps more common scenario, is that you want to use a different UI framework. (Avalonia, for example. Or WinUI.) -But Rx 5.0 takes the position that if an applications targets Windows, Rx should make its WPF and Windows Forms functionality available. (In fact, Rx doesn't support this on versions of Windows older than 10.0.19041, aka Windows 10 2004. So if your TFM specifies an older version, or no version at all (which implicitly means Windows 7 by the way) then Rx's WPF and Windows Forms won't be available.) +But Rx 5.0 takes the position that if an applications targets Windows, Rx should make its WPF and Windows Forms functionality available. (In fact, Rx doesn't support this for TFMs specifying a Windows API version older than 10.0.19041, aka Windows 10 2004. So if your TFM specifies an older version, or no version at all (which implicitly means Windows 7 by the way) then Rx's WPF and Windows Forms support won't be available.) The problem with that is that if you use any self-contained form of deployment (including Native AOT) in which the .NET runtime and its libraries are shipped as part of the application, that means your application will be shipping the WPF and Windows Forms parts of the .NET runtime library. Normally those are optional—the basic .NET runtime does not include them—so this is not a case of "well you'd be doing that anyway." -Let's look at the impact. The first column of the following table shows the size of the deployable output (excluding debug symbols, which get included in the published output by default) for the code shown above. The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. +Let's look at the impact. The first column of the following table shows the size of the deployable output for the code shown above (excluding debug symbols; these will be present in the published output by default but including them here skew the results for the smaller outputs). The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. | Deployment type | Size without Rx | Size with Rx targeting 18362 | Size with Rx targeting 19041 | |--|--|--|--| @@ -334,7 +335,7 @@ Let's look at the impact. The first column of the following table shows the size | Self-contained trimmed | 18.3MB | 18.3MB | 65.7MB | | Native AOT | 5.9MB | 6.2MB | 17.4MB | -But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`, the oldest Windows version for which Rx offers support on .NET 6.0 and later. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though I actually have no use for it. +But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`, the oldest Windows version for which Rx offers support on .NET 6.0 and later. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though nothing in my code actually uses it. In the framework-dependent row it makes only a small difference (because the copy of `System.Reactive.dll` we get is a little larger). But that's misleading: the resulting executable will now required host systems to have not just the basic .NET 8.0 runtime installed, but also the optional Windows Desktop components. So unless the target machine already has that installed, I will in fact have a larger install to perform. @@ -357,9 +358,36 @@ In my view, the best solution to this whole problem would have been for all of t This is easy to say with hindsight of course, particularly since there are now many different options for building client-side UI with .NET. In a world where Avalonia, MAUI, Windows Forms, WPF, and WinUI are all possibilities for a .NET application, the idea that `System.Reactive` should do everything looks obviously unsustainable, in a way that it didn't back in the Rx 4.0 days. -### The return of the plug-in problems in Rx 5.0 +#### The return of the plug-in problems in Rx 5.0 + +In the section describing the [Great Unification](#rx-40s-great-unification), I explained how Rx 4.0 did a better job of dealing with the plug-in problems than Rx 3.1's attempt to solve the same problems had managed. Unfortunately, these problems returned in Rx 5.0. + +But why? This is the critical text from that section: + +> ...there was no longer any value in building multiple versions of Rx.NET targeting different versions of .NET Framework. +> +>This was a critical change in the landscape, because it created an opportunity for Rx.NET. +> +> Since there was now just a single .NET Framework target (`net46`)... + +In Rx 5.0, it was still true that there was just a single .NET Framework target. But it had changed. It was now `net472` instead of `net46`. And that turns out to create a new version of the problem. + +Remember, the basic plug-in problem occurs when a single version of Rx contains multiple distinct assemblies with the same strong name that can run on the same version of .NET Framework. Rx 4.0 looks like it might have that problem because it contains `net46` and `netstandard2.0` targets. .NET Framework 4.6.2 supports both of these TFMs. However, the way NuGet packages get resolved means that for any version of .NET Framework that supports `netstandard2.0` (4.6.2 or later), it will consider the `net46` TFM to be a better match than the `netstandard2.0` one. + +In short, there is no version of .NET Framework for which the build system will select the `netstandard2.0` component. Older versions of .NET don't support .NET Standard 2.0. And for newer versions, it will always pick the `net46` library. + +Unfortunately, it's different in Rx 5.0. + +Which Rx target will be used when we target .NET Framework versions 4.6.2, 4.7, or 4.7.1? None of these can load the `net472` target because that required .NET Framework 4.7.2 or later. But they can all load the `netstandard2.0` one. + +This opens the door to a plug-in problem. If someone built a Visual Studio plug-in targetting .NET Framework 4.6.2 that uses Rx 5.0, that plug-in would include a copy of the `netstandard2.0` copy of `System.Reactive.dll`. A plug-in targetting .NET Framework 4.7.2 that also uses Rx 5.0 will include a copy of the `net472` DLL. If that first plug-in loads first, it will cause the `netstandard2.0` DLL to load, and since that has exactly the same strong name as the `net472` DLL, the second plug-in is also going to get that `netstandard2.0` one. So if that second plug-in tries to use, say, Rx's WPF features, it will fail with a `TypeLoadException` or `MissingMethodException`. + +And yet, nobody seems to have reported this regression. Why would that be? + +It seems likely that the answer is that unlike the problems back in the Rx 2.0 and 3.0 days, the problem does not occur by default. Anyone using Rx 5.0 or later in a Visual Studio plug-in will most likely be targetting .NET Framework 4.7.2 or later. By the time Rx 5.0 shipped, there has never been any good reason to write a plug-in that targets an older version of Visual Studio. Of course, there are plenty of older plug-ins still around but those will be using older versions of Rx.NET. + +In the unlikely event of needing to write a new plug-in that targets a version of .NET Framework older than 4.7.2, you can always use an old version of Rx. So this problem is not the showstopper it was in older versions of Rx.NET. -TBD. ### The workaround From cbb4422b349194c30deec4e2f0bfc070c39761c8 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Wed, 21 Feb 2024 16:48:03 +0000 Subject: [PATCH 16/19] Completed updates made after prototype learnings --- ...0003-windows-tfms-and-desktop-framework.md | 142 ++++++++++-------- 1 file changed, 82 insertions(+), 60 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 50b7002cd..d0a53fca7 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -399,7 +399,7 @@ If your application has encountered [the problem](#the-problem), you can add thi ``` -This only needs to go in the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. The problem afflicts only executables, not DLLs. +This only needs to go in the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. Nor does it need to go into NuGet packages. The problem afflicts only executables, not DLLs. here's an updated version of the table from the previous section. The final two columns are for the same application as last time, targeting `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. @@ -414,7 +414,7 @@ As you can see, this is much more reasonable. In the first two cases, using Rx.N So that seems pretty effective. -Why not just set [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/-msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) back to `false`? The short answer is: it doesn't work. But why? The problem is that these really only determine whether the code in your project can use WPF or Windows Forms features. Your project might not use them, but that doesn't change the fact that if any of the components you depend on do have a dependency on the .NET runtime Windows Desktop components, your application will automtically pick up that dependency even if you've not turned on the WPF or Windows Forms features for your own build. +Why not just set [`UseWPF`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/-msbuild-props-desktop#usewpf) and [`UseWindowsForms`](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props-desktop#usewindowsforms) back to `false`? The short answer is: it doesn't work. But why? The problem is that these really only determine whether the code in your project can use WPF or Windows Forms features. Your project might not use them, but that doesn't change the fact that if any of the components you depend on do have a dependency on the .NET runtime Windows Desktop components, your application will automatically pick up that dependency even if you've not turned on the WPF or Windows Forms features for your own build. ### Community input @@ -429,17 +429,21 @@ We started a [public discussion about this topic](https://github.com/dotnet/reac I would not dismiss anything Anais has to say about Rx lightly. She is a long-standing Rx veteran and continues to be a positive force in the Rx community. In particular, I think we need to pay close attention to the concerns she raises over the damage we might do if we overcomplicate Rx's packaging. -However, the evidence compels me to disagree with the implication of her rhetorical question. My answers to her two questions here are: yes, and no, respectively. +However, the evidence compels me to disagree with the implication of her first rhetorical question. My answers to her questions are: -Yes, clearly people are worried about this. We know this because people have complained, and some major projects have abandoned Rx purely because of this problem. They did not do that lightly. This demonstrates that yes, people really are worried. +* yes, people are demonstrably walking away from Rx because of this +* yes, even in 2023 +* no + +We know people are worried about this because they have complained, and some major projects have abandoned Rx purely because of this problem. They did not do that lightly. This demonstrates that yes, people really are worried. For what it's worth I'm not convinced it's all about disk space per se. Network bandwidth is an issue. Not everyone has hundreds of megabits of internet connectivity. I live in the UK, which is a reasonably advanced economy, and I can get 1GB internet to my house, but only because I live in a built up area. A significant proportion of the population can't get even 10% of that speed. And that's in a country with unusually high average population density and a fairly high-tech economy. There are plenty of countries where poor connectivity is even more common than here, with no prospect of change any time soon. So adding 90MB unnecessarily to a download is a problem for significant parts of the world. -Even in a world of gigabit internet, web page download sizes matter. We added trimming support to Rx 6.0 specifically in response to people wanting to use it in Blazor. Bear in mind that in that scenario removed a mere 1MB from the download size, and yet it was a feature people care about. If 1MB can be a problem, I think it's safe to say that there are still scenarios today where 90MB matters. +Even in a world of gigabit internet, web page download sizes matter. We added trimming support to Rx 6.0 specifically in response to people wanting to use it in Blazor. Bear in mind that in that scenario, trimming removed a mere 1MB from the download size, and yet it was a feature people care about. If 1MB can be a problem, I think it's safe to say that there are still scenarios today where 90MB matters. But regardless of what the reasons might be, people are demonstrably worried enough about disk space in 2023 to drop Rx. And that means we do actually have to do something. -As for Anais's second point, no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed in the next section) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. +As for Anais's second point, no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed here in the [next section](#the-peculiar-faith-in-the-power-of-breaking-changes)) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. And while I disagree with the premise that 90MB is no big deal, I think Anais offers some extremely important analysis. She worries that a solution to this problem could be: @@ -447,10 +451,6 @@ And while I disagree with the premise that 90MB is no big deal, I think Anais of My conclusion is that we should keep things as simple as possible, but no simpler. And that "no simpler" is the tricky bit. -By the way, it's worth noting that [glopesdev offered a slightly view, but one still focused on how easy it is to understand the packagin](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617): - -> I don't buy the argument of counting nodes in a "dependency tree" as a measure of complexity. Humans often care more about names than they care about numbers. My definition of "sanity" is not just that we have one package, it's that the root of the namespace System.Reactive should not be in a package called System.Reactive.Base. - I think the problem we have today is that the _great unification_ was, with hindsight, an oversimplification. So for me the questions are: * what's the simplest approach that's not an oversimplification? @@ -471,9 +471,9 @@ A feature of the community response that surprised me was that a lot of people s * [A disruptive change is worth it if it solves the current problem and meets future plans, it also helps to ship new features and bug fixes faster.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-8170628) * [I really would like the team to consider that a major update is a major update and that breaking changes are acceptable](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-8279135) -About half of these make no attempt to explain why they think a breaking change will help. Simply breaking everything is no guarantee of fixing the problem! +About half of these make no attempt to explain why they think a breaking change will help. Simply breaking everything is no guarantee of fixing the problem! You can't make an omelette without breaking eggs, but that doesn't mean you should leap at the first opportunity to smash some eggs. You're more likely to end up with a mess than an omelette. -`System.Reactive` seems to get about half a million downloads a month, so it seems reasonable to conclude that any change has the potential to affect a lot of people. You may find breaking changes acceptable, but there might be tens of thousands who take a different view. Moreover, I've helped enough teams out with dependency messes caused by ill-thought-out breaking changes to want to be extremely cautious about introducing breaking changes in a widely used package. +`System.Reactive` seems to get about half a million downloads a month, so it seems reasonable to conclude that any change has the potential to affect a lot of people. You personally may find breaking changes acceptable, but there might be tens of thousands who take a different view. Moreover, during my career I've helped out many teams out with dependency messes caused by ill-thought-out breaking changes. I've seen the damage that breaking changes do. I therefore want to be extremely cautious about introducing breaking changes in a widely used package. It's also worth pointing out that not all breaking changes are created equal. Here are some examples: @@ -481,9 +481,11 @@ It's also worth pointing out that not all breaking changes are created equal. He 2. CLR serialization is most of the way through a very long process of deprecation 3. We could (hypothetically) remove the UI-specific feature from `System.Reactive` in v7 -These are all, technically, binary breaking changes. For example, `FileStream` has become much less aggressive about synchronizing some of its internal state with the operating system. This has dramatically improved performance in certain scenarios, but it does mean any code written for .NET Core 3.1 that made certain assumptions about exactly how `FileStream` was implemented might break on .NET 8.0. Microsoft categorises this as a binary breaking change, but the reality is that the overwhelming majority of users of `FileStream` will be unaffected. Their code will run a little faster but nothing else will change. Microsoft went to great lengths to minimize any practical incompatibilities, and also, for a few releases, they provided a way to revert to the old behaviour. This was carefully thought out, there was a multi-release plan for how the new behaviour would be phased in, with safeguards in place in case the negative impact was worse than expected, and with escape hatches for anyone adversely affected. +These are all, technically, binary breaking changes, but they are very different from one another. + +For example, consider the nature of one of the binary breaking changes in `FileStream`: this class has become much less aggressive about synchronizing some of its internal state with the operating system. This has dramatically improved performance in certain scenarios, but it does mean any code written for .NET Core 3.1 that made certain assumptions about exactly how `FileStream` was implemented might break on .NET 8.0. Microsoft categorises this as a binary breaking change, but the reality is that the overwhelming majority of users of `FileStream` will be unaffected. Their code will run a little faster but nothing else will change. Microsoft went to great lengths to minimize any practical incompatibilities, and also, for a few releases, they provided a way to revert to the old behaviour. This was carefully thought out, there was a multi-release plan for how the new behaviour would be phased in, with safeguards in place in case the negative impact was worse than expected, and with escape hatches for anyone adversely affected. While this is technically a breaking change, most users of this class will be unaffected by it. -The second case, the gradual removal of CLR serialization, is quite different. In this case, functionality is being removed. Anyone dependent on that functionality will be out of luck once it goes entirely. This is being done because CLR serialization is a security liability, and it's a feature nobody should really be using. But lots of people have used it in the past, which is why it was brought over from .NET Framework to .NET. So although this is a more brutal kind of breaking change than 1) above, it is being handled in a way designed to give developers using it a very long runway for finally breaking free of it. There is a [published plan](https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md) for how it will be phased out. There has been a phased approach in which there were initially just warnings, and then a change where it was disabled by default but could easily be re-enabled. It will eventually vanish completely, but we had years of notice that it was on the way out. Even though CLR serialization was re-introduced in .NET Core explicitly as a stopgap measure, very clearly signposted as something intended only to support porting of code from .NET Framework, and not something to be used in new development, the deprecation was done over about half a decade, and 5 releases of .NET. +The second case, the gradual removal of CLR serialization, is quite different. In this case, functionality is being removed. Anyone dependent on that functionality will be out of luck once it goes entirely. This is being done because CLR serialization is a security liability, and it's a feature nobody should really be using. But lots of people have used it in the past, which is why it was brought over from .NET Framework to .NET. So although this is a more brutal kind of breaking change than 1) above, it is being handled in a way designed to give developers using it a very long runway for finally breaking free of it. There is a [published plan](https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md) for how it will be phased out. There has been a phased approach in which there were initially just warnings, and then a change where it was disabled by default but could easily be re-enabled. It will eventually vanish completely, but we had years of notice that it was on the way out. (And even then, there will still be an unsupported NuGet package that implements the behaviour if you really want it.) Even though CLR serialization was re-introduced in .NET Core explicitly as a stopgap measure, very clearly signposted as something intended only to support porting of code from .NET Framework, and not something to be used in new development, the deprecation was done over about half a decade, and 5 releases of .NET. Now compare that with 3), the idea that we should relax about backwards compatibility and just remove the problematic APIs from Rx. That very different from 2) (which in turn is very different from 1). This would be a sudden shock for existing users of Rx. This is absolutely guaranteed to cause problems for anyone who was using Rx in a way that it was very much designed to be used. That's a very different sort of breaking change from one that only affects people who knowingly used a doomed API, or one that doesn't affect anyone using an API normally. @@ -495,9 +497,9 @@ I should clarify that we're not totally opposed to breaking changes, we just wan We think that in the long run we do need to make a breaking change to deal with this issue properly: we need to get the UI-framework-specific pieces out of `System.Reactive`. So this can never be as gentle a breaking change as 1) above: we are intending to remove something from the API. -It won't be quite as gentle as 2) either. The thing about CLR serialization is that anyone using it in .NET Core knew its days were numbered from the start. But in Rx we're talking about a change that had not been envisaged back in 2018 when the Rx API adopted its current form. +It won't be quite as gentle as 2) either. The thing about CLR serialization is that anyone using it in .NET Core knew its days were numbered from the start. But in Rx we're talking about a change that had not been envisaged back in 2018 when the Rx API adopted its current form. People currently using Rx.NET's WPF and Windows Forms features don't know that change is coming. -But I don't want to do anything as brutal as 3. And the difference between 2 and 3 is essentially the combination of fair warning and time. +But I don't want to do anything as brutal as 3. The difference between 2 and 3 is essentially the combination of fair warning and time, so the upshot is that we are open to breaking changes, but they need to happen gradually, and people need to discover that the changes are coming in plenty of time to respond. #### 'Clean starts' aren't, due to extension method ambiguity @@ -513,9 +515,9 @@ For me this pretty much destroys the idea of a 'clean break' to enable breaking Some people think it's wrong of us to try to maintain high levels of backwards compatibility. I think that's debatable, but if you are going to take that position, then it demands the question: how much backwards compatibility is enough? How much can we break? -As it happens, I made clear from the start that total compatibility was not a goal. For example. I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache. +As it happens, I made clear from the start of our community engagement that total compatibility was not a goal. For example. I am quite keen to throw UWP under a bus if at all possible because its continued presence in the Rx codebase is a massive headache, and likely to cause more problems over time. (It has never been properly supported in the modern .NET SDK, and the system we use to work around this is an open source project that is no longer under active development.) -Here's a discussion where this was raised: +Here's the discussion that raised the idea that we might be holding ourselves to too high a standard: > [I don't understand why given this we are assuming that System.Reactive needs to somehow hold itself to a higher standard than the core framework itself.](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617) @@ -523,9 +525,9 @@ This talks about how .NET itself has had some emphatically non-backwards-compati I think this argument misses that frameworks are fundamentally different from libraries. It's easier for a framework to break completely with the past than it is for a widely used general purpose library. -I don't believe Rx.NET can impose the kind of split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be on one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to choose your framework carefully. Upgrading to newer versions of frameworks is a far more momentous decision than upgrading a library precisely because we expect frameworks to make breaking changes, particularly when they are essentially brand new things that make no pretence of being simply an update to their predecessors. +I don't believe Rx.NET can impose the kind of split that Microsoft did by introducing .NET Core. (Or the similar split they imposed by introducing ASP.NET Core.) When you chose a host runtime (or an application framework) that's an application-level decision, and you're choosing to be in one world instead of another. That is a top-down decision: you choose .NET or .NET Framework and then a whole bunch of consequences flow from which you chose. As the application developer you're going to choose your framework carefully. Upgrading to newer versions of frameworks is a far more momentous decision than upgrading a library precisely because we expect frameworks to make breaking changes, particularly when they are essentially brand new things that make no pretence of being simply an update to their predecessors. -As a developer writing a library, I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects. Or any of the other library types which are nearly identical across both worlds. +As a developer writing a library, I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects. Or any of the other library types which are nearly identical across the .NET FX and modern .NET worlds. So the question we need to ask is this: is Rx more like a framework or a common library feature? Are we more like ASP.NET Core, or `System.Linq`? I'd say the fact that we offer a `netstandard2.0` target points very much towards the latter. @@ -561,13 +563,14 @@ That said there's one possible benefit: specifying a Windows 10 version-specific Currently, I don't know if this is possible. When [endjin](https://endjin.com) took over maintenance of Rx.NET, we did try to find out why the Windows-specific Rx.NET targets chose 10.0.19041. I never got an answer. I think we can't go back any further than 10.0.17763 because I _think_ [C#/WinRT](https://learn.microsoft.com/en-us/windows/apps/develop/platform/csharp-winrt/) requires that version or later, and in practice 10.0.18362 might be the lowest we can use in practice because that's the oldest SDK that Visual Studio 2022 supports. + #### Use obsolete One of the comments was a recommendation to deal with this problem using `[Obsolete]`: https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7604157 -The reason we hadn't originally proposed this is that we thought we needed to take action quickly to resolve the problem. `[Obsolete]` is really only suitable when you form a multi-year to remove an API, and people were having real problems right now. +The reason we hadn't originally proposed this is that we thought we needed to take action quickly to resolve the problem. `[Obsolete]` is really only suitable when you form a multi-year plan to remove an API, and people were having real problems right now. However, now that a [workaround](#the-workaround) seems to be available, using `[Obsolete]` does in fact look likely to be the best option. @@ -578,7 +581,7 @@ There are a few constraints that we need to impose on any possible solution to t #### Can't remove types until a long Obsolete period -The simplest thing we could do to solve the main problem this document describes would be to remove all UI-framework-specific types from the public API surface area of `System.Reactive`. This would entail simply removing the `net6.0-windows10.0.19041`, `net472`, and `uap10.0.18362` targets. Applications using .NET 6.0 or later would get the `net6.0` target, and everything else would use the `netstandard2.0` target. The UI-framework-specific types could be moved into UI-specific NuGet packages, so applications would not simply be left in the lurch: all functionality would remain available, it would simply be distributed slightly differently. +The simplest thing we could do to solve the main problem this document describes would be to remove all UI-framework-specific types from the public API surface area of `System.Reactive`. This would entail simply removing the `net6.0-windows10.0.19041`, `net472`, and `uap10.0.18362` targets. Applications using .NET 6.0 or later would get the `net6.0` target, and everything else would use the `netstandard2.0` target. The UI-framework-specific types could be moved into UI-specific NuGet packages, and we'd also need a Windows Runtime package as a home for Windows-specific but non-UI-framework-specific features such as integration with `IAsyncOperation`. Applications would not simply be left in the lurch: all functionality would remain available, it would simply be distributed slightly differently. Unfortunately, this would create some serious new problems. Consider an application that depends on two libraries that use different versions of Rx. Let's suppose LibraryBefore depends on Rx 6.0, and LibraryAfter depends on some hypothetical future Rx 7.0 that makes the change just described. So we have this sort of dependency tree: @@ -591,7 +594,7 @@ Unfortunately, this would create some serious new problems. Consider an applicat Suppose `LibraryBefore` is using some WPF-specific feature in Rx 6.0—let's say it calls the [`ObserveOnDispatcher` extension method](subscribeon-and-observeon-in-ui-applications). Since it depends on Rx 6.0, it's going to require that method, and its containing `DispatcherObservable` type, to be in `System.Reactive`. -To fully understand why this creates a problem you need to think about what actually gets compiled into components. Here's how the use of `ObserveOnDispatcher` looks in the IL emitted for a library depending on Rx 6.0: +This creates a problem because of what actually gets compiled into components that use Rx. Here's how the use of `ObserveOnDispatcher` looks in the IL emitted for a library depending on Rx 6.0: ```cil call class [System.Runtime]System.IObservable`1 [System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher(class [System.Runtime]System.IObservable`1) @@ -601,7 +604,7 @@ If you're not familiar with .NET's IL, I'll just break that down for you. the `c The IL shown above is how ILDASM, the IL disassembler, interprets it for us. Instead of just showing us the metadata token, it goes and finds the relevant row in the table. In fact it finds a bunch of related rows—there's a table for parameters, and it also has to go and find all of the rows corresponding to the various types referred to: in this case there's the return type, the type of the one and only normal parameter, and also a type argument because this is a generic method. -In fact there's only one really important part in that IL, which I'll call out here: +In fact there's only one part in that IL that's really important to this discussion, which I'll call out here: ``` [System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher @@ -616,7 +619,7 @@ This essentially says that the method we want is: It's point 1 that matters here. This indicates that the method is defined in `System.Reactive`. That's what's going to cause us problems in this scenario. But why? -With that in mind, let's get back to our example. We've established that `LibraryBefore` is going to contain at least one IL `call` instruction that indicates that it expects to find the `ObserveOnDispatcher` method in the `System.Reactive` assembly. +Let's applying this to our example. We've established that `LibraryBefore` is going to contain at least one IL `call` instruction that indicates that it expects to find the `ObserveOnDispatcher` method in the `System.Reactive` assembly. What's `LibraryAfter` going to look like? Remember in this hypothetical scenario, Rx 7.0 has moved all WPF-specific types out of `System.Reactive` and into some new component we're calling `System.Reactive.Wpf` in this example. So code in `LibraryAfter` calling the exact same method (the `DispatcherObservable` class's `ObserveOnDispatcher` extension method) would look like this in IL: @@ -636,7 +639,7 @@ So what? Well, when an application uses two libraries that use two different versions of the same NuGet package, the .NET SDK _unifies_ the reference. In this case, both `LibraryBefore` and `LibraryAfter` use the `System.Reactive` NuGet package, but one wants v6.0.0 and the other wants some hypothetical future v7.0.0. -Unification means that the .NET SDK picks exactly one version of each NuGet package. And the default is that the highest minimum requirement wins. (It's possible for `LibraryBefore` to impose an upper bound: it might state its version requirements as `>= 6.0.0` and `< 7.0.0`. In that case, this would cause a build failure because there's an unresolvable conflict. But most packages specify only a lower bound. When you add a dependency to `System.Reactive` 6.0.0, the .NET SDK interprets that as `>= 6.0.0` unless you say otherwise.) +Unification means that the .NET SDK picks exactly one version of each NuGet package. And the default is that the highest minimum requirement wins. (It's possible for `LibraryBefore` to impose an upper bound: it might state its version requirements as `>= 6.0.0` and `< 7.0.0`. In that case, this would cause a build failure because there's an unresolvable conflict. But most packages specify only a lower bound. When you write a NuGet package with a dependency on `System.Reactive` 6.0.0, when the .NET SDK needs to pick versions for an application that uses your library it interprets that as `>= 6.0.0` unless you say otherwise.) So `MyApp` is going to get `System.Reactive` 7.0.0. That's the version that will actually be loaded into memory when the application runs. @@ -646,11 +649,11 @@ What does that mean for the `LibraryBefore`? Well if it happens never to run the [System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher ``` -The JIT compiler will then inspect the `System.Reactive` assembly and discover that it does not define a type called `System.Reactive.Linq.DispatcherObservable`. The JIT compiler will then throw an exception to report that the IL refers to a method that does not in fact exist. +The JIT compiler will then inspect the `System.Reactive` assembly and discover that it does not define a type called `System.Reactive.Linq.DispatcherObservable`. The JIT compiler will then throw a `TypeLoadException` to report that the IL refers to a method in a type that does not in fact exist. -And that's why we can't just remove types from `System.Reactive`. +And that's why we can't just remove types from `System.Reactive`. Code built against Rx.NET 6.0 might break if used a project where some other component uses Rx.NET 7.0. -Type forwarders seem like they might offer a solution, but unfortunately not. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, in theory we do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." +Type forwarders seem like they might offer a solution, but unfortunately not. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, in theory we could do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." Doesn't that solve the problem? Not really, because if the `System.Reactive` assembly contains type forwarders to some proposed `System.Reactive.Wpf` assembly, the .NET SDK will require the resulting `System.Reactive` NuGet package to have a dependency on `System.Reactive.Wpf`. @@ -658,7 +661,7 @@ And that gets us back to square one: if taking a dependency on `System.Reactive` It would be technically possible to meddle with the build system's normal behaviour in order to produce a `System.Reactive` assembly with a suitable type forwarder, but for the resulting NuGet package not to have the corresponding dependency. However, this is unsupported, and is likely to cause a lot of confusion for people who actually do want the WPF functionality, because adding a reference to just `System.Reactive` (which has been all that has been required for Rx v4 through v6) would still enable code using WPF features to compile when upgrading to this hypothetical form of v7, but it would result in runtime errors due to the `System.Reactive.Wpf` assembly not being found. So this is not an acceptable workaround. -In short, if we were to introduce a breaking change, we don't just create a situation where applications need to fix a few things when they upgrade. We create a situation where using certain combinations of other libraries become unusable. The only ways to resolve this are either to hope all your libraries get upgraded soon, or to decide not to use all of the libraries you want to. +In short, any time we introduce a breaking change, we don't just create a situation where applications need to fix a few things when they upgrade. We create a situation where using certain combinations of other libraries become unusable. The only ways to resolve this are either to hope all your libraries get upgraded soon, or to decide not to use all of the libraries you want to. As is often the case, it's the dependency scenarios that cause trouble. For this reason, our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. @@ -703,12 +706,12 @@ This needs to work. This constraint prevents us from simply removing UI-specific When a developer using Rx.NET upgrades to a newer version, what is the smallest version increment for which we are prepared to consider breakage to be acceptable? What's the smallest length of time between an Rx.NET release coming out, and a subsequent release making a breaking change? What are our minimum requirements around giving fair notice with `[Obsolete]` annotations? -Let's start by picking some upper and lower bounds as a starting point. These aren't meant to be the final answer. The idea here is to pick timespans where we can agree that the upper bound isn't long enough, or that the lower bound is too short: +Let's start by picking some upper and lower bounds as a starting point. These aren't meant to be the final answer. The idea here is to pick timespans where we can agree that the upper bound is long enough, and that the lower bound is too short: * If a project upgrades from Rx.NET 1.0 to 7.0, it is probably reasonable for some things to break * Components with a dependency on Rx 5.0 or 6.0 running in an application that upgrades that to Rx 7.0 must not be broken by this upgrade -That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. (That said, some people seem to be [positively enthusiastic about breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), and would actually disgree with the lower bound I've put here. But our view is that the version of Rx that was still the very latest version as recently as 18th May 2023. Less than a year has passed, which seems like not nearly enough time to be removing API features that even in the current version are not marked as obsolete. So although we recognize that there are some people who think it would be OK to break things more quickly than the lower bound stated above suggests, we aren't going to do that, meaning that this is a lower bound in practice.) +That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. (That said, some people seem to be [positively enthusiastic about breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), and would actually disgree with the lower bound I've put here. But our view is that Rx 5.0 was still the very latest version as recently as 18th May 2023. Less than a year has passed, which seems like not nearly enough time to be removing API features that even in the current version are not marked as obsolete. So although we recognize that there are some people who think it would be OK to break things more quickly than the lower bound stated above suggests, we aren't going to do that, meaning that this is a lower bound in practice.) There's another dimension we need to take into account: @@ -747,18 +750,23 @@ While we would prefer to have no breaking changes at all, there are some circums Simply removing UI-specific types in `System.Reactive` v7 would be a type 2 change (because of [the need for things to work when we have indirect dependencies on multiple versions](#transitive-references-to-different-versions)). -We consider type 2 changes to be unacceptable unless [sufficient time has passed](#minimum-acceptable-path-for-breaking-changes). We consider all breaking changes undesirable, but would be open to type 1 changes on a shorter (e.g. 1-4 year) timescale if no better alternative exists. +We consider type 2 changes to be unacceptable unless [sufficient time has passed and sufficient warning was given](#minimum-acceptable-path-for-breaking-changes). We consider all breaking changes undesirable, but would be open to type 1 changes on a shorter (e.g. 1-4 year) timescale if no better alternative exists. #### Plug-in systems must work -We must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. +An earlier draft of this ADR stated that we must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. That was before discovering that Rx 5.0 had already re-introduced it. So we now have the slightly less ambitious target of not making it any worse than it already is. It's worth re-iterating that this is a non-issue on .NET 6.0+ hosts. Applications hosting plug-ins on those runtimes will use [`AssemblyLoadContext`](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext), enabling each plug-in to make its own decisions about which components it uses. If plug-ins make conflicting decisions, it doesn't matter, because `AssemblyLoadContext` is designed to allow that. -So we only need to consider .NET Framework. But there has been one critical change since Rx.NET first started running into the problems that led to the current situation: Rx.NET now defines exactly one .NET Framework target. The original problem occurred only because we had both `net40` and `net45` targets. Now, we have only `net472`. +So we only need to consider .NET Framework. There have been two relevant changes since Rx.NET first started running into the problems that led to the current situation: + +* As of v4.0, Rx.NET now defines exactly one .NET Framework target. The original problem occurred only because we had both `net40` and `net45` targets. In v6.0, we have only `net472`. +* As of v5.0, the version of .NET Framework we target is later than the oldest version of .NET Framework that the `netstandard2.0` target will run on + +The first of these fixed the problem, and the second brought it back as [discussed earlier](#the-return-of-the-plug-in-problems-in-rx-50). But as that earlier discussion says, it's not quite as bad now because there's rarely any good reason to write plug-ins that target pre-4.7.2 versions of .NET Framework. So as long as we keep the version of .NET Framework that we target as old as is practical, we should continue to minimize this problem. -However, in design options where we consider splitting Rx.NET across multiple packages (e.g., where we move the UI-framework-specific pieces out into separate UI-framework-specific packages) we also need to be careful that we don't recreate the problems that occurred in Rx 3.0. We need to be certain that plug-ins that use two Rx.NET assemblies (e.g., the main Rx.NET assembly and the one containing WPF integration) can't end up with a mismatched pair because some other plug-in already forced the loading of an older version of just one of those assemblies. +However, in design options that would split Rx.NET across multiple packages (e.g., where we move the UI-framework-specific pieces out into separate UI-framework-specific packages) we also need to be careful that we don't recreate the other kinds of plug-in problems that occurred in Rx 3.0. We need to be certain that plug-ins that use two Rx.NET assemblies (e.g., the main Rx.NET assembly and the one containing WPF integration) can't end up with a mismatched pair because some other plug-in already forced the loading of an older version of just one of those assemblies. Since we change Rx.NET assembly versions with each major release, we shouldn't get a situation where `PlugInOne` loads, say, `System.Reactive` 7.0.0, and `PlugInTwo` wants v8.0.0 of `System.Reactive` and (say) `System.Reactive.Wpf` but ends up with v7 of `System.Reactive` (because that's what `PlugInOne` already loaded) and v8 of `System.Reactive`. These two different versions of `System.Reactive` would have different strong names, so .NET Framework will load both versions in this scenario. @@ -775,7 +783,7 @@ One of the issues that occured with the [Rx 3 era attempt to fix this problem](h Rx 4.0 solved this (and Rx 6.0 still uses the same approach) by collapsing everything down to a single NuGet package, `System.Reactive`. To access UI-framework-specific functionality, you no long needed to add a reference to a UI-framework-specific package such as `System.Reactive.Windows.Forms`. From a developer's perspective, the functionality was right there in `System.Reactive`. The exact API surface area you saw was determined by your target platform. If you built a UWP application, you would get `lib\uap10.0.16299\System.Reactive.dll` which had the UWP-specific dispatcher support built in. If your application was built for `net6.0-windows10.0.19041` (or a .NET 6.0 TFM specifying a later Windows SDK) you would get `lib\net6.0-windows10.0.19041\System.Reactive.dll` which has the Windows Forms and WPF dispatcher and related types build in. If your target was `net472` or a later .NET Framework you would get `lib\net472\System.Reactive.dll`, which also included the Windows Forms and WPF dispatcher support (but built for .NET Framework, instead of .NET 6.0). And if you weren't using any client-side UI framework, then the behaviour depended on whether you were using .NET Framework, or .NET 6.0+. With .NET Framework you'd get the `net472` DLL, which would include the WPF and Windows Forms support, but it didn't really matter because those frameworks were present as part of any .NET Framework installation. And if you target .NET 6.0 you'll get the `lib\net6.0\System.Reactive.dll`, which has no UI framework support, but that's fine because any .NET 6.0+ TFM that doesn't mention 'windows' doesn't offer either WPF or Windows Forms. -It's worth noting that even if we do go back to a multi-package approach, this problem goes away if you use the modern project system. The problem described in [#305](https://github.com/dotnet/reactive/issues/305) occurs only with the old `packages.config` system. If you try to reproduce the problem in a project that uses the .NET SDK project system introduced back in MSBuild 15 (Visual Studio 2017), the problem won't occur. Visual Studio 2017 is the oldest version of Visual Studio with mainstream support. +It's worth noting that even if we do go back to a multi-package approach, this problem goes away if you use the modern project system. The problem described in [#305](https://github.com/dotnet/reactive/issues/305) occurs only with the old `packages.config` system. If you try to reproduce the problem in a project that uses the .NET SDK project system introduced back in MSBuild 15 (Visual Studio 2017), the problem doesn't occur. Visual Studio 2017 is the oldest version of Visual Studio with mainstream support. Although some projects continue to use `packages.config` today, proper package references have been available for a long time now, even for projects that are still stuck on the older style of project file. @@ -788,16 +796,20 @@ It is explicitly a non-goal to support `packages.config`. Our position is that p We are not interpreting this to mean that the current position, in which there is only one package (`System.Reactive`) and it serves all purposes must necessarily be preserved. We are currently of the view that it is acceptable for UI-framework-specific support to be in separate components. That would necessarily be the case for frameworks Rx.NET does not provide built-in support for such as Avalonia. And there are now so many UI frameworks for .NET that we think it's a self-evident truth that Rx.NET can't provide built-in support for all of them. And once you accept that some UI-framework-specific support is to be found in separate NuGet packages, it is arguably less confusing if they all work that way. -People have demonstrably been confused by the current (v4 through v6) setup. For all that a single-package approach is simple, people still get tripped up by the highly non-obvious requirement to change their TFM from `net6.0-windows` to `net6.0-windows10.0.19041`. (And over a year after taking over maintenance of this project, I've still not yet managed to find a satisfactory answer as to why it's that particular version. So I don't know how this is meant to be simple for anyone just getting started with Rx.NET.) +People have demonstrably been confused by the current (v4 through v6) setup. For all that a single-package approach is simple, people still get tripped up by the highly non-obvious requirement to change their TFM from `net6.0-windows` to `net6.0-windows10.0.19041`. (And over a year after taking over maintenance of this project, I've still not yet managed to find a satisfactory answer as to why it's that particular version. It seems clear that 10.0.18362 is a practical minimum for whichever component ends up containing the Windows Runtime support, because current .NET SDKs don't support older versions, but the choice of 19041 continues to be a mystery.) So I don't know how this is meant to be simple for anyone just getting started with Rx.NET.) So we are open to multiple packages. But we don't want to let that mean that we open the floodgates to a much more granular approach. +[glopesdev makes a related, about confusion and naming](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7627617): + +> I don't buy the argument of counting nodes in a "dependency tree" as a measure of complexity. Humans often care more about names than they care about numbers. My definition of "sanity" is not just that we have one package, it's that the root of the namespace System.Reactive should not be in a package called System.Reactive.Base. + #### .NET Standard mustn't break things -A possible fly in the ointment is the `netstandard2.0` component, because that could in principle run in a .NET Framework process. We could still end up some plug-in loading that one first, blocking any attempt by subsequent plug-ins to load the `net472` version. +The `netstandard2.0` component is a fly in the ointment because that could in principle run in a .NET Framework process. This is the reason that the [plug-in problem re-appeared in Rx 5.0](#the-return-of-the-plug-in-problems-in-rx-50). It's conceivable because even in a fully post-.NET Framework 4.7.2 world we could still end up with some plug-in somehow loading the `netstandard2.0` DLL, blocking any attempt by subsequent plug-ins to load the `net472` version. -I've put this section in here not because I expect it to happen, but just because we need to double check that it doesn't. In the plug-in scenario described above, it shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the `net472` version over the `netstandard2.0` one. (If they target an older version such as `net462`, then perhaps they would pick `netstandard2.0` instead. But if that's the case, then the current Rx 6.0 design fails in that scenario too, so unwinding the earlier design decisions won't make things any worse than they already are.) +I've put this section in here not because I expect it to happen, but just because we need to double check that it doesn't. In the plug-in scenario described above, it shouldn't happen. The build processes for the individual plug-ins know they are targeting .NET Framework, so they should prefer the `net472` version over the `netstandard2.0` one. (If they target an older version such as `net462`, then they would pick `netstandard2.0` instead, but that's already true for Rx 5.0 and 6.0. We can't do anything about that scenario.) Another consideration is that modern NuGet tooling is better than it was in 2016 when the current design was established. Alternative solutions might be possible now that would not have worked when Rx 4.0 was introduced. @@ -806,7 +818,17 @@ When it comes to .NET 6.0 and later, these problems should a non-issue because b #### Threading constraints are back thanks to WASM -TBD. +There is gradually increasing interest in WASM. This was the main motivation for adding trimming support in Rx 6.0. We have recently had reports of problems that occur only in browser WASM scenarios. So far these have all revolved around scheduler issues that arise from the fact that you can't create threads in browser WASM today. + +This effectively takes us right back to a time that had seemed like it was thankfully behind us: the need to support targets where you couldn't create threads. Only this time it's much worse: + +* browser WASM doesn't get its own TFM: it will happily attempt to load any `net6.0` component +* at least Windows 8 and Windows Phone had some limited thread pool support; in browser WASM there really is only one thread today + +We don't currently have a coherent plan for how we will address this. Currently we do not officially support browser WASM because there is no way to run unit test suites in that runtime. This means we just don't know what works. The MS Test team have made positive noises in response to questions about whether this might become possible, but currently there's no commitment and no timescale, so we can't say when we might offer proper support for WASM. + +However, we need to bear it in mind with the packaging plan, because it is precisely the sort of issue that led to some of the early packaging design and resulting confusion in Rx.NET. We need to have some idea of how we are going to deal with this before setting packaging plans in stone. + ### Types in `System.Reactive` that are partially or completely UI-framework-specific @@ -853,17 +875,15 @@ The following sections describe the design choices that have been considered to The status quo is always an option. It's the default, but it can also be a deliberate choice. The availability of a [workaround](#the-workaround) makes this a more attractive option than it had seemed when we first started looking at this problem. -Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is just to continue doing things in the same way. This is not a good solution. Back when Rx 5.0 was the current version, some people seemed to think that the changes we adopted in Rx 6.0 would be sufficient to solve the problems described in this document, but as is now clear, they don't. - -`System.Reactive` 5.0 targeted `netstandard2.0`, `net472`, `netcoreapp3.1`, `net5.0`, `net5.0-windows10.0.19041`, and `uap10.0.16299`. Rx 6.0 targets `netstandard2.0`, `net472`, `net6.0`, `net6.0-windows10.0.19041`, and `uap10.0.16299`. (In other words, it dropped .NET Core 3.1 and .,NET 5.0, both of which went out of support in 2022, and effectively upgraded the .NET 5.0 target to .NET 6.0.) This is in fact exactly what we did for Rx 6.0, but despite what some people seemed to believe, this was never going to solve the problems this ADR discusses. +Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is just to continue doing things in the same way. This is not a good solution. Back when Rx 5.0 was the current version, some people seemed to think that the changes we adopted in Rx 6.0 would be sufficient to solve the problems described in this document. We did not think that they would, and it is now clear that they don't. -This meets all of the [constraints](#constraints), for the simple reason that those are all concerned with not making things worse than they already are. This _is_ where we already are, so it can't possibly be any worse than where we are. But the big problem is the one stated at the start of this ADR: the fact that self-contained deployments are vastly bloated by unwanted dependencies on Windows Forms and WPF. +Changing nothing meets many of the [constraints](#constraints). Specifically, it meets all those are all concerned with not making things worse than they already are. This _is_ where we already are, so it can't possibly be any worse than where we are. But the big problem is the one stated at the start of this ADR: the fact that self-contained deployments are vastly bloated by unwanted dependencies on Windows Forms and WPF. -I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that. The [workaround](#the-workaround) provides a way to undo this problem, but workaround usually have their own problems, and we don't want people to have to discover and then apply a fix just to be able to use Rx.NET. I think separating out these parts is the only way to achieve this. +I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that. The [workaround](#the-workaround) provides a way to undo this problem, but workarounds usually have their own problems, and we don't want people to have to discover and then apply a fix just to be able to use Rx.NET. I think separating out these parts is the only way to achieve this. This design option also doesn't have a good answer for how we provide UI-framework-specific support for other frameworks. (E.g., how would we offer a `DispatcherScheduler` for MAUI's `IScheduler`?) -So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from primary problem this ADR discusses. +So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from primary problem this ADR discusses. Nor have I seen a proposal that would achieve this. The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it _is_ the existing design. @@ -929,7 +949,7 @@ IObservable numbers = RxV6::System.Reactive.Linq.Observable.Range(0, 3); // This plugs the Range observable from Rx v6 into the Select operator from Rx // v7. This is allowed, because all versions of Rx respect the IObservable -// contract. It might be suboptimal, because Rx operators recognized one +// contract. It might be suboptimal, because Rx operators recognize one // another when you compose them, and can perform certain optimizations, and // that won't happen here because the v7 library isn't going to recognize this // Range operator as one of its own. But it will still work correctly. @@ -943,7 +963,7 @@ strings.Subscribe(Console.WriteLine); This uses the Rx v6 implementation of `Observable.Range`, but the two places where this uses extension methods (the call to `Select` and the call to the delegate-based `Subscribe` overload) will use the Rx v6 implementation, because that's what we specified in the `using` directives when importing the relevant namespaces. -So it's possible to make it work, but we consider this to be confusing and painful. You should only really need to use the `extern alias` mechanism if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. And it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. +So it's possible to make it work, but we consider this to be confusing and painful. You should only really need to use the `extern alias` mechanism if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. Admittedly, it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. The only way to avoid this would be to change not just the NuGet package names but also all the namespaces. (Even this still has the potential to cause confusion if any file ends up needing to us namespaces from both worlds.) That somewhat resembles what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretence of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. @@ -959,7 +979,7 @@ So for all these reasons, we are rejecting this design option. In this design, we introduce a new NuGet package that supersedes `System.Reactive` (much as `System.Reactive` superseded `Rx-Main` back in Rx v4.0). This new package would essentially be `System.Reactive` with all the UI-specific functionality removed. We would add new per-UI-framework packages and the UI-specific functionality would move into these. We would continue to build and publish `System.Reactive`, which would become a facade, containing nothing but type forwarders. `System.Reactive` would continue to support the same targets. Its use would be deprecated, but we would likely continue to support it for a very long time—we still build the type facades that were added to enable pre-great-unification code to continue to use the older package structure. -The main appeal of this option was that it offered a fairly quick fix for the main problem in this ADR. We could immediately publish a package that made Rx's core functionality available in a way that would definitely not add any unwanted UI framework references. Before we discovered the [workaround](#the-workaround), we needed a solution urgently, because there didn't seem to be any way to work around the problems that this ADR addresses. This seemed like the fastest way to get to that point. However, now that we have a workaround, this removes the time pressure. +The main appeal of this option was that it offered a fairly quick fix for the main problem in this ADR. We could immediately publish a package that made Rx's core functionality available in a way that would definitely not add any unwanted UI framework references. Before we discovered the [workaround](#the-workaround), we thought we needed a solution urgently, because there didn't seem to be any way to work around the problems that this ADR addresses. This seemed like the fastest way to get to that point. However, now that we have a workaround, this removes the time pressure. The biggest downside of this approach is that it's yet another change in how to use Rx.NET, so this conflicts with the constraint of [minimizing confusion](#minimize-confusion-as-far-as-possible). If it had been the only way to unblock projects that wanted to use Rx.NET on `net6.0-windows.10.*` targets without bringing in WPF and Windows Forms, this might have been an acceptable downside. But that availability of a workaround effectively removes this option's unique upside, at which point the downside looks less acceptable. @@ -989,8 +1009,7 @@ In this option, `System.Reactive` would continue to be the package through which We would need to define brand new types in these new frameworks to replace the ones that will ultimately be removed. We can't use type forwarders because it's not possible to deprecate use of a type through a type forwarder, but not have it be deprecated if you use it directly. -This also has a similar unresolved question around `ThreadPoolScheduler` was was discussed in [option 4](#option-4-systemreactive-remains-the-primary-package-and-becomes-a-facade). - +This also has a similar unresolved question around `ThreadPoolScheduler` was was discussed in [option 4](#option-4-systemreactive-remains-the-primary-package-and-becomes-a-facade). The most likely solution would be to deprecate the members of `ThreadPoolScheduler` that are unique to UWP, and to add a new type with the same functionality (but non-deprecated) in a separate component. This would be the first step on a path that would eventually see UWP get the same `ThreadPoolScheduler` as everything else. ### Other options less seriously considered @@ -1001,8 +1020,7 @@ There were a few other ideas that we rejected fairly quickly. As discussed in https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7628930: ->It would be technically possible for us to build a NuGet package for System.Reactive which contains a net8.0 target and which did NOT contain a net8.0-windows... target, but where the DLL in the net8.0 folder contained a DLL that did in fact have references to WPF and Windows Forms assemblies. And if you do this, then as long as you never attempt to use any of the WPF/WinForms features, you'd never know those references are there. The attraction of this is that if you do happen to be running in an application that has specified, say, true, then you will be able to use the WPF-specific features in this System.Reactive. - +>It would be technically possible for us to build a NuGet package for `System.Reactive` which contains a `net8.0` target and which did NOT contain a `net8.0-windows...` target, but where the DLL in the `net8.0` folder contained a DLL that did in fact have references to WPF and Windows Forms assemblies. And if you do this, then as long as you never attempt to use any of the WPF/WinForms features, you'd never know those references are there. The attraction of this is that if you do happen to be running in an application that has specified, say, `true`, then you will be able to use the WPF-specific features in this `System.Reactive`. While this might work, employing hacks that deliberately subvert the way the build system works are exactly the sort of thing that causes trouble a few years down the line. So we don't want to go down that path. @@ -1020,7 +1038,9 @@ We will go with [option 6](#option-6-ui-framework-specific-packages-deprecating- * `System.Reactive` will remain as the main package for using Rx.NET * we will add a new NuGet packages for each UI framework, and put UI-framework-specific support in these -* the UI-frameworks-specific types in `System.Reactive` will be marked as `[Obsolete]` with the [very long-term](#minimum-acceptable-path-for-breaking-changes) intention of removing them entirely, likely not before 2030 +* there will be an additional package for functionality specific to Windows Runtime that is not UI-framework-specific +* the UI-frameworks-specific and Windows-Runtime-specific types in `System.Reactive` will be marked as `[Obsolete]` with the [very long-term](#minimum-acceptable-path-for-breaking-changes) intention of removing them entirely, likely not before 2030 +* the UWP-specific members of its `ThreadPoolScheduler` will be marked as `[Obsolete]` As it says in [the announcement for the first Rx release](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5): @@ -1036,11 +1056,11 @@ So it is time to de-unify the UI-framework-specific parts. The main relevant consequences are: -* This minimizes disruption: `System.Reactive` continues to be the main way to use Rx.NET +* This minimizes disruption: `System.Reactive` continues to be the main way to use Rx.NET; applications with non-Windows-specific TFMs should be entirely unaffected * Applications encountering the problem described in this ADR will need to use [the workaround](#the-workaround) for the foreseeable future * The eventual end state (ca. 2030) is that `System.Reactive` will be free from UI-framework-specific types -* New assemblies, one for each UI framework we support (Windows Forms, WPF, and UWP) will be added containing replacements for the UI-framework-specific types in `System.Reactive` -* We will deprecate the existing UI-framework-specific types so that developers will be notified that they are on their way out +* New assemblies, one for each UI framework we support (Windows Forms, WPF, and UWP) and one for the non-framework-specific Windows Runtime support will be added containing replacements for the relevant types in `System.Reactive` +* We will deprecate the existing UI-framework-specific and Windows-Runtime-specific types so that developers will be notified that they are on their way out * We won't be able to add type forwarders from the old types in `System.Reactive` to their replacements because you can't deprecate a type forwarder * We'll be able to take UWP out of the majority of the source tree in 2-3 years * All UI-frameworks-specific support will be on an equal footing, enabling MAUI, Blazor, and Avalonia to be supported in exactly the same way as Windows Forms and WPF (i.e., through separate NuGet packages) @@ -1050,6 +1070,8 @@ Although there are some existing old packages that used to contain some UI-frame Our current plan is to leave these facade components untouched, since the split of types is a bit confusing, the naming is a bit inconsistent, and their purpose for years has been to provide backwards compatibility for Rx v3 era apps. We will introduce new components each with a clear purpose: * `System.Reactive.Integration.WindowsForms` -* `System.Reactive.Integration.WPF` -* `System.Reactive.Integration.UWP` +* `System.Reactive.Integration.Wpf` +* `System.Reactive.Integration.Uwp` +* `System.Reactive.Integration.WindowsRuntime` +We expect people to hate these names on first acquaintance. (Particularly the `Integration` bit.) If we're honest, we don't love them, but all other suggestions to date have either had problems (e.g. there are already packages with the suggested names) or aren't noticeably better. \ No newline at end of file From 7bc5fc9b647ee77efa26a46cf3493862e24b4648 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Tue, 29 Oct 2024 06:55:41 +0000 Subject: [PATCH 17/19] Fix typo in ADR Co-authored-by: Steven Weerdenburg --- .../adr/0003-windows-tfms-and-desktop-framework.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index d0a53fca7..846d70d03 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -124,7 +124,7 @@ This meant that it would be possible, in principle, to write a library that depe This was years before .NET Standard was introduced, and at the time, if you wanted to write cross-platform libraries, you had to create something called a Portable Class Library (PCL). Rx wanted to offer a common API across all platforms while also providing optimized platform-specific schedulers, so it introduced a platform abstraction layer and a system it called "enlightenments" (named after a similar feature in Virtual Machine architectures). This worked, but resulted in a somewhat confusing proliferation of DLLs. -An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The the original idea was that this would be a stable component that didn't need frequent releases, because the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. This defeated the entire purpose of having a separate component for the core interfaces. +An additional dimension to the confusion is that even within any single target platform, Rx was split across several different components, and it wasn't entirely obvious why. There was a separate `System.Reactive.Interfaces` component defining the core interfaces Rx defines that aren't in the runtime libraries such as `IScheduler` and `ISubject`. The original idea was that this would be a stable component that didn't need frequent releases, because the core Rx interfaces would change very rarely. That expectation was proven correct over time, but unfortunately, the rationale behind the packaging decision was apparently forgotten, because instead of `System.Reactive.Interfaces` v2.0.0 being the one true definition for all time, new versions of this component were produced with each new version of Rx even when nothing changed. This defeated the entire purpose of having a separate component for the core interfaces. (In fact things were a little weirder because some of the versions of .NET supported by Rx 1.0 defined the core `IObservable` and `IObserver` interfaces in the runtime class libraries but some did not. These interfaces were not present in .NET 3.5, for example, which Rx 1.0 supported. So Rx had to bring its own definition of these for some platforms. You might expect these to live in `System.Reactive.Interfaces` but they did not, because Microsoft wanted that package to be the same on all platforms. So on platforms where `IObversable/er` were not built in, there was yet another DLL in the mix, further adding to the confusion around exactly what assemblies you needed to ship with your app if you wanted to use Rx.) From 77afdc8651f369fe3c25b15ecb61df6c71e05f19 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Tue, 29 Oct 2024 06:55:56 +0000 Subject: [PATCH 18/19] Fix another typo in ADR Co-authored-by: Steven Weerdenburg --- .../adr/0003-windows-tfms-and-desktop-framework.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index 846d70d03..e299d120e 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -276,7 +276,7 @@ Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs chan My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available (unless that component is specifically designed to be used only in environments where these frameworks will be available). That's why I put "None" in the 2nd column for that row. However, it seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target. Since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. -I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. It's moot because .NET Core 3.0 is no out of support, but unfortunately, the decision made in the .NET 3.0 Core timeframe remains with us. +I don't know what happens if you use Rx 4.2 on .NET Core 3.0 in an environment where you don't in fact have Windows Forms or WPF. (There are two reasons that could happen. First, you might not be running on Windows. Second, more subtly, you might be running on Windows, but in an environment where .NET Core 3.0's WPF and Windows Forms support has not been installed. That is an optional feature of .NET Core 3.0. It typically isn't present on a web server, for example.) It might be that it doesn't work at all. Or maybe it works so long as you never attempt to use any of the UI-framework-specific parts of Rx. It's moot because .NET Core 3.0 is now out of support, but unfortunately, the decision made in the .NET 3.0 Core timeframe remains with us. The addition of OS-specific TFMs cleared things up a bit in .NET 5.0. You knew that with a TFM of `net5.0-windows` you would definitely be running on Windows, although that was no guarantee that .NET 5's Windows Forms and WPF support was actually available. (On Windows, you can install just the [.NET 5.0 Runtime](https://dotnet.microsoft.com/en-us/download/dotnet/5.0) without including the .NET Desktop Runtime if you want.) And a TFM of `net5.0` increased the chances of their not being available because you might not even be running on Windows. So let's look at the options again in this new .NET 5.0 world, listing all the TFMs that [Rx 5.0](https://github.com/dotnet/reactive/releases/tag/rxnet-v5.0.0) (the first version to support .NET 5.0) offered: From 88a4bf90014487fb09ae7422c61cc6ed4382cb84 Mon Sep 17 00:00:00 2001 From: Ian Griffiths Date: Tue, 29 Oct 2024 09:44:31 +0000 Subject: [PATCH 19/19] Update packaging ADR after read through --- ...0003-windows-tfms-and-desktop-framework.md | 106 +++++++++--------- 1 file changed, 53 insertions(+), 53 deletions(-) diff --git a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md index e299d120e..2c3f4989a 100644 --- a/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md +++ b/Rx.NET/Documentation/adr/0003-windows-tfms-and-desktop-framework.md @@ -69,7 +69,7 @@ Draft To decide on a good solution, we need to take a lot of information into account. It is first necessary to characterise [the problem](#the-problem) clearly. It is also necessary to understand [the history that led up to the problem](#the-road-to-the-current-problem), and the [constraints that any solution must fulfil](#constraints). The [workaround](#the-workaround) needs to be understood in detail because it will be the interim solution for many years. -We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and them must each be evaluated in the light of all the other information. +We [started a public discussion](https://github.com/dotnet/reactive/discussions/2038) of this problem, and have received a great deal of [useful input from the Rx.NET community](#community-input). There are [several ways we could try to solve this](#the-design-options), and they must each be evaluated in the light of all the other information. The following sections address all of this before moving onto a [decision](#decision). @@ -113,7 +113,7 @@ This problem arose from a series of changes made about half a decade ago that we The first public previews of Rx appeared back in 2009 before NuGet was a thing. This meant Rx was initially distributed in the old-fashioned way: you installed an SDK on development machines that made Rx's assemblies available for local development, and had to arrange to copy the necessary redistributable files onto target machines as part of your application's installation or deployment process. By the time [the first supported Rx release shipped in June 2011](https://web.archive.org/web/20110810091849/http://www.microsoft.com/download/en/details.aspx?id=26649), NuGet did exist, but it was early days, so for quite a while Rx had [two official distribution channels: NuGet and an installable SDK](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5). -There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled as part of the OS. Windows 8 had its own version of .NET that worked quite differently from anything else. These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: +There were several different versions of .NET around at this time besides the .NET Framework. (This was a long time before .NET Core, by the way.) Silverlight and Windows Phone both had their own runtimes, and the latter had a version of Rx preinstalled as part of the OS. Windows 8 had its own version of .NET that worked quite differently from anything else. (This is why you will occasionally see `netcore45` or `netcore451` TFMs despite the fact that .NET Core skipped from v3.1 straight to .NET 5.0. It's also why the first few .NET Core TFMs all have `app` in their names, e.g. `netcoreapp1.0`. By this time, `netcore` meant the Windows 8 store app version of .NET, so they had to use a different name.) These all had very different subsections of the .NET runtime class libraries, especially when it came to threading support. Rx was slightly different on each of these platforms because attempting to target the lowest common denominator would have meant sub-optimal performance everywhere. There were two main ways in which each of the different Rx versions varied: * The scheduler support was specialized to work as well as possible on each distinct target * Each platform had a different UI framework (or frameworks) available, so Rx's UI framework integration was different for each target @@ -150,7 +150,7 @@ Each of these subfolders of each NuGet package's `lib` folder contains a version This fragmentation caused [a problem with plug-in systems (#97)](https://github.com/dotnet/reactive/issues/97). People often ran into this when writing extensions for Visual Studio. Visual Studio was a common place to have these problems simply because a lot of people wrote extensions for it, and it was common for any one user to use a lot of plug-ins, but any .NET Framework based application with a plug-in based extensibility mechanism could have the same problems. -If one plug-in was written to use Rx.NET 2.2.0 and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these pages. +If one plug-in was written to use Rx.NET 2.2.0 and if that plug-in was compiled for .NET Framework 4.0, deploying that plug-in would entail providing a copy of the assemblies from the `net40` folder of each of the four packages referenced by `Rx-Main`. If another plug-in was also written to use the same version of Rx.NET but was compiled for .NET Framework 4.5, its deployment files would include the DLLs from the `net45` folders of each of these packages. Visual Studio is capable of loading components compiled for older versions of .NET Framework, so a version of Visual Studio running on .NET Framework 4.5 would happily load either of these plug-ins. But if it ended up loading both, that would mean that each plug-in was trying to supply its own set of Rx DLLs. That caused a problem. @@ -172,11 +172,11 @@ Here's what would happen. Let's say a we have two plug-ins, `PlugInOneBuiltFor40 (Visual Studio uses a more complex folder layout in reality, but that's not significant. _Any_ plug-in host will have the same issue.) -The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one built for .NET 4.5. Crucially, _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are for Rx 2.2.0.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Interfaces.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. +The critical thing to notice here is that for each of the four Rx assemblies, we have two copies, one built for .NET 4.0 and one built for .NET 4.5. Crucially, _they have the same version number_. In all cases they come from a NuGet package with version 2.2.0. But for the problems I'm describing, what matters more is the .NET assembly version numbers. (.NET versioning is a separate mechanism from NuGet versioning. There's no rule requiring these two version numbers to be related in any way, although by convention they often are, and they are for Rx 2.2.0.) The assembly version numbers are all 2.2.0.0. (.NET assemblies have 4 parts, one more than NuGet packages. But in Rx 2.2.0, the 4th part of the .NET assembly version was always set to 0.) The vital thing to understand here is that for any Rx component, e.g. `System.Reactive.Core.dll`, we have two _different_ copies (a .NET 4.0 and a .NET 4.5 one) but they have _exactly the same name in .NET_. -Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Interfaces`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Interfaces` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, this already-loaded copy will be the `net40` version, but the assembly resolver doesn't know that it's different from what `PlugInTwoBuildFor45` wants. +Let's see why that causes a problem. Suppose Visual Studio happens to load `PlugInOneBuiltFor40` first. That will be able to use its copies of the Rx assemblies. But when the second plug-in, `PlugInTwoBuiltFor45`, first attempts to use `System.Reactive.Core`, the .NET assembly resolver would notice that it has already loaded an assembly named `System.Reactive.Core` with version number 2.2.0.0, the exact version `PlugInTwoBuiltFor45` is asking for. In the scenario I'm describing, this already-loaded copy will be the `net40` version, but the assembly resolver doesn't know that it's different from what `PlugInTwoBuildFor45` wants. -The .NET Framework assembly resolver assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, we fail to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Interfaces` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. As it happens, these have the same public API surface area, so in this particular case we wouldn't get `TypeLoadException` or `MissingMethodException` failures. But there is a behavioural difference. It's quite an obscure one, relating to whether an [`OperationCanceledException`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception) reports the correct [`CancellationToken`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception.cancellationtoken) when you use Rx's `ToTask` or `ForEachAsync`. (As far as I can tell, this is the only respect in which the `net40` and `net45` versions of Rx were different at that time.) If `PlugInTwoBuiltFor45` depended on the correct behavior here, that would be a problem because it would end up using the `net40` version, and it was not possible to implement this correctly on .NET Framework 4.0. +The .NET Framework assembly resolver assumes that the full name (the combination of simple name, version, public key token, and culture) uniquely identifies an assembly. By supplying two different assemblies that have exactly the same full name, Rx 2.2.0 fails to comply with that basic assumption, so the assembly resolver doesn't do what we want. It doesn't even bother to look at the copy of `System.Reactive.Core` in the `PlugInTwoBuiltFor45` folder, because it already has an assembly with the right name in memory. The second component ends up using the `net40` version, and not the `net45` version it shipped. As it happens, these have the same public API surface area, so in this particular case we wouldn't get `TypeLoadException` or `MissingMethodException` failures. But there is a behavioural difference. It's quite an obscure one, relating to whether an [`OperationCanceledException`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception) reports the correct [`CancellationToken`](https://learn.microsoft.com/en-us/dotnet/api/system.operationcanceledexception.cancellationtoken) when you use Rx's `ToTask` or `ForEachAsync`. (As far as I can tell, this is the only respect in which the `net40` and `net45` versions of Rx were different at that time.) If `PlugInTwoBuiltFor45` depended on the correct behavior here, that would be a problem because it would end up using the `net40` version, and it was not possible to implement this correctly on .NET Framework 4.0. Although this was an extremely specific problem, the bigger problem was that if future versions of Rx ended up with greater divergences on different .NET Framework versions, plug-ins wanting newer versions could well end up encountering `TypeLoadException` or `MissingMethodException` failures as a result of not getting the version they require. @@ -208,9 +208,9 @@ It's worth noting at this point that the problem I've just described doesn't nee By this time Rx.NET was no longer building .NET 4.0 versions, but it did offer `net45`, `net451`, `net462`, and `net463` versions. So in a suitably updated version of the plug-in scenario described above, imagine we have `PlugInTwoBuiltFor45` and `PlugInThreeBuiltfor46` both using Rx v3.1.1. `PlugInTwoBuiltFor45` would be using versions of the Rx components with a .NET assembly version of `3.0.1000.0`, while `PlugInThreeBuiltfor46` would be using version `3.0.3000.0`. The .NET Framework assembly resolver would consider these to be distinct assemblies because they have different full names, so it would happily load both versions simultaneously, avoiding the problem. -This change predates .NET Core/modern .NET, and this newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than equal to the requested version to be a match. A basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) Fortunately, it typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue on the newer runtimes that have this different versioning behavior. +This change predates .NET Core/modern .NET, and this newer lineage of runtimes has a different approach to assembly versioning: whereas .NET Framework requires a strict version match, .NET Core and its successors (e.g. .NET 6.0, .NET 8.0) consider any assembly with a version number greater than or equal to the requested version to be a match. A basic assumption of this Rx 3.0 versioning tactic—that the assembly resolver wants an exact match on the version—is no longer true on all versions of .NET. (A common theme of the problems described in this ADR is that many decisions were based on assumptions that were valid at the time but no longer are.) Fortunately, it typically doesn't matter for plug-in scenarios because the `AssemblyLoadContext` side-steps this whole issue on the newer runtimes that have this different versioning behavior. -Unfortunately, Rx 3.1's change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.1000.0 versions do not). +Unfortunately, Rx 3.1's change in version numbering went on to cause various new issues. There's [a partial list of these issues in a comment in issue 199](https://github.com/dotnet/reactive/issues/199#issuecomment-266138120), and if you look through [#205](https://github.com/dotnet/reactive/issues/205) you'll see a few links to other problems. Even at the time this change was proposed, it was [acknowledged that there was a potential problem with binding redirects](https://github.com/dotnet/reactive/issues/205#issuecomment-228577028). Binding redirects often specify version ranges, which means if you upgrade 3.x to 4.x, it's possible that 3.0.2000.0 would get upgraded to 4.0.1000.0, which could actually mean a downgrade in surface area (because the x.x.2000.0 versions might have target-specific functionality that the x.x.1000.0 versions do not). As happens quite a lot in the history of this problem, something that worked fine in a simple set up turned out to have issues when dependency trees got more complex. Applications (or plug-ins) using Rx directly had no problems, but if you were using multiple components that depended on Rx, and if those components had support for different mixtures of targets, you could hit problems. @@ -245,7 +245,7 @@ That sounds very convenient, but it turned out to be a simplification too far. #### Problems arising from the great unification -The _great unification_ worked fine until .NET Core 3.0 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.0, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. +The _great unification_ worked fine until .NET Core 3.0 came out. That threw a spanner in the works, because it undermined a basic assumption that the _great unification_ made: the assumption that your target runtime would determine what UI application frameworks were available. Before .NET Core 3.0, the availability of a UI framework was determined entirely by which runtime you were using. If you were on .NET Framework, both WPF and Windows Forms would be available, and if you were running on any other .NET runtime, they would be unavailable. If you were running on the oddball version of .NET available on UWP (which, confusingly, is associated with TFMs starting with `uap`, and even more confusingly, is also associated with the `netcore50` TFM) the only UI framework available would be the UWP one, and that wasn't available on any other runtime. But .NET Core 3.0 ended that simple relationship. Consider this table: @@ -272,7 +272,7 @@ Why have I put "None" in the middle column of the `netcoreapp3.0` row, bearing i As part of Rx.NET's [preparation for .NET 5 support](https://github.com/dotnet/reactive/pull/1291), a `net5.0` target was added. This did **not** include Windows Forms and WPF features. That is unarguably correct, because if you were to create a new project targeting `net5.0` and set either `UseWPF` or `UseWindowsForms` (or both) to `true` you'd get a build error telling you that you can only do that when the target platform is Windows. It recommends that you use an OS-specific TFM, such as `net5.0-windows`. -Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs changed in .NET 5.0](https://github.com/dotnet/designs/blob/main/accepted/2020/net5/net5.md). We didn't have OS-specific TFMs before .NET 5.0. So with .NET 5.0 and later, we can append `-windows` to indicate that we need to run on Windows. Since there was no way to do that before, `netcoreapp3.0` doesn't tell you anything about what the target OS needs to be. +Why is it like this for .NET 5.0, but not .NET Core 3.0? It's because [TFMs changed in .NET 5.0](https://github.com/dotnet/designs/blob/main/accepted/2020/net5/net5.md). OS-specific TFMs did not exist before .NET 5.0. So with .NET 5.0 and later, we can append `-windows` to indicate that we need to run on Windows. Since there was no way to do that before, `netcoreapp3.0` doesn't tell you anything about what the target OS needs to be. My view is that since the `netcoreapp3.0` TFM doesn't enable you to know whether Windows Forms and WPF will necessarily be available, that it would be better not to ship a component with this TFM that requires that it will be available (unless that component is specifically designed to be used only in environments where these frameworks will be available). That's why I put "None" in the 2nd column for that row. However, it seems like when Rx team added .NET Core 3.0 support, they chose a maximalist interpretation of their concept that a reference to `System.Reactive` means that you get access to all Rx functionality that is applicable to your target. Since running on .NET Core 3.0 _might_ mean that Windows Forms and WPF are available, Rx decides it _will_ include its support for that. @@ -326,7 +326,7 @@ But Rx 5.0 takes the position that if an applications targets Windows, Rx should The problem with that is that if you use any self-contained form of deployment (including Native AOT) in which the .NET runtime and its libraries are shipped as part of the application, that means your application will be shipping the WPF and Windows Forms parts of the .NET runtime library. Normally those are optional—the basic .NET runtime does not include them—so this is not a case of "well you'd be doing that anyway." -Let's look at the impact. The first column of the following table shows the size of the deployable output for the code shown above (excluding debug symbols; these will be present in the published output by default but including them here skew the results for the smaller outputs). The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. +Let's look at the impact. The first column of the following table shows the size of the deployable output for the code shown above (excluding debug symbols; these will be present in the published output by default but including them here skews the results for the smaller outputs). The second column shows the impact of adding a reference to `System.Reactive` and writing a single line of code that uses it (to ensure that Rx doesn't get removed due to not really being used), but for that column I targetted `net80-windows10.0.18362`. Remember, Rx doesn't support WPF or Windows Forms for versions before 10.0.19041, so this shows the impact of adding Rx without its WPF or Windows Forms support. As you can see, it adds a little over a megabyte in the first two rows—the size of `System.Reactive.dll` in fact—and in the last two rows it has a smaller impact because trimming can remove most of that. | Deployment type | Size without Rx | Size with Rx targeting 18362 | Size with Rx targeting 19041 | |--|--|--|--| @@ -337,7 +337,7 @@ Let's look at the impact. The first column of the following table shows the size But the third column looks very different. In this case I've targetted `net8.0-windows10.0.19041.0`, the oldest Windows version for which Rx offers support on .NET 6.0 and later. Rx has decided that since it is able to provide Windows Forms and WPF support for that target, it _will_ provide it, even though nothing in my code actually uses it. -In the framework-dependent row it makes only a small difference (because the copy of `System.Reactive.dll` we get is a little larger). But that's misleading: the resulting executable will now required host systems to have not just the basic .NET 8.0 runtime installed, but also the optional Windows Desktop components. So unless the target machine already has that installed, I will in fact have a larger install to perform. +In the framework-dependent row it makes only a small difference (because the copy of `System.Reactive.dll` we get is a little larger). But that's misleading: the resulting executable will now require host systems to have not just the basic .NET 8.0 runtime installed, but also the optional Windows Desktop components. So unless the target machine already has that installed, I will in fact have a larger install to perform. The self-contained deployment is the worst. It has roughly doubled in size—it is 90MB larger! And for absolutely no change in behaviour. I compiled exactly the same code for the last two columns, it's just that in the 18362 column I chose a target runtime that would prevent Rx from trying to offer the Windows Forms and WPF support that I'm not using. @@ -351,7 +351,7 @@ But it's still not great. And there are lots of scenarios in which Native AOT si (In case you're wondering why the framework-dependent deployment is so large, 20.8MB, most of that is the `Microsoft.Windows.SDK.NET.dll` library. This gets included as a result of using a Windows-specific TFM, and using some of the WinRT-style APIs that it makes available. That library is where the types such as `MouseCapabilities` my example uses come from.) -So this is why, in the earlier table, I said that for the `net5.0-windows10.0.19401` the answer to the question "Which UI framework should Rx support?" should be "None." But why did I qualify it as "probably?" It's because I think they might not have had a choice: they had already painted themselves into a corner by this time. In order to avoid this, they would have had to have designed Rx 4 differently, and that ship had already sailed. +So this is why, in the earlier table, I said that for the `net5.0-windows10.0.19401` the answer to the question "Which UI framework should Rx support?" should be "None." But why did I qualify it as "probably?" It's because I think the people maintaining Rx.NET back then might not have had a choice: they had already painted themselves into a corner by this time. In order to avoid this, they would have had to have designed Rx 4 differently, and that ship had already sailed. In my view, the best solution to this whole problem would have been for all of the UI-frameworks-specific pieces of Rx to remain in separate libraries. Although the simplicity of getting all Rx can offer with a single package reference is appealing, we simply wouldn't have the problem we have today if the framework-specific pieces had remained separate. @@ -376,9 +376,9 @@ Remember, the basic plug-in problem occurs when a single version of Rx contains In short, there is no version of .NET Framework for which the build system will select the `netstandard2.0` component. Older versions of .NET don't support .NET Standard 2.0. And for newer versions, it will always pick the `net46` library. -Unfortunately, it's different in Rx 5.0. +Unfortunately, it's different in Rx 5.0. The only .NET Framework TFM offered by Rx 5.0 is `net472`. -Which Rx target will be used when we target .NET Framework versions 4.6.2, 4.7, or 4.7.1? None of these can load the `net472` target because that required .NET Framework 4.7.2 or later. But they can all load the `netstandard2.0` one. +Which Rx target will be used when we target .NET Framework versions 4.6.2, 4.7, or 4.7.1? None of these can load the `net472` target because that requires .NET Framework 4.7.2 or later. But they can all load the `netstandard2.0` one. This opens the door to a plug-in problem. If someone built a Visual Studio plug-in targetting .NET Framework 4.6.2 that uses Rx 5.0, that plug-in would include a copy of the `netstandard2.0` copy of `System.Reactive.dll`. A plug-in targetting .NET Framework 4.7.2 that also uses Rx 5.0 will include a copy of the `net472` DLL. If that first plug-in loads first, it will cause the `netstandard2.0` DLL to load, and since that has exactly the same strong name as the `net472` DLL, the second plug-in is also going to get that `netstandard2.0` one. So if that second plug-in tries to use, say, Rx's WPF features, it will fail with a `TypeLoadException` or `MissingMethodException`. @@ -391,7 +391,7 @@ In the unlikely event of needing to write a new plug-in that targets a version o ### The workaround -If your application has encountered [the problem](#the-problem), you can add this to the `csproj`: +If your application has encountered [the problem](#the-problem) (an unasked for and problematic dependency on WPF and Windows Forms) you can add this to the `csproj`: ```xml @@ -401,7 +401,7 @@ If your application has encountered [the problem](#the-problem), you can add thi This only needs to go in the project that builds your application's executable output. It does not need to go in every project—if you've split code across multiple libraries, those don't need to have this. Nor does it need to go into NuGet packages. The problem afflicts only executables, not DLLs. -here's an updated version of the table from the previous section. The final two columns are for the same application as last time, targeting `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. +Here's an updated version of the table from the previous section. The final two columns are for the same application as last time, targeting `net8.0-windows10.0.19041.0`. One shows the same values as the final column from the previous section, in which Rx has brought in Windows Forms and WPF. The final column here shows the effect of applying the workaround. | Deployment type | Size without Rx | Size with Rx, no workaround | Size with Rx and workaround | |--|--|--|--| @@ -443,7 +443,7 @@ Even in a world of gigabit internet, web page download sizes matter. We added tr But regardless of what the reasons might be, people are demonstrably worried enough about disk space in 2023 to drop Rx. And that means we do actually have to do something. -As for Anais's second point, no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed here in the [next section](#the-peculiar-faith-in-the-power-of-breaking-changes)) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. +As for Anais's final point, no, I do not want to nuke anything. It's possible Anais had read some of the other comments (discussed here in the [next section](#the-peculiar-faith-in-the-power-of-breaking-changes)) where there seemed to be a surprising appetite for completely abandoning all hope of compatibility. Seen in that context, the extreme language of "nuking" is understandable. And while I disagree with the premise that 90MB is no big deal, I think Anais offers some extremely important analysis. She worries that a solution to this problem could be: @@ -483,11 +483,11 @@ It's also worth pointing out that not all breaking changes are created equal. He These are all, technically, binary breaking changes, but they are very different from one another. -For example, consider the nature of one of the binary breaking changes in `FileStream`: this class has become much less aggressive about synchronizing some of its internal state with the operating system. This has dramatically improved performance in certain scenarios, but it does mean any code written for .NET Core 3.1 that made certain assumptions about exactly how `FileStream` was implemented might break on .NET 8.0. Microsoft categorises this as a binary breaking change, but the reality is that the overwhelming majority of users of `FileStream` will be unaffected. Their code will run a little faster but nothing else will change. Microsoft went to great lengths to minimize any practical incompatibilities, and also, for a few releases, they provided a way to revert to the old behaviour. This was carefully thought out, there was a multi-release plan for how the new behaviour would be phased in, with safeguards in place in case the negative impact was worse than expected, and with escape hatches for anyone adversely affected. While this is technically a breaking change, most users of this class will be unaffected by it. +For example, consider the nature of one of the binary breaking changes in `FileStream`: this class has become much less aggressive about synchronizing some of its internal state with the operating system. This has dramatically improved performance in certain scenarios, but it does mean any code written for .NET Core 3.1 that made certain assumptions about exactly how `FileStream` was implemented might break on .NET 8.0. Microsoft categorises this as a binary breaking change, but the reality is that the overwhelming majority of users of `FileStream` will be unaffected. Their code will run a little faster but nothing else will change. Microsoft went to great lengths to minimize any practical incompatibilities, and also, for a few releases, they provided a way to revert to the old behaviour. This was carefully thought out, there was a multi-release plan for how the new behaviour would be phased in, with safeguards in place in case the negative impact was worse than expected, and with escape hatches for anyone adversely affected. While this is technically a breaking change, most users of this class will experience no negative effects. The second case, the gradual removal of CLR serialization, is quite different. In this case, functionality is being removed. Anyone dependent on that functionality will be out of luck once it goes entirely. This is being done because CLR serialization is a security liability, and it's a feature nobody should really be using. But lots of people have used it in the past, which is why it was brought over from .NET Framework to .NET. So although this is a more brutal kind of breaking change than 1) above, it is being handled in a way designed to give developers using it a very long runway for finally breaking free of it. There is a [published plan](https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md) for how it will be phased out. There has been a phased approach in which there were initially just warnings, and then a change where it was disabled by default but could easily be re-enabled. It will eventually vanish completely, but we had years of notice that it was on the way out. (And even then, there will still be an unsupported NuGet package that implements the behaviour if you really want it.) Even though CLR serialization was re-introduced in .NET Core explicitly as a stopgap measure, very clearly signposted as something intended only to support porting of code from .NET Framework, and not something to be used in new development, the deprecation was done over about half a decade, and 5 releases of .NET. -Now compare that with 3), the idea that we should relax about backwards compatibility and just remove the problematic APIs from Rx. That very different from 2) (which in turn is very different from 1). This would be a sudden shock for existing users of Rx. This is absolutely guaranteed to cause problems for anyone who was using Rx in a way that it was very much designed to be used. That's a very different sort of breaking change from one that only affects people who knowingly used a doomed API, or one that doesn't affect anyone using an API normally. +Now compare that with 3), the idea that we should relax about backwards compatibility and just remove the problematic APIs from Rx. That very different from 2) (which in turn is very different from 1). This would be a sudden shock for existing users of Rx. This is absolutely guaranteed to cause problems for anyone who was using Rx in a way that it was very much designed to be used. That's a very different sort of breaking change from one that only affects people who knowingly used a doomed API, or one that doesn't affect anyone using an API in normal ways. I should clarify that we're not totally opposed to breaking changes, we just want to ensure the following: @@ -529,9 +529,9 @@ I don't believe Rx.NET can impose the kind of split that Microsoft did by introd As a developer writing a library, I'm not making that kind of choice when I decide to use, say, `System.Text.RegularExpressions`. Or `List`. Or, and I think this is probably actually the most relevant comparison for Rx, LINQ to Objects. Or any of the other library types which are nearly identical across the .NET FX and modern .NET worlds. -So the question we need to ask is this: is Rx more like a framework or a common library feature? Are we more like ASP.NET Core, or `System.Linq`? I'd say the fact that we offer a `netstandard2.0` target points very much towards the latter. +So the question we need to ask is this: is Rx more like a framework or a common library feature? Are we more like ASP.NET Core, or `System.Linq`? I'd say the fact that we offer a `netstandard2.0` target points very much towards the latter. (Also, the very nature of Rx.NET is that it makes us a lot like `System.Linq`: we are a LINQ provider with aspirations of universal applicability.) -As already discussed in [the section on the appetite for breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), when you look at breaking changes in .NET itself, although framework changes often are disruptive, changes to general-purpose library features typically try to preserve compatibility as much as is feasible even when the change is officially categorised as a binary breaking change. And it's like this because these kinds of library features are very often used quietly by other libraries as an implementation detail. They get imposed on an application from the bottom up, giving application authors much less control over what versions they use. So standards of compatibility need to be higher for this kind of library. +As already discussed in [the section on the appetite for breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), when you look at breaking changes in .NET itself, although framework changes often are disruptive, changes to general-purpose library features (like `FileStream`) typically try to preserve compatibility as much as is feasible even when the change is officially categorised as a binary breaking change. And it's like this because these kinds of library features are very often used quietly by other libraries as an implementation detail. They get imposed on an application from the bottom up, giving application authors much less control over what versions they use. So standards of compatibility need to be higher for this kind of library. We're not actually holding Rx.NET to as a high a level as the .NET runtime class libraries in this regard. But we do need to be careful not to make changes that are absolutely certain to cause problems for large classes of users. @@ -561,7 +561,7 @@ So in general, specifying a Windows 10.0.19041 TFM doesn't really cause any prob That said there's one possible benefit: specifying a Windows 10 version-specific TFM does result in an interop DLL that's roughly 20MB in size being included in your application. Arguably the ability to remove that would be a benefit. -Currently, I don't know if this is possible. When [endjin](https://endjin.com) took over maintenance of Rx.NET, we did try to find out why the Windows-specific Rx.NET targets chose 10.0.19041. I never got an answer. I think we can't go back any further than 10.0.17763 because I _think_ [C#/WinRT](https://learn.microsoft.com/en-us/windows/apps/develop/platform/csharp-winrt/) requires that version or later, and in practice 10.0.18362 might be the lowest we can use in practice because that's the oldest SDK that Visual Studio 2022 supports. +Currently, I don't know if this is possible. When [endjin](https://endjin.com) took over maintenance of Rx.NET, we did try to find out why the Windows-specific Rx.NET targets chose 10.0.19041. I never got an answer. (My best hypothesis is that 10.0.19041 was the latest available SDK at the point when Rx.NET first added a `-windows` TFM, and it was chosen not because Rx.NET needs it, but just because that was _current_.) We can't go back any further than 10.0.17763 because [C#/WinRT](https://learn.microsoft.com/en-us/windows/apps/develop/platform/csharp-winrt/) requires that version or later, and 10.0.18362 seems to be the lowest we can use in practice because that's the oldest SDK that Visual Studio 2022 supports. #### Use obsolete @@ -581,9 +581,9 @@ There are a few constraints that we need to impose on any possible solution to t #### Can't remove types until a long Obsolete period -The simplest thing we could do to solve the main problem this document describes would be to remove all UI-framework-specific types from the public API surface area of `System.Reactive`. This would entail simply removing the `net6.0-windows10.0.19041`, `net472`, and `uap10.0.18362` targets. Applications using .NET 6.0 or later would get the `net6.0` target, and everything else would use the `netstandard2.0` target. The UI-framework-specific types could be moved into UI-specific NuGet packages, and we'd also need a Windows Runtime package as a home for Windows-specific but non-UI-framework-specific features such as integration with `IAsyncOperation`. Applications would not simply be left in the lurch: all functionality would remain available, it would simply be distributed slightly differently. +The simplest thing we could do to solve the main problem this document describes would be to remove all UI-framework-specific types from the public API surface area of `System.Reactive`. This would mean dropping the `net6.0-windows10.0.19041`, `net472`, and `uap10.0.18362` targets. Applications using .NET 6.0 or later would get the `net6.0` target, and everything else would use the `netstandard2.0` target. The UI-framework-specific types could be moved into UI-specific NuGet packages, and we'd also need a Windows Runtime package as a home for Windows-specific but non-UI-framework-specific features such as integration with `IAsyncOperation`. Applications would not be left in the lurch: all functionality would remain available, it would just be distributed slightly differently. -Unfortunately, this would create some serious new problems. Consider an application that depends on two libraries that use different versions of Rx. Let's suppose LibraryBefore depends on Rx 6.0, and LibraryAfter depends on some hypothetical future Rx 7.0 that makes the change just described. So we have this sort of dependency tree: +Unfortunately, this would create some serious new problems. Consider an application that depends on two libraries that use different versions of Rx. Let's suppose `LibraryBefore` depends on Rx 6.0, and `LibraryAfter` depends on some hypothetical future Rx 7.0 that makes the change just described. So we have this sort of dependency tree: * `MyApp` * `LibraryBefore` @@ -600,7 +600,7 @@ This creates a problem because of what actually gets compiled into components th call class [System.Runtime]System.IObservable`1 [System.Reactive]System.Reactive.Linq.DispatcherObservable::ObserveOnDispatcher(class [System.Runtime]System.IObservable`1) ``` -If you're not familiar with .NET's IL, I'll just break that down for you. the `call class` part indicates that we're calling a method defined by a class. The `call` instruction needs to identify a specific method. The raw binary for the IL does this with a metadata token—essentially a reference to a particular row in a table of methods. Compiled .NET components contain what is essentially a small, highly specialized relational database, and one of the tables is a list of every single method used by the component. A `call` instruction incorporates a number that's effectively an offset into that table. (I've left out a complication caused by the distinction between internally defined methods and imported ones, but that's more detail than is necessary here.) +If you're not familiar with .NET's IL, I'll just break that down for you. the `call class` part indicates that we're calling a method defined by a class. The `call` instruction needs to identify a specific method. The raw binary for the IL does this with a metadata token—essentially a reference to a particular row in a table of methods. Compiled .NET components contain what is essentially a small, highly specialized relational database, and one of the tables is a list of every single method used by the component. A `call` instruction incorporates a number that's effectively an offset into that table. (I've left out a complication caused by the distinction between internally defined methods and imported ones, because that's more detail than is necessary here.) The IL shown above is how ILDASM, the IL disassembler, interprets it for us. Instead of just showing us the metadata token, it goes and finds the relevant row in the table. In fact it finds a bunch of related rows—there's a table for parameters, and it also has to go and find all of the rows corresponding to the various types referred to: in this case there's the return type, the type of the one and only normal parameter, and also a type argument because this is a generic method. @@ -615,11 +615,11 @@ This essentially says that the method we want is: 1. defined in the `System.Reactive` assembly 2. defined in the `System.Reactive.Linq.DispatcherObservable` class in that assembly 3. called `ObserveOnDispatcher` -4. a generic method with one type parameter, and we want to use `int32` (what C# calls `int`) as the argument to that type +4. a generic method with one type parameter, and we want to use `int32` (what C# calls `int`) as the type argument It's point 1 that matters here. This indicates that the method is defined in `System.Reactive`. That's what's going to cause us problems in this scenario. But why? -Let's applying this to our example. We've established that `LibraryBefore` is going to contain at least one IL `call` instruction that indicates that it expects to find the `ObserveOnDispatcher` method in the `System.Reactive` assembly. +Let's apply this to our example. We've established that `LibraryBefore` is going to contain at least one IL `call` instruction that indicates that it expects to find the `ObserveOnDispatcher` method in the `System.Reactive` assembly. What's `LibraryAfter` going to look like? Remember in this hypothetical scenario, Rx 7.0 has moved all WPF-specific types out of `System.Reactive` and into some new component we're calling `System.Reactive.Wpf` in this example. So code in `LibraryAfter` calling the exact same method (the `DispatcherObservable` class's `ObserveOnDispatcher` extension method) would look like this in IL: @@ -651,7 +651,7 @@ What does that mean for the `LibraryBefore`? Well if it happens never to run the The JIT compiler will then inspect the `System.Reactive` assembly and discover that it does not define a type called `System.Reactive.Linq.DispatcherObservable`. The JIT compiler will then throw a `TypeLoadException` to report that the IL refers to a method in a type that does not in fact exist. -And that's why we can't just remove types from `System.Reactive`. Code built against Rx.NET 6.0 might break if used a project where some other component uses Rx.NET 7.0. +And that's why we can't just remove types from `System.Reactive`. Code built against Rx.NET 6.0 might break if used in a project where some other component uses Rx.NET 7.0. Type forwarders seem like they might offer a solution, but unfortunately not. If we want to move `ObserveOnDispatcher` out of `System.Reactive` and into `System.Reactive.Wpf`, in theory we could do that in a backwards compatible way by adding a type forwarding entry for the `System.Reactive.Linq.DispatcherObservable`. Basically `System.Reactive` contains a note telling the CLR "I know you've been told that this type is in this assembly, but it's actually in `System.Reactive.Wpf`, so please look there instead." @@ -661,7 +661,7 @@ And that gets us back to square one: if taking a dependency on `System.Reactive` It would be technically possible to meddle with the build system's normal behaviour in order to produce a `System.Reactive` assembly with a suitable type forwarder, but for the resulting NuGet package not to have the corresponding dependency. However, this is unsupported, and is likely to cause a lot of confusion for people who actually do want the WPF functionality, because adding a reference to just `System.Reactive` (which has been all that has been required for Rx v4 through v6) would still enable code using WPF features to compile when upgrading to this hypothetical form of v7, but it would result in runtime errors due to the `System.Reactive.Wpf` assembly not being found. So this is not an acceptable workaround. -In short, any time we introduce a breaking change, we don't just create a situation where applications need to fix a few things when they upgrade. We create a situation where using certain combinations of other libraries become unusable. The only ways to resolve this are either to hope all your libraries get upgraded soon, or to decide not to use all of the libraries you want to. +In short, any time we introduce a breaking change, we don't just create a situation where applications need to fix a few things when they upgrade. We create a situation where using certain combinations of other libraries become **unusable**. The only ways to resolve this are either to hope all your libraries get upgraded soon, or to decide not to use all of the libraries you want to. As is often the case, it's the dependency scenarios that cause trouble. For this reason, our goal is that upgrading from Rx 6.0 to Rx 7.0 should not be a breaking change. The rules of semantic versioning do in fact permit breaking changes, but because Rx.NET defines types in `System.*` namespaces, and because a lot of people don't seem to have realised that it has not been a Microsoft-supported project for many years now, people have very high expectations about backwards compatibility. @@ -675,7 +675,7 @@ We are considering making an exception to the constraint just discussed for UWP. We don't want to drop UWP support completely, but we are prepared to contemplate removing the UWP-specific target (`uap10.0.16299`) much earlier than any of the other targets, possibly right away. UWP has long supported .NET Standard 2.0, so Rx.NET would still be available if we did this. The only change would be that the UWP-specific types would no longer be in `System.Reactive`. (We would move them into a separate NuGet package.) -This is problematic for all of the reasons just discussed in the preceding section. However, as far as we know UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. +This is problematic for all of the reasons just discussed in the preceding section. However, UWP never really became hugely popular, and the fact that Microsoft never added proper support for it to the .NET SDK sets a precedent that makes us comfortable with dropping it relatively abruptly. Existing Rx.NET users using UWP will have two choices: 1) remain on Rx 6.0, or 2) rebuild code that was using UWP-specific types in `System.Reactive` to use the new UWP-specific package we would be adding. There are a couple of options here: @@ -690,15 +690,15 @@ In either case, we would retain an emergency fallback position: if it turns out Consider an application MyApp that has two dependencies (which might be indirect, non-obvious dependencies) of this form: -* UiLibA uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` -* NonUiLibB uses Rx 7, and it does not use any UI-framework-specific Rx features but it does use new functionality in Rx 7 +* `UiLibA` uses `System.Reactive` v5.0. This is a UI library and it and depends on `DispatcherScheduler` +* `NonUiLibB` uses Rx 7, and it does not use any UI-framework-specific Rx features but it does use new functionality in Rx 7 Imagine for the sake of argument that Rx 7 adds a `RateLimit` operator to deal with the fact that almost nobody likes `Throttle`. So we now have this situation: -* UiLibA will require `DispatcherScheduler` to be available through `System.Reactive` v5.0 -* NonUiLibB will require `Observable.RateLimit` to be available through whatever reference it's using to get Rx 7 +* `UiLibA` will require `DispatcherScheduler` to be available through `System.Reactive` +* `NonUiLibB` will require `Observable.RateLimit` to be available through whatever reference it's using to get Rx 7 This needs to work. This constraint prevents us from simply removing UI-specific types from `System.Reactive` in v7.0. @@ -711,7 +711,7 @@ Let's start by picking some upper and lower bounds as a starting point. These ar * If a project upgrades from Rx.NET 1.0 to 7.0, it is probably reasonable for some things to break * Components with a dependency on Rx 5.0 or 6.0 running in an application that upgrades that to Rx 7.0 must not be broken by this upgrade -That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. (That said, some people seem to be [positively enthusiastic about breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), and would actually disgree with the lower bound I've put here. But our view is that Rx 5.0 was still the very latest version as recently as 18th May 2023. Less than a year has passed, which seems like not nearly enough time to be removing API features that even in the current version are not marked as obsolete. So although we recognize that there are some people who think it would be OK to break things more quickly than the lower bound stated above suggests, we aren't going to do that, meaning that this is a lower bound in practice.) +That's a very incomplete picture, but it establishes that the acceptable position lies somewhere between those two extremes. (That said, some people seem to be [positively enthusiastic about breaking changes](#the-peculiar-faith-in-the-power-of-breaking-changes), and would actually disgree with the lower bound I've put here. But our view is that Rx 5.0 was still the very latest version as recently as 18th May 2023. Less than two years have passed, which seems like not nearly enough time to be removing API features that even in the current version are not marked as obsolete. So although we recognize that there are some people who think it would be OK to break things more quickly than the lower bound stated above suggests, we aren't going to do that, meaning that this is a lower bound in practice.) There's another dimension we need to take into account: @@ -730,7 +730,7 @@ Some would argue that this makes it acceptable for the application to break in t * This might create a situation where the application developers are forced to abandon one or other of the components they want to use * Although the current .NET runtime introduced an exciting world where 3 years counts as "long term support" the .NET Framework still sits in the Windows world in which support typically lasts for 10 years, so it's not entirely unreasonable to expect compatibility with a 5 year old release -Given that the current LTS policy is for .NET to last 3 years, I think the absolute minimum requirement is that if we want to make a breaking change, we can't do it until at least 3 years after marking the relevant API surface area as `[Obsolete]`. But I also think we need to take into account the times at which people are likely to discover that we've added such annotations. .NET 8.0 came out in November 2023, and many applications will already have upgraded to that and may be planning to keep things that way for at least 2 years. So if we were to release a new version of Rx.NET in March 2024, it's possible that developers for some application might not notice we've done that until 2026 when they start getting ready to upgrade to .NET 10.0. So if we progressed from `[Obsolete]` to removing an API entirely three years from now, that looks like just 1 year of notice to those developers. +Given that the current LTS policy is for .NET to last 3 years, I think the absolute minimum requirement is that if we want to make a breaking change, we can't do it until at least 3 years after marking the relevant API surface area as `[Obsolete]`. But I also think we need to take into account the times at which people are likely to discover that we've added such annotations. .NET 8.0 came out in November 2023, and many applications will already have upgraded to that and may be planning to keep things that way for at least 2 years. So if we were to release a new version of Rx.NET in late 2024, it's possible that developers for some application might not notice we've done that until 2026 when they start getting ready to upgrade to .NET 10.0. So if we progressed from `[Obsolete]` to removing an API entirely three years from now, that looks like just 1 year of notice to those developers. This suggests that for obsolescence we announce today, we can't really implement it any earlier than 2029, 5 years from now. @@ -750,12 +750,12 @@ While we would prefer to have no breaking changes at all, there are some circums Simply removing UI-specific types in `System.Reactive` v7 would be a type 2 change (because of [the need for things to work when we have indirect dependencies on multiple versions](#transitive-references-to-different-versions)). -We consider type 2 changes to be unacceptable unless [sufficient time has passed and sufficient warning was given](#minimum-acceptable-path-for-breaking-changes). We consider all breaking changes undesirable, but would be open to type 1 changes on a shorter (e.g. 1-4 year) timescale if no better alternative exists. +We consider type 2 changes to be unacceptable unless [sufficient time has passed and sufficient warning was given](#minimum-acceptable-path-for-breaking-changes) (or a critical security concern makes it absolutely necessary). We consider all breaking changes undesirable, but would be open to type 1 changes on a shorter (e.g. 1-4 year) timescale if no better alternative exists. #### Plug-in systems must work -An earlier draft of this ADR stated that we must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. That was before discovering that Rx 5.0 had already re-introduced it. So we now have the slightly less ambitious target of not making it any worse than it already is. +An earlier draft of this ADR stated that we must not re-introduce the problem in which host applications with a plug-in model can get into a state where plug-ins disagree about which exact DLLs are loaded for a particular version of Rx. That was before discovering that Rx 5.0 had already re-introduced it exactly this problem. So we now have the slightly less ambitious target of not making it any worse than it already is. It's worth re-iterating that this is a non-issue on .NET 6.0+ hosts. Applications hosting plug-ins on those runtimes will use [`AssemblyLoadContext`](https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext), enabling each plug-in to make its own decisions about which components it uses. If plug-ins make conflicting decisions, it doesn't matter, because `AssemblyLoadContext` is designed to allow that. @@ -774,7 +774,7 @@ But there is a more subtle scenario. `PlugInOne` and `PlugInTwo` might both depe The effect of this is to impose the following constraint: within a single major release, there must be both backward and forward compatibility between `System.Reactive` and any UI framework support Rx components. E.g., `System.Reactive` v8.0.X and `System.Reactive.Wpf` v8.0.Y must be able to coexist for any combination of X and Y. -Note that we aren't going to try and fix the problem where `PlugInOne` loading and older version of some Rx.NET assembly prevents `PlugInTwo` from loading a later version of the same assembly when they have the same major version number. That problem exists today, and is fundamentally unavoidable with a .NET Framework plug-in system. +Note that we aren't going to try and fix the problem where `PlugInOne` loading an older version of some Rx.NET assembly prevents `PlugInTwo` from loading a later version of the same assembly when they have the same major version number. That problem exists today, and is fundamentally unavoidable with a .NET Framework plug-in system. #### Coherent versions across multiple components @@ -796,7 +796,7 @@ It is explicitly a non-goal to support `packages.config`. Our position is that p We are not interpreting this to mean that the current position, in which there is only one package (`System.Reactive`) and it serves all purposes must necessarily be preserved. We are currently of the view that it is acceptable for UI-framework-specific support to be in separate components. That would necessarily be the case for frameworks Rx.NET does not provide built-in support for such as Avalonia. And there are now so many UI frameworks for .NET that we think it's a self-evident truth that Rx.NET can't provide built-in support for all of them. And once you accept that some UI-framework-specific support is to be found in separate NuGet packages, it is arguably less confusing if they all work that way. -People have demonstrably been confused by the current (v4 through v6) setup. For all that a single-package approach is simple, people still get tripped up by the highly non-obvious requirement to change their TFM from `net6.0-windows` to `net6.0-windows10.0.19041`. (And over a year after taking over maintenance of this project, I've still not yet managed to find a satisfactory answer as to why it's that particular version. It seems clear that 10.0.18362 is a practical minimum for whichever component ends up containing the Windows Runtime support, because current .NET SDKs don't support older versions, but the choice of 19041 continues to be a mystery.) So I don't know how this is meant to be simple for anyone just getting started with Rx.NET.) +People have demonstrably been confused by the current (v4 through v6) setup. For all that a single-package approach is simple, people still get tripped up by the highly non-obvious requirement to change their TFM from `net6.0-windows` to `net6.0-windows10.0.19041`. (And over a year after taking over maintenance of this project, I've still not yet managed to find a satisfactory answer as to why it's that particular version. It seems clear that 10.0.18362 is a practical minimum for whichever component ends up containing the Windows Runtime support, because current .NET SDKs don't support older versions, but the choice of 19041 continues to be a mystery.) So we are open to multiple packages. But we don't want to let that mean that we open the floodgates to a much more granular approach. @@ -879,11 +879,11 @@ Rx 5.0 and 6.0 have both shipped, and a lot of people use them, so one option is Changing nothing meets many of the [constraints](#constraints). Specifically, it meets all those are all concerned with not making things worse than they already are. This _is_ where we already are, so it can't possibly be any worse than where we are. But the big problem is the one stated at the start of this ADR: the fact that self-contained deployments are vastly bloated by unwanted dependencies on Windows Forms and WPF. -I think this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that. The [workaround](#the-workaround) provides a way to undo this problem, but workarounds usually have their own problems, and we don't want people to have to discover and then apply a fix just to be able to use Rx.NET. I think separating out these parts is the only way to achieve this. +We believe this is a fundamental problem with anything that continues to have a unified structure: if `System.Reactive` inevitably gives you WPF and Windows Forms support whenever you target a `netX.0-windowX`-like framework, you're going to have this problem. There has to be some way to indicate whether or not you want that. The [workaround](#the-workaround) provides a way to undo this problem, but workarounds usually have their own problems, and we don't want people to have to discover and then apply a fix just to be able to use Rx.NET. I think separating out these parts is the only way to achieve this. This design option also doesn't have a good answer for how we provide UI-framework-specific support for other frameworks. (E.g., how would we offer a `DispatcherScheduler` for MAUI's `IScheduler`?) -So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from primary problem this ADR discusses. Nor have I seen a proposal that would achieve this. +So in short, I ([@idg10](https://github.com/idg10)) haven't been able to think of a design that maintains the unified approach that doesn't also suffer from primary problem this ADR discusses. Nor have I seen a viable proposal that would achieve this. The only attraction of this design option is that it is least likely to cause any unanticipated new problems, because it _is_ the existing design. @@ -912,7 +912,7 @@ public IObservable FormatNumbers(IObservable xs) => xs.Select(x => Which implementation of `Select` is the compiler going to pick? The problem is that you've now got two definitions of `System.Reactive.Linq.Observable` from two different assemblies. Any file that has a `using System.Reactive.Linq;` directive is going to have both the v6 and the v7 definition of `Observable` in scope. Which one is the compiler supposed to pick? -That call to `Select` is ambiguous. That's the problem you create if you go down this so-called "clean break" path. +That call to `Select` is ambiguous. The compiler would emit an error because it doesn't know which to use. That's the problem you create if you go down this so-called "clean break" path. It is technically possible to disambiguate identically-named classes with the rarely-used [`extern alias` feature](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/extern-alias). You can associate an alias with a NuGet reference like this: @@ -963,7 +963,7 @@ strings.Subscribe(Console.WriteLine); This uses the Rx v6 implementation of `Observable.Range`, but the two places where this uses extension methods (the call to `Select` and the call to the delegate-based `Subscribe` overload) will use the Rx v6 implementation, because that's what we specified in the `using` directives when importing the relevant namespaces. -So it's possible to make it work, but we consider this to be confusing and painful. You should only really need to use the `extern alias` mechanism if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. Admittedly, it wouldn't be for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. +So it's possible to make it work, but we consider this to be confusing and painful. You should only really need to use the `extern alias` mechanism if something somewhere has gone horribly wrong. We don't want to make it the norm in Rx usage. Admittedly, it wouldn't be needed for simple applications that use Rx directly. But it would become necessary any time you have indirect references to versions of Rx from both sides of this alleged 'clean break'. The only way to avoid this would be to change not just the NuGet package names but also all the namespaces. (Even this still has the potential to cause confusion if any file ends up needing to us namespaces from both worlds.) That somewhat resembles what the Azure SDK team did a few years back: they introduced a new set of libraries under a completely different set of namespaces. There was no attempt at or pretence of continuity. New NuGet packages were released, and the old ones they replaced have gradually been deprecated. @@ -1001,7 +1001,8 @@ One unresolved issue with this option (and which is not unique to this option) i David Karnok [suggested we turn `DispatcherScheduler` into a delegator and move the real implementation elsewhere](https://github.com/dotnet/reactive/discussions/2038#discussioncomment-7557205). -Unless I've missed something, I don't think this helps us at all because all of these types have UI-specific types as part of their public API. So even if you remove all implementation details, you still end up with dependencies on the relevant frameworks. +Unless I've missed something, I don't think this helps us at all because all of these types have UI-specific types as part of their public API. So even if you remove all implementation details, you still end up with dependencies on the relevant frameworks. (Remember, it's those unwanted dependencies on WPF and Windows Forms that are the problem here.) + #### Option 6: UI-framework specific packages, deprecating the `System.Reactive` versions @@ -1040,8 +1041,7 @@ We will go with [option 6](#option-6-ui-framework-specific-packages-deprecating- * we will add a new NuGet packages for each UI framework, and put UI-framework-specific support in these * there will be an additional package for functionality specific to Windows Runtime that is not UI-framework-specific * the UI-frameworks-specific and Windows-Runtime-specific types in `System.Reactive` will be marked as `[Obsolete]` with the [very long-term](#minimum-acceptable-path-for-breaking-changes) intention of removing them entirely, likely not before 2030 -* the UWP-specific members of its `ThreadPoolScheduler` will be marked as `[Obsolete]` - +* the UWP-specific members of its `ThreadPoolScheduler` will be marked as `[Obsolete]` and will also be removed in the long term, but possibly sooner than 2030 As it says in [the announcement for the first Rx release](https://cc.bingj.com/cache.aspx?q=microsoft+download+reactive+extension+sdk&d=5018270206749605&mkt=en-GB&setlang=en-GB&w=LCqKaZy3VgjqC_Zlig1e4qmTo82s8qt5): @@ -1069,9 +1069,9 @@ Although there are some existing old packages that used to contain some UI-frame Our current plan is to leave these facade components untouched, since the split of types is a bit confusing, the naming is a bit inconsistent, and their purpose for years has been to provide backwards compatibility for Rx v3 era apps. We will introduce new components each with a clear purpose: -* `System.Reactive.Integration.WindowsForms` -* `System.Reactive.Integration.Wpf` -* `System.Reactive.Integration.Uwp` -* `System.Reactive.Integration.WindowsRuntime` +* `System.Reactive.For.WindowsForms` +* `System.Reactive.For.Wpf` +* `System.Reactive.For.Uwp` +* `System.Reactive.For.WindowsRuntime` -We expect people to hate these names on first acquaintance. (Particularly the `Integration` bit.) If we're honest, we don't love them, but all other suggestions to date have either had problems (e.g. there are already packages with the suggested names) or aren't noticeably better. \ No newline at end of file +We expect people to hate these names on first acquaintance. (Our initial proposal was to have `Integration` instead of `For`, and people definitely hated that. Anais Betts suggested this `For` name instead. We liked it more, as did most other people, but all options are at least slightly controversial.) All other suggestions to date have either had problems (e.g. there are already packages with the suggested names) or aren't noticeably better. \ No newline at end of file