Skip to content

Update test project to a bit newer TFM #2952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Youssef1313 wants to merge 2 commits into
base: master
Choose a base branch
from
+3 −5
Open

Update test project to a bit newer TFM #2952

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1a57523
Update test project to a bit newer TFM
Youssef1313 Jan 2, 2026
8d7879e
Adjust
Youssef1313 Jan 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions ...ltipleFrameworks/BenchmarkDotNet.IntegrationTests.ManualRunning.MultipleFrameworks.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
<Import Project="..\..\build\common.props" />
<PropertyGroup>
<AssemblyTitle>BenchmarkDotNet.IntegrationTests.ManualRunning.MultipleFrameworks</AssemblyTitle>
<!-- We test the oldest frameworks supported by BDN (net461 and netcoreapp2.0), as well as newer versions of those frameworks. -->
<TargetFrameworks>net461;net48;netcoreapp2.0;net8.0</TargetFrameworks>
<TargetFrameworks>net461;net48;net8.0;net9.0</TargetFrameworks>
<!-- Deprecated runtime and vulnerabilities from old runtime, just in case any are found in the future. -->
<NoWarn>$(NoWarn);NETSDK1138;NU1903</NoWarn>
<!-- Suppress warning MSB3277: Found conflicts between different versions of "System.Buffers" that could not be resolved. -->
Expand Down Expand Up @@ -41,10 +40,9 @@
<ProjectReference Include="..\..\src\BenchmarkDotNet\BenchmarkDotNet.csproj" />
</ItemGroup>
<ItemGroup>
<!-- Microsoft.NET.Test.Sdk breaks netcoreapp2.0 -->
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" Condition=" '$(TargetFramework)' != 'netcoreapp2.0' " />
Copy link
Member Author

@Youssef1313 Youssef1313 Jan 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excluding Microsoft.NET.Test.Sdk on netcoreapp2.0 likely meant that the tests never executed for netcoreapp2.0.

Copy link
Collaborator

@timcassell timcassell Jan 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never had any problems running the tests in VS (we don't run these in CI so it doesn't matter there).

Copy link
Member Author

@Youssef1313 Youssef1313 Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And do the netcoreapp2.0 tests specifically run correctly?
I haven't tried but I doubt. Without this package reference, there is no testhost to run the tests.

Copy link
Collaborator

@timcassell timcassell Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you doubt?

image

Copy link
Member Author

@Youssef1313 Youssef1313 Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The testhost.exe is supposed to come through Microsoft.NET.Test.Sdk. So, without this package, there is no testhost.exe.

However, for this specific case, it looks like Microsoft.NET.Test.Sdk is still brought via xunit.runner.visualstudio 2.4.1. But that's no longer the case for newer xunit.runner.visualstudio packages that don't declare Microsoft.NET.Test.Sdk as a dependency.

In addition, version 15.0.0 of Microsoft.NET.Test.Sdk which is brought by xunit.runner.visualstudio 2.4.1 is also extremely old.

I personally think it's a lot of hassle with low value. A tfm is out of support for a reason. It's not supported, hence it's not worth testing against, especially when it comes with many drawbacks like forcing old dependencies, forcing suppressing warnings (including NuGet security warnings), and preventing moving forward (e.g, migrating from VSTest to Microsoft.Testing.Platform). IMHO, these are big enough drawbacks compared to the value of "manually testing out of support TFMs".

Copy link
Collaborator

@timcassell timcassell Jan 3, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we want to update our testing framework, we can update the TFMs if it's necessary for the update. I don't see the point of updating the TFMs by themselves. The ManualRunning.MultipleFrameworks project is small enough for someone to experiment with newer XUnit or TUnit before doing it for the entire repo.

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<!-- We use older version 2.4.1 for combatibility with netcoreapp2.0. -->
<!-- We use older version 2.4.1 for combatibility with net461 -->
Copy link
Member Author

@Youssef1313 Youssef1313 Jan 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably you can consider updating the tests from net461 to net462.

<PackageReference Include="xunit.runner.visualstudio" Version="[2.4.1]">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
Expand Down