Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/run-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
run: ./build.cmd in-tests-core -e
# Upload Artifacts with Unique Name
- name: Upload test results
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
if: always()
with:
name: test-windows-core-trx-${{ github.run_id }}
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ The measured data can be exported to different formats (md, html, csv, xml, json

![](https://raw.githubusercontent.com/dotnet/BenchmarkDotNet/ec962b0bd6854c991d7a3ebd77037579165acb36/docs/images/v0.12.0/rplot.png)

*Supported runtimes:* .NET 5+, .NET Framework 4.6.1+, .NET Core 3.1+, Mono, NativeAOT
*Supported runtimes:* .NET 5+, .NET Framework 4.6.1+, .NET Core 2.0+, Mono, NativeAOT
*Supported languages:* C#, F#, Visual Basic
*Supported OS:* Windows, Linux, macOS
*Supported architectures:* x86, x64, ARM, ARM64, Wasm and LoongArch64
Expand All @@ -135,8 +135,8 @@ If you want to compare benchmarks with each other,
mark one of the benchmarks as the [baseline](https://benchmarkdotnet.org/articles/features/baselines.html)
via `[Benchmark(Baseline = true)]`: BenchmarkDotNet will compare it with all of the other benchmarks.
If you want to compare performance in different environments, use [jobs](https://benchmarkdotnet.org/articles/configs/jobs.html).
For example, you can run all the benchmarks on .NET Core 3.1 and Mono via
`[SimpleJob(RuntimeMoniker.NetCoreApp31)]` and `[SimpleJob(RuntimeMoniker.Mono)]`.
For example, you can run all the benchmarks on .NET 8.0 and Mono via
`[SimpleJob(RuntimeMoniker.Net80)]` and `[SimpleJob(RuntimeMoniker.Mono)]`.

If you don't like attributes, you can call most of the APIs via the fluent style and write code like this:

Expand Down
2 changes: 1 addition & 1 deletion build/common.props
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>

<Nullable>annotations</Nullable>
<!-- Lets supress the "System.Collections.Immutable 8.0.0 doesn't support netcoreapp3.1" warning -->
<!-- Suppress warning for nuget package used in old (unsupported) tfm. -->
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

Expand Down
8 changes: 4 additions & 4 deletions docs/articles/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,17 @@ See also: [BenchmarkDotNet#237](https://github.com/dotnet/BenchmarkDotNet/issues

* **Q** Why can't I install BenchmarkDotNet in a new .NET Core Console App in Visual Studio 2017?

**A** BenchmarkDotNet supports only netcoreapp3.1+.
**A** BenchmarkDotNet supports only netcoreapp2.0+.
Some old Visual Studio 2017 can create a new application which targets netcoreapp1.0.
You should upgrade it up to 3.1.
You should upgrade it up to 2.0.
If you want to target netcoreapp1.0 in your main assembly, it's recommended to create a separated project for benchmarks.

* **Q** I created a new .NET Core Console App in Visual Studio 2017. Now I want to run my code on CoreCLR, full .NET Framework, and Mono. How can I do it?

**A** Use the following lines in your `.csproj` file:

```xml
<TargetFrameworks>netcoreapp3.1;net46</TargetFrameworks>
<TargetFrameworks>netcoreapp2.0;net46</TargetFrameworks>
<PlatformTarget>AnyCPU</PlatformTarget>
```

Expand All @@ -33,7 +33,7 @@ If you want to target netcoreapp1.0 in your main assembly, it's recommended to c
[CoreJob, ClrJob, MonoJob]
```

* **Q** My source code targets old versions of .NET Framework or .NET Core, but BenchmarkDotNet requires `net461` and `netcoreapp3.1`. How can I run benchmarks in this case?
* **Q** My source code targets old versions of .NET Framework or .NET Core, but BenchmarkDotNet requires `net461` and `netcoreapp2.0`. How can I run benchmarks in this case?

**A** It's a good practice to introduce an additional console application (e.g. `MyAwesomeLibrary.Benchmarks`) which will depend on your code and BenchmarkDotNet.
Due to the fact that users usually run benchmarks in a develop environment and don't distribute benchmarks for users, it shouldn't be a problem.
Expand Down
6 changes: 3 additions & 3 deletions docs/articles/guides/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ How do you know that BenchmarkDotNet has failed to build the project? BDN is goi
// ***** BenchmarkRunner: Start *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start *****
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// command took 0.51s and exited with 1
// ***** Done, took 00:00:00 (0.66 sec) *****
// Found 1 benchmarks:
Expand All @@ -20,10 +20,10 @@ How do you know that BenchmarkDotNet has failed to build the project? BDN is goi
// Build Error: Standard output:

Standard error:
C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4\BenchmarkDotNet.Autogenerated.csproj(36,1): error MSB4025: The project file could not be loaded. Unexpected end of file while parsing Comment has occurred. Line 36, position 1.
C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4\BenchmarkDotNet.Autogenerated.csproj(36,1): error MSB4025: The project file could not be loaded. Unexpected end of file while parsing Comment has occurred. Line 36, position 1.

// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// It can be found in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html
```

Expand Down
2 changes: 1 addition & 1 deletion docs/articles/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ name: Overview
Create new console application and install the [BenchmarkDotNet](https://www.nuget.org/packages/BenchmarkDotNet/) NuGet package. We support:

* *Projects:* classic and modern with PackageReferences
* *Runtimes:* Full .NET Framework (4.6+), .NET Core (3.1+), Mono, NativeAOT
* *Runtimes:* Full .NET Framework (4.6+), .NET Core (2.0+), Mono, NativeAOT
* *OS:* Windows, Linux, MacOS
* *Languages:* C#, F#, VB

Expand Down
13 changes: 4 additions & 9 deletions src/BenchmarkDotNet.Annotations/Jobs/RuntimeMoniker.cs
Original file line number Diff line number Diff line change
Expand Up @@ -57,27 +57,22 @@ public enum RuntimeMoniker
/// <summary>
/// .NET Core 2.0
/// </summary>
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
// Assigning explicit values so we can check for them without the compiler erroring.
NetCoreApp20 = 10,
NetCoreApp20,

/// <summary>
/// .NET Core 2.1
/// </summary>
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
NetCoreApp21 = 11,
NetCoreApp21,

/// <summary>
/// .NET Core 2.2
/// </summary>
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
NetCoreApp22 = 12,
NetCoreApp22,

/// <summary>
/// .NET Core 3.0
/// </summary>
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
NetCoreApp30 = 13,
NetCoreApp30,

/// <summary>
/// .NET Core 3.1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ internal override bool IsSupported(RuntimeMoniker runtimeMoniker)
case RuntimeMoniker.NetCoreApp50:
#pragma warning restore CS0618 // Type or member is obsolete
return false;
case RuntimeMoniker.NetCoreApp20:
case RuntimeMoniker.NetCoreApp21:
case RuntimeMoniker.NetCoreApp22:
return OsDetector.IsWindows();
case RuntimeMoniker.NetCoreApp30:
case RuntimeMoniker.NetCoreApp31:
return OsDetector.IsWindows() || OsDetector.IsLinux();
default:
Expand Down
5 changes: 5 additions & 0 deletions src/BenchmarkDotNet.Diagnostics.dotTrace/DotTraceDiagnoser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,11 @@ internal override bool IsSupported(RuntimeMoniker runtimeMoniker)
case RuntimeMoniker.NetCoreApp50:
#pragma warning restore CS0618 // Type or member is obsolete
return false;
case RuntimeMoniker.NetCoreApp20:
case RuntimeMoniker.NetCoreApp21:
case RuntimeMoniker.NetCoreApp22:
return OsDetector.IsWindows();
case RuntimeMoniker.NetCoreApp30:
case RuntimeMoniker.NetCoreApp31:
return OsDetector.IsWindows() || OsDetector.IsLinux();
default:
Expand Down
1 change: 0 additions & 1 deletion src/BenchmarkDotNet/BenchmarkDotNet.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
<PackageReference Include="Microsoft.Diagnostics.Tracing.TraceEvent" Version="3.1.8" PrivateAssets="contentfiles;analyzers" />
<PackageReference Include="Microsoft.DotNet.PlatformAbstractions" Version="3.1.6" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.12.0" />
<!-- Do not update these packages, or else netcoreapp3.1 may no longer work -->
<PackageReference Include="System.Management" Version="6.0.0" />
</ItemGroup>
<ItemGroup Condition=" '$(TargetFramework)' == 'netstandard2.0' ">
Expand Down
4 changes: 4 additions & 0 deletions src/BenchmarkDotNet/ConsoleArguments/ConfigParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,10 @@ private static Job CreateJobForGivenRuntime(Job baseJob, string runtimeId, Comma
.WithRuntime(runtimeMoniker.GetRuntime())
.WithToolchain(CsProjClassicNetToolchain.From(runtimeId, options.RestorePath?.FullName, options.CliPath?.FullName));

case RuntimeMoniker.NetCoreApp20:
case RuntimeMoniker.NetCoreApp21:
case RuntimeMoniker.NetCoreApp22:
case RuntimeMoniker.NetCoreApp30:
case RuntimeMoniker.NetCoreApp31:
#pragma warning disable CS0618 // Type or member is obsolete
case RuntimeMoniker.NetCoreApp50:
Expand Down
4 changes: 2 additions & 2 deletions src/BenchmarkDotNet/Diagnosers/EventPipeProfiler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ public IEnumerable<ValidationError> Validate(ValidationParameters validationPara
{
var runtime = benchmark.Job.ResolveValue(EnvironmentMode.RuntimeCharacteristic, EnvironmentResolver.Instance);

if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp31)
if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp30)
{
yield return new ValidationError(true, $"{nameof(EventPipeProfiler)} supports only .NET Core 3.1+", benchmark);
yield return new ValidationError(true, $"{nameof(EventPipeProfiler)} supports only .NET Core 3.0+", benchmark);
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/BenchmarkDotNet/Diagnosers/ThreadingDiagnoser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,9 @@ public IEnumerable<ValidationError> Validate(ValidationParameters validationPara
{
var runtime = benchmark.Job.ResolveValue(EnvironmentMode.RuntimeCharacteristic, EnvironmentResolver.Instance);

if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp31)
if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp30)
{
yield return new ValidationError(true, $"{nameof(ThreadingDiagnoser)} supports only .NET Core 3.1+", benchmark);
yield return new ValidationError(true, $"{nameof(ThreadingDiagnoser)} supports only .NET Core 3.0+", benchmark);
}
}
}
Expand Down
5 changes: 5 additions & 0 deletions src/BenchmarkDotNet/Disassemblers/DisassemblyDiagnoser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,11 @@ public IEnumerable<ValidationError> Validate(ValidationParameters validationPara
{
var runtime = benchmark.Job.ResolveValue(EnvironmentMode.RuntimeCharacteristic, EnvironmentResolver.Instance);

if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp30)
{
yield return new ValidationError(true, $"{nameof(DisassemblyDiagnoser)} supports only .NET Core 3.0+", benchmark);
}

if (ptrace_scope.Value == "2")
{
yield return new ValidationError(false, $"ptrace_scope is set to 2, {nameof(DisassemblyDiagnoser)} is going to work only if you run as sudo");
Expand Down
2 changes: 1 addition & 1 deletion src/BenchmarkDotNet/Engines/GcStats.cs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ private GcStats(int gen0Collections, int gen1Collections, int gen2Collections, l

public long? GetBytesAllocatedPerOperation(BenchmarkCase benchmarkCase)
{
bool excludeAllocationQuantumSideEffects = benchmarkCase.GetRuntime().RuntimeMoniker <= (RuntimeMoniker) 10; // the issue got fixed for .NET Core 2.0+ https://github.com/dotnet/coreclr/issues/10207
bool excludeAllocationQuantumSideEffects = benchmarkCase.GetRuntime().RuntimeMoniker <= RuntimeMoniker.NetCoreApp20; // the issue got fixed for .NET Core 2.0+ https://github.com/dotnet/coreclr/issues/10207

long? allocatedBytes = GetTotalAllocatedBytes(excludeAllocationQuantumSideEffects);
return allocatedBytes == null ? null
Expand Down
12 changes: 5 additions & 7 deletions src/BenchmarkDotNet/Environments/Runtimes/CoreRuntime.cs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,9 @@ namespace BenchmarkDotNet.Environments
{
public class CoreRuntime : Runtime
{
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
public static readonly CoreRuntime Core20 = new (RuntimeMoniker.NetCoreApp20, "netcoreapp2.0", ".NET Core 2.0");
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
public static readonly CoreRuntime Core21 = new (RuntimeMoniker.NetCoreApp21, "netcoreapp2.1", ".NET Core 2.1");
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
public static readonly CoreRuntime Core22 = new (RuntimeMoniker.NetCoreApp22, "netcoreapp2.2", ".NET Core 2.2");
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
public static readonly CoreRuntime Core30 = new (RuntimeMoniker.NetCoreApp30, "netcoreapp3.0", ".NET Core 3.0");
public static readonly CoreRuntime Core31 = new (RuntimeMoniker.NetCoreApp31, "netcoreapp3.1", ".NET Core 3.1");
public static readonly CoreRuntime Core50 = new (RuntimeMoniker.Net50, "net5.0", ".NET 5.0");
Expand Down Expand Up @@ -69,6 +65,10 @@ internal static CoreRuntime FromVersion(Version version)
{
switch (version)
{
case Version v when v.Major == 2 && v.Minor == 0: return Core20;
case Version v when v.Major == 2 && v.Minor == 1: return Core21;
case Version v when v.Major == 2 && v.Minor == 2: return Core22;
case Version v when v.Major == 3 && v.Minor == 0: return Core30;
case Version v when v.Major == 3 && v.Minor == 1: return Core31;
case Version v when v.Major == 5 && v.Minor == 0: return GetPlatformSpecific(Core50);
case Version v when v.Major == 6 && v.Minor == 0: return GetPlatformSpecific(Core60);
Expand All @@ -77,9 +77,7 @@ internal static CoreRuntime FromVersion(Version version)
case Version v when v.Major == 9 && v.Minor == 0: return GetPlatformSpecific(Core90);
case Version v when v.Major == 10 && v.Minor == 0: return GetPlatformSpecific(Core10_0);
default:
return version >= new Version(3, 1)
? CreateForNewVersion($"net{version.Major}.{version.Minor}", $".NET {version.Major}.{version.Minor}")
: throw new PlatformNotSupportedException($"netcoreapp{version.Major}.{version.Minor} is no longer supported. Use a newer runtime version or use BenchmarkDotNet v0.14.X or older.");
return CreateForNewVersion($"net{version.Major}.{version.Minor}", $".NET {version.Major}.{version.Minor}");
}
}

Expand Down
8 changes: 8 additions & 0 deletions src/BenchmarkDotNet/Extensions/RuntimeMonikerExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ internal static Runtime GetRuntime(this RuntimeMoniker runtimeMoniker)
return ClrRuntime.Net48;
case RuntimeMoniker.Net481:
return ClrRuntime.Net481;
case RuntimeMoniker.NetCoreApp20:
return CoreRuntime.Core20;
case RuntimeMoniker.NetCoreApp21:
return CoreRuntime.Core21;
case RuntimeMoniker.NetCoreApp22:
return CoreRuntime.Core22;
case RuntimeMoniker.NetCoreApp30:
return CoreRuntime.Core30;
case RuntimeMoniker.NetCoreApp31:
return CoreRuntime.Core31;
case RuntimeMoniker.Net50:
Expand Down
2 changes: 2 additions & 0 deletions src/BenchmarkDotNet/Templates/CsProj.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
<AppendTargetFrameworkToOutputPath>true</AppendTargetFrameworkToOutputPath>
<!-- fix for NETSDK1150: https://docs.microsoft.com/en-us/dotnet/core/compatibility/sdk/5.0/referencing-executable-generates-error -->
<ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
<!-- Suppress warning for nuget package used in old (unsupported) tfm. -->
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>
</PropertyGroup>

Expand Down
2 changes: 2 additions & 0 deletions src/BenchmarkDotNet/Templates/MonoAOTLLVMCsProj.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>
<SelfContained>true</SelfContained>
<!-- Suppress warning for nuget package used in old (unsupported) tfm. -->
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand Down
2 changes: 2 additions & 0 deletions src/BenchmarkDotNet/Templates/WasmCsProj.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
<ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles>
<ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
<EnableDefaultWasmAssembliesToBundle>false</EnableDefaultWasmAssembliesToBundle>
<!-- Suppress warning for nuget package used in old (unsupported) tfm. -->
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
<StartupObject>BenchmarkDotNet.Autogenerated.UniqueProgramName</StartupObject>
</PropertyGroup>

Expand Down
4 changes: 0 additions & 4 deletions src/BenchmarkDotNet/Toolchains/CsProj/CsProjCoreToolchain.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,9 @@ namespace BenchmarkDotNet.Toolchains.CsProj
[PublicAPI]
public class CsProjCoreToolchain : Toolchain, IEquatable<CsProjCoreToolchain>
{
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
[PublicAPI] public static readonly IToolchain NetCoreApp20 = From(NetCoreAppSettings.NetCoreApp20);
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
[PublicAPI] public static readonly IToolchain NetCoreApp21 = From(NetCoreAppSettings.NetCoreApp21);
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
[PublicAPI] public static readonly IToolchain NetCoreApp22 = From(NetCoreAppSettings.NetCoreApp22);
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.14.X or older.", true)]
[PublicAPI] public static readonly IToolchain NetCoreApp30 = From(NetCoreAppSettings.NetCoreApp30);
[PublicAPI] public static readonly IToolchain NetCoreApp31 = From(NetCoreAppSettings.NetCoreApp31);
[PublicAPI] public static readonly IToolchain NetCoreApp50 = From(NetCoreAppSettings.NetCoreApp50);
Expand Down
Loading