Skip to content

Add Nuke steps to generate simplified "supported" integrations CSVs #7355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Add Nuke steps to generate simplified "supported" integrations CSVs #7355

wants to merge 3 commits into from

Conversation

bouwkast
Copy link
Collaborator

@bouwkast bouwkast commented Aug 6, 2025

Summary of changes

Adds a Nuke step tracer/build.ps1 GenerateSupportedIntegrationsCsv to generate three CSVs around our supported integrations

It generates three CSVs: one for .NET Framework, one for .NET Core, one as a combination.

I have not validated the accuracy of the results but seems close enough to start looking at internally and build on IMO

Reason for change

IDM team is now interested in collating data across tracing libraries of what various libraries are supported in various tracers.

Implementation details

  • Build.SupportedIntegrationsTable Nuke target to generate the CSVs
  • SupportedIntegrationsTableGenerator class that does the actual conversion of supported_versions.json into the CSVs.

Note that this is run locally.

Test coverage

Ran it locally seems fine.

Other details

This is NOT prescriptive in that these generated CSVs aren't the source of truth.
This was something I messed around with for a day during the last R&D week to see how to use various LLM coding assistants mainly looking into parsing our supported_versions.json file into something a bit more consumable.

A better approach that I also experimented with was based off our generated calltargets and a Roslyn analyzer to go through our various attributes to pull out actual source of truth data (and can include other bits like what methods we actually instrument) which seems more useful, but probably too much information at the moment.

All of this information is basically pulled from supported_versions.json but easier this way to disseminate the information internally without caring about "Assembly" vs "NuGet" (something that must be considered though long term).

bouwkast added 2 commits August 6, 2025 13:31
@bouwkast
Add Nuke steps to generate supported integrations
120de82 
This was something I messed around with last R&D
week with Cursor / other LLM coding agents to
see how they would fair taking in some JSON
structure and wrangling it into a "different"
format.

There isn't much benefit of this TBH compared
to the current supported_versions.json file
but since that one has additional information
:shrug:
@bouwkast
Simplify the data
cc36343
@github-actions github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Aug 6, 2025
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Aug 6, 2025 •
edited
Loading

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f71601d | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Aug 11, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7355 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.390
  • 1 benchmarks are slower, with geometric mean 2.428
  • 3 benchmarks have fewer allocations
  • 6 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 5.98 KB 6.12 KB 137 B 2.29%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.5μs 57.1ns 328ns 0 0 0 5.49 KB
master StartStopWithChild netcoreapp3.1 13.6μs 53.8ns 201ns 0 0 0 5.69 KB
master StartStopWithChild net472 21.9μs 118ns 739ns 0.978 0.435 0.109 5.98 KB
#7355 StartStopWithChild net6.0 11.4μs 54.4ns 211ns 0 0 0 5.49 KB
#7355 StartStopWithChild netcoreapp3.1 13.5μs 67ns 300ns 0 0 0 5.7 KB
#7355 StartStopWithChild net472 22.1μs 120ns 733ns 0.962 0.214 0 6.12 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.35 KB 3.31 KB -46 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 947μs 120ns 463ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 183ns 707ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.19ms 110ns 427ns 0 0 0 3.35 KB
#7355 WriteAndFlushEnrichedTraces net6.0 931μs 84.2ns 315ns 0 0 0 2.71 KB
#7355 WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 143ns 516ns 0 0 0 2.7 KB
#7355 WriteAndFlushEnrichedTraces net472 1.2ms 91.2ns 353ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 336μs 1.5μs 5.81μs 0 0 0 198.49 KB
master AllCycleSimpleBody netcoreapp3.1 505μs 2.47μs 11.1μs 0 0 0 206.2 KB
master AllCycleSimpleBody net472 451μs 90.7ns 351ns 37.9 2.23 0 246.48 KB
master AllCycleMoreComplexBody net6.0 338μs 1.25μs 4.86μs 0 0 0 202 KB
master AllCycleMoreComplexBody netcoreapp3.1 499μs 218ns 843ns 0 0 0 209.62 KB
master AllCycleMoreComplexBody net472 458μs 99.4ns 372ns 37.9 2.23 0 249.99 KB
master ObjectExtractorSimpleBody net6.0 323ns 0.902ns 3.49ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 397ns 2.23ns 13.9ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.0259ns 0.0934ns 0.0439 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.29μs 2.03ns 7.31ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.9μs 37.4ns 145ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.67μs 0.648ns 2.42ns 0.601 0 0 3.8 KB
#7355 AllCycleSimpleBody net6.0 334μs 1.51μs 5.64μs 0 0 0 198.49 KB
#7355 AllCycleSimpleBody netcoreapp3.1 524μs 1.72μs 6.65μs 0 0 0 206.2 KB
#7355 AllCycleSimpleBody net472 449μs 108ns 405ns 37.9 2.23 0 246.48 KB
#7355 AllCycleMoreComplexBody net6.0 342μs 1.45μs 5.61μs 0 0 0 202 KB
#7355 AllCycleMoreComplexBody netcoreapp3.1 503μs 1.75μs 6.79μs 0 0 0 209.62 KB
#7355 AllCycleMoreComplexBody net472 460μs 97ns 350ns 37.9 2.23 0 249.99 KB
#7355 ObjectExtractorSimpleBody net6.0 312ns 1.68ns 8.87ns 0 0 0 280 B
#7355 ObjectExtractorSimpleBody netcoreapp3.1 419ns 2.09ns 8.88ns 0 0 0 272 B
#7355 ObjectExtractorSimpleBody net472 311ns 0.0268ns 0.0967ns 0.0445 0 0 281 B
#7355 ObjectExtractorMoreComplexBody net6.0 6.35μs 17.2ns 66.6ns 0 0 0 3.78 KB
#7355 ObjectExtractorMoreComplexBody netcoreapp3.1 7.75μs 28.4ns 106ns 0 0 0 3.69 KB
#7355 ObjectExtractorMoreComplexBody net472 6.73μs 2.7ns 10.5ns 0.576 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 78.3μs 194ns 753ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 96.6μs 261ns 1.01μs 0 0 0 32.4 KB
master EncodeArgs net472 112μs 33.4ns 129ns 5.03 0 0 32.51 KB
master EncodeLegacyArgs net6.0 140μs 8.69ns 30.1ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 196μs 146ns 567ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 263μs 32.4ns 121ns 0 0 0 2.16 KB
#7355 EncodeArgs net6.0 77.4μs 28.2ns 109ns 0 0 0 32.4 KB
#7355 EncodeArgs netcoreapp3.1 97.2μs 44.2ns 171ns 0 0 0 32.4 KB
#7355 EncodeArgs net472 109μs 7.69ns 28.8ns 4.9 0 0 32.51 KB
#7355 EncodeLegacyArgs net6.0 144μs 28.9ns 104ns 0 0 0 2.14 KB
#7355 EncodeLegacyArgs netcoreapp3.1 201μs 194ns 751ns 0 0 0 2.14 KB
#7355 EncodeLegacyArgs net472 261μs 31.3ns 121ns 0 0 0 2.17 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #7355

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 2.428 302,762.27 734,958.69

Faster 🎉 in #7355

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1 1.390 868,004.69 624,532.50

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 394μs 133ns 515ns 0 0 0 4.56 KB
master RunWafRealisticBenchmark netcoreapp3.1 825μs 12.9μs 129μs 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 431μs 86.9ns 337ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 286μs 83.1ns 311ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 303μs 48.5ns 181ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 314μs 37.8ns 146ns 0 0 0 2.29 KB
#7355 RunWafRealisticBenchmark net6.0 397μs 84.7ns 317ns 0 0 0 4.56 KB
#7355 RunWafRealisticBenchmark netcoreapp3.1 616μs 4.54μs 42.8μs 0 0 0 4.48 KB
#7355 RunWafRealisticBenchmark net472 433μs 75.6ns 293ns 0 0 0 4.68 KB
#7355 RunWafRealisticBenchmarkWithAttack net6.0 290μs 43.6ns 163ns 0 0 0 2.24 KB
#7355 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 678μs 11.6μs 104μs 0 0 0 2.22 KB
#7355 RunWafRealisticBenchmarkWithAttack net472 314μs 21.4ns 80.2ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.2μs 59.9ns 224ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 72μs 96.2ns 360ns 0 0 0 17.42 KB
master SendRequest net472 0.0189ns 0.00191ns 0.00714ns 0 0 0 0 b
#7355 SendRequest net6.0 61.4μs 108ns 417ns 0 0 0 14.52 KB
#7355 SendRequest netcoreapp3.1 71μs 52.9ns 205ns 0 0 0 17.42 KB
#7355 SendRequest net472 0.00187ns 0.000843ns 0.00326ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 1 B 3 B 2 B 200.00%

Fewer allocations 🎉 in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 7 B 2 B -5 B -71.43%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.99ms 1μs 3.87μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.12ms 7.85μs 28.3μs 0 0 0 640 KB
master OriginalCharSlice net472 2.6ms 142ns 549ns 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.37ms 455ns 1.76μs 0 0 0 7 B
master OptimizedCharSlice netcoreapp3.1 1.74ms 281ns 975ns 0 0 0 1 B
master OptimizedCharSlice net472 1.96ms 223ns 833ns 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 860μs 30.7ns 119ns 0 0 0 1 B
master OptimizedCharSliceWithPool netcoreapp3.1 819μs 55.9ns 216ns 0 0 0 1 B
master OptimizedCharSliceWithPool net472 1.18ms 63.5ns 246ns 0 0 0 0 b
#7355 OriginalCharSlice net6.0 1.92ms 2.23μs 8.65μs 0 0 0 640.01 KB
#7355 OriginalCharSlice netcoreapp3.1 2.15ms 11.5μs 58.4μs 0 0 0 640 KB
#7355 OriginalCharSlice net472 2.69ms 155ns 602ns 100 0 0 641.95 KB
#7355 OptimizedCharSlice net6.0 1.41ms 314ns 1.22μs 0 0 0 2 B
#7355 OptimizedCharSlice netcoreapp3.1 1.72ms 196ns 758ns 0 0 0 1 B
#7355 OptimizedCharSlice net472 1.99ms 294ns 1.14μs 0 0 0 0 b
#7355 OptimizedCharSliceWithPool net6.0 881μs 39.2ns 152ns 0 0 0 3 B
#7355 OptimizedCharSliceWithPool netcoreapp3.1 812μs 284ns 1.1μs 0 0 0 1 B
#7355 OptimizedCharSliceWithPool net472 1.13ms 32.7ns 127ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 644μs 2.7μs 10.4μs 0 0 0 41.65 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 648μs 2.43μs 8.76μs 0 0 0 41.87 KB
master WriteAndFlushEnrichedTraces net472 871μs 1.6μs 5.99μs 8.33 0 0 55.95 KB
#7355 WriteAndFlushEnrichedTraces net6.0 655μs 411ns 1.59μs 0 0 0 41.72 KB
#7355 WriteAndFlushEnrichedTraces netcoreapp3.1 667μs 1.32μs 4.57μs 0 0 0 41.93 KB
#7355 WriteAndFlushEnrichedTraces net472 844μs 2.5μs 9.01μs 7.81 0 0 56.1 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.95μs 1.43ns 5.14ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.56μs 7.58ns 29.4ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.67μs 3.45ns 13.3ns 0.147 0.0134 0 987 B
#7355 ExecuteNonQuery net6.0 1.9μs 4.51ns 17.5ns 0 0 0 1.02 KB
#7355 ExecuteNonQuery netcoreapp3.1 2.61μs 8.66ns 33.5ns 0 0 0 1.02 KB
#7355 ExecuteNonQuery net472 2.73μs 2.21ns 8.57ns 0.15 0.0137 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.72μs 0.843ns 3.27ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.26μs 7.44ns 28.8ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.52μs 4ns 15.5ns 0.159 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.77μs 7.19ns 26.9ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.52μs 5.66ns 21.9ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.66μs 4.72ns 17ns 0.165 0 0 1.1 KB
#7355 CallElasticsearch net6.0 1.76μs 2.91ns 11.3ns 0 0 0 1.03 KB
#7355 CallElasticsearch netcoreapp3.1 2.36μs 6.57ns 25.4ns 0 0 0 1.03 KB
#7355 CallElasticsearch net472 3.6μs 3.81ns 14.3ns 0.162 0 0 1.04 KB
#7355 CallElasticsearchAsync net6.0 1.81μs 8.14ns 31.5ns 0 0 0 1.01 KB
#7355 CallElasticsearchAsync netcoreapp3.1 2.51μs 2.47ns 9.55ns 0 0 0 1.08 KB
#7355 CallElasticsearchAsync net472 3.86μs 11.4ns 44.1ns 0.174 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.87μs 4.43ns 16.6ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.29μs 5.47ns 21.2ns 0 0 0 952 B
master ExecuteAsync net472 2.51μs 1.19ns 4.46ns 0.138 0 0 915 B
#7355 ExecuteAsync net6.0 1.84μs 1.4ns 5.22ns 0 0 0 952 B
#7355 ExecuteAsync netcoreapp3.1 2.33μs 10.6ns 38.4ns 0 0 0 952 B
#7355 ExecuteAsync net472 2.56μs 1.46ns 5.66ns 0.141 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.74μs 9.03ns 35ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.42μs 27.5ns 107ns 0 0 0 2.9 KB
master SendAsync net472 11.9μs 13.1ns 50.9ns 0.48 0 0 3.18 KB
#7355 SendAsync net6.0 6.87μs 16.9ns 63.1ns 0 0 0 2.36 KB
#7355 SendAsync netcoreapp3.1 8.54μs 34.6ns 134ns 0 0 0 2.9 KB
#7355 SendAsync net472 12.1μs 5.37ns 20.1ns 0.484 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.79 KB 86.15 KB 43.36 KB 101.33%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 249.99 KB 262.94 KB 12.95 KB 5.18%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 45.62 KB 46.42 KB 808 B 1.77%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 282.1 KB 286.72 KB 4.62 KB 1.64%

Fewer allocations 🎉 in #7355

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 277.9 KB 275.27 KB -2.62 KB -0.94%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.4μs 306ns 2.85μs 0 0 0 45.62 KB
master StringConcatBenchmark netcoreapp3.1 50.8μs 594ns 5.69μs 0 0 0 42.79 KB
master StringConcatBenchmark net472 57.6μs 152ns 550ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 457μs 2.31μs 10.3μs 0 0 0 249.99 KB
master StringConcatAspectBenchmark netcoreapp3.1 450μs 5.32μs 52.4μs 0 0 0 277.9 KB
master StringConcatAspectBenchmark net472 408μs 2μs 12μs 0 0 0 282.1 KB
#7355 StringConcatBenchmark net6.0 42.1μs 239ns 1.62μs 0 0 0 46.42 KB
#7355 StringConcatBenchmark netcoreapp3.1 49.8μs 295ns 2.69μs 0 0 0 86.15 KB
#7355 StringConcatBenchmark net472 57.7μs 127ns 476ns 0 0 0 57.34 KB
#7355 StringConcatAspectBenchmark net6.0 475μs 2.19μs 8.48μs 0 0 0 262.94 KB
#7355 StringConcatAspectBenchmark netcoreapp3.1 497μs 2.23μs 8.03μs 0 0 0 275.27 KB
#7355 StringConcatAspectBenchmark net472 414μs 2.14μs 10.3μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.56μs 4.56ns 17.6ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.57μs 16.3ns 63ns 0 0 0 1.7 KB
master EnrichedLog net472 3.84μs 5.58ns 21.6ns 0.247 0 0 1.64 KB
#7355 EnrichedLog net6.0 2.63μs 8.05ns 31.2ns 0 0 0 1.7 KB
#7355 EnrichedLog netcoreapp3.1 3.43μs 16.8ns 71.5ns 0 0 0 1.7 KB
#7355 EnrichedLog net472 3.7μs 0.946ns 3.41ns 0.242 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 87.3ns 302ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 127μs 28.1ns 105ns 0 0 0 4.31 KB
master EnrichedLog net472 169μs 175ns 654ns 0 0 0 4.52 KB
#7355 EnrichedLog net6.0 123μs 65.7ns 246ns 0 0 0 4.31 KB
#7355 EnrichedLog netcoreapp3.1 130μs 616ns 3.2μs 0 0 0 4.31 KB
#7355 EnrichedLog net472 171μs 324ns 1.26μs 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.93μs 8.26ns 30.9ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.8μs 27.7ns 107ns 0 0 0 2.26 KB
master EnrichedLog net472 7.62μs 11.3ns 43.6ns 0.304 0 0 2.08 KB
#7355 EnrichedLog net6.0 4.83μs 5.13ns 19.9ns 0 0 0 2.26 KB
#7355 EnrichedLog netcoreapp3.1 6.61μs 9.41ns 35.2ns 0 0 0 2.26 KB
#7355 EnrichedLog net472 7.57μs 5.49ns 21.3ns 0.303 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.09μs 5.53ns 20.7ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.6μs 9.92ns 38.4ns 0 0 0 1.2 KB
master SendReceive net472 3.14μs 1.42ns 5.31ns 0.189 0 0 1.2 KB
#7355 SendReceive net6.0 2.02μs 10.2ns 43.2ns 0 0 0 1.2 KB
#7355 SendReceive netcoreapp3.1 2.78μs 12.9ns 51.6ns 0 0 0 1.2 KB
#7355 SendReceive net472 3.16μs 4.01ns 15ns 0.189 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.13μs 11.9ns 46ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.62μs 7.11ns 26.6ns 0 0 0 1.63 KB
master EnrichedLog net472 6.59μs 6.83ns 26.5ns 0.298 0 0 2.03 KB
#7355 EnrichedLog net6.0 4.09μs 11.1ns 41.4ns 0 0 0 1.58 KB
#7355 EnrichedLog netcoreapp3.1 5.55μs 13.3ns 49.9ns 0 0 0 1.63 KB
#7355 EnrichedLog net472 6.55μs 8.27ns 32ns 0.295 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 749ns 3.61ns 14.9ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 919ns 5.01ns 27ns 0 0 0 576 B
master StartFinishSpan net472 903ns 0.643ns 2.41ns 0.0909 0 0 578 B
master StartFinishScope net6.0 908ns 4.39ns 19.1ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.11μs 4.91ns 23.5ns 0 0 0 696 B
master StartFinishScope net472 1.11μs 1.15ns 4.45ns 0.1 0 0 658 B
#7355 StartFinishSpan net6.0 768ns 1.46ns 5.65ns 0 0 0 576 B
#7355 StartFinishSpan netcoreapp3.1 977ns 1.16ns 4.49ns 0 0 0 576 B
#7355 StartFinishSpan net472 891ns 0.333ns 1.29ns 0.0892 0 0 578 B
#7355 StartFinishScope net6.0 903ns 1.88ns 7.28ns 0 0 0 696 B
#7355 StartFinishScope netcoreapp3.1 1.18μs 1.29ns 4.98ns 0 0 0 696 B
#7355 StartFinishScope net472 1.07μs 0.263ns 0.911ns 0.102 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.47ns 30.5ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.35μs 6.51ns 26.1ns 0 0 0 696 B
master RunOnMethodBegin net472 1.41μs 1.12ns 4.2ns 0.101 0 0 658 B
#7355 RunOnMethodBegin net6.0 1.03μs 4.96ns 19.8ns 0 0 0 696 B
#7355 RunOnMethodBegin netcoreapp3.1 1.37μs 3.85ns 14.9ns 0 0 0 696 B
#7355 RunOnMethodBegin net472 1.42μs 0.839ns 3.25ns 0.0996 0 0 658 B

@dd-trace-dotnet-ci-bot
Copy link

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (72ms)  : 71, 74
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7355) - mean (68ms)  : 66, 71
     .   : milestone, 68,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (1,033ms)  : 1014, 1053
     .   : milestone, 1033,
    master - mean (1,033ms)  : 1009, 1058
     .   : milestone, 1033,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (108ms)  : 107, 110
     .   : milestone, 108,
    master - mean (107ms)  : 105, 108
     .   : milestone, 107,

    section Baseline
    This PR (7355) - mean (107ms)  : 105, 110
     .   : milestone, 107,
    master - mean (106ms)  : 104, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (723ms)  : 710, 735
     .   : milestone, 723,
    master - mean (725ms)  : 715, 735
     .   : milestone, 725,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (95ms)  : 94, 97
     .   : milestone, 95,
    master - mean (94ms)  : 93, 96
     .   : milestone, 94,

    section Baseline
    This PR (7355) - mean (95ms)  : 93, 97
     .   : milestone, 95,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (675ms)  : 664, 687
     .   : milestone, 675,
    master - mean (681ms)  : 673, 690
     .   : milestone, 681,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (94ms)  : 93, 96
     .   : milestone, 94,
    master - mean (94ms)  : 92, 95
     .   : milestone, 94,

    section Baseline
    This PR (7355) - mean (93ms)  : 91, 95
     .   : milestone, 93,
    master - mean (93ms)  : 91, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (616ms)  : 604, 628
     .   : milestone, 616,
    master - mean (625ms)  : 615, 634
     .   : milestone, 625,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (210ms)  : 202, 219
     .   : milestone, 210,
    master - mean (208ms)  : 198, 219
     .   : milestone, 208,

    section Baseline
    This PR (7355) - mean (208ms)  : 192, 224
     .   : milestone, 208,
    master - mean (202ms)  : 191, 213
     .   : milestone, 202,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (1,206ms)  : 1161, 1250
     .   : milestone, 1206,
    master - mean (1,216ms)  : 1169, 1263
     .   : milestone, 1216,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (293ms)  : 282, 304
     .   : milestone, 293,
    master - mean (287ms)  : 276, 298
     .   : milestone, 287,

    section Baseline
    This PR (7355) - mean (293ms)  : 279, 307
     .   : milestone, 293,
    master - mean (292ms)  : 278, 306
     .   : milestone, 292,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (968ms)  : 926, 1009
     .   : milestone, 968,
    master - mean (968ms)  : 934, 1002
     .   : milestone, 968,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (293ms)  : 282, 304
     .   : milestone, 293,
    master - mean (285ms)  : 271, 299
     .   : milestone, 285,

    section Baseline
    This PR (7355) - mean (297ms)  : 280, 314
     .   : milestone, 297,
    master - mean (284ms)  : 269, 299
     .   : milestone, 284,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (962ms)  : 927, 997
     .   : milestone, 962,
    master - mean (949ms)  : 899, 1000
     .   : milestone, 949,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7355) - mean (291ms)  : 277, 305
     .   : milestone, 291,
    master - mean (291ms)  : 274, 308
     .   : milestone, 291,

    section Baseline
    This PR (7355) - mean (289ms)  : 277, 301
     .   : milestone, 289,
    master - mean (289ms)  : 269, 308
     .   : milestone, 289,

    section CallTarget+Inlining+NGEN
    This PR (7355) - mean (862ms)  : 819, 904
     .   : milestone, 862,
    master - mean (873ms)  : 830, 917
     .   : milestone, 873,
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@bouwkast