Implement "enahanced telemetry reporting" #7360

andrewlock · 2025-08-07T12:41:36Z

Summary of changes

Implements "enhance telemetry reporting" for .NET

Reason for change

We want to report the telemetry found from all sources, to implement this RFC, so that customers can more easily understand where their telemetry comes from.

Implementation details

The .NET implementation was already pretty close to the desired result, so there were just some main changes to make

Read sources in lowest to highest priority, and always report all found values.
- This differs from previously, where we would read from highest to lowest and stop on first found values.
- In order to ensure that the "last" reported telemetry value is the actual value we use, we have to record the telemetry value twice in cases where we have a parse failure (which should be rare in practice).
- To do this properly, we had to explicitly expose the Origin on IConfigurationSource and captuate the "value for telemetry" where required (for non-simple types).
Report the "default" value to be used, even if it's going to be overridden by an explicit value.
- This is in keeping with the general "report all sources" principle.
- We have to do the same "fixing" of values when we have parsing errors as above.
- To avoid expensive unnecessary computation where a default value is not "simple" we go against this approach when a Func<T> is provided for calculating the default.
For "fallback" keys, we just treat them the same as we do today. The backend will handle merging the keys

Test coverage

Mostly covered by existing tests, and added/updated unit tests to meet new expectations

Other details

Spec: https://docs.google.com/document/d/1vhIimn2vt4tDRSxsHn6vWSc8zYHl0Lv0Fk7CQps04C4/edit?tab=t.0#heading=h.7tw832mj0ztp

Dependent (stacked) on

dd-trace-dotnet-ci-bot · 2025-08-07T13:51:36Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (72ms)  : 71, 74
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7360) - mean (69ms)  : 66, 71
     .   : milestone, 69,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (987ms)  : 967, 1007
     .   : milestone, 987,
    master - mean (1,033ms)  : 1009, 1058
     .   : milestone, 1033,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (109ms)  : 107, 110
     .   : milestone, 109,
    master - mean (107ms)  : 105, 108
     .   : milestone, 107,

    section Baseline
    This PR (7360) - mean (107ms)  : 105, 110
     .   : milestone, 107,
    master - mean (106ms)  : 104, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (709ms)  : 700, 719
     .   : milestone, 709,
    master - mean (725ms)  : 715, 735
     .   : milestone, 725,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (96ms)  : 94, 97
     .   : milestone, 96,
    master - mean (94ms)  : 93, 96
     .   : milestone, 94,

    section Baseline
    This PR (7360) - mean (95ms)  : 93, 97
     .   : milestone, 95,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (663ms)  : 653, 673
     .   : milestone, 663,
    master - mean (681ms)  : 673, 690
     .   : milestone, 681,

gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (95ms)  : 93, 96
     .   : milestone, 95,
    master - mean (94ms)  : 92, 95
     .   : milestone, 94,

    section Baseline
    This PR (7360) - mean (94ms)  : 92, 96
     .   : milestone, 94,
    master - mean (93ms)  : 91, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (599ms)  : 590, 609
     .   : milestone, 599,
    master - mean (625ms)  : 615, 634
     .   : milestone, 625,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (214ms)  : 201, 227
     .   : milestone, 214,
    master - mean (208ms)  : 198, 219
     .   : milestone, 208,

    section Baseline
    This PR (7360) - mean (208ms)  : 197, 219
     .   : milestone, 208,
    master - mean (202ms)  : 191, 213
     .   : milestone, 202,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (1,184ms)  : 1148, 1221
     .   : milestone, 1184,
    master - mean (1,216ms)  : 1169, 1263
     .   : milestone, 1216,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (296ms)  : 281, 311
     .   : milestone, 296,
    master - mean (287ms)  : 276, 298
     .   : milestone, 287,

    section Baseline
    This PR (7360) - mean (295ms)  : 281, 309
     .   : milestone, 295,
    master - mean (292ms)  : 278, 306
     .   : milestone, 292,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (960ms)  : 922, 998
     .   : milestone, 960,
    master - mean (968ms)  : 934, 1002
     .   : milestone, 968,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (290ms)  : 274, 305
     .   : milestone, 290,
    master - mean (285ms)  : 271, 299
     .   : milestone, 285,

    section Baseline
    This PR (7360) - mean (292ms)  : 279, 305
     .   : milestone, 292,
    master - mean (284ms)  : 269, 299
     .   : milestone, 284,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (936ms)  : 902, 970
     .   : milestone, 936,
    master - mean (949ms)  : 899, 1000
     .   : milestone, 949,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7360) - mean (286ms)  : 273, 299
     .   : milestone, 286,
    master - mean (291ms)  : 274, 308
     .   : milestone, 291,

    section Baseline
    This PR (7360) - mean (288ms)  : 276, 300
     .   : milestone, 288,
    master - mean (289ms)  : 269, 308
     .   : milestone, 289,

    section CallTarget+Inlining+NGEN
    This PR (7360) - mean (849ms)  : 804, 893
     .   : milestone, 849,
    master - mean (873ms)  : 830, 917
     .   : milestone, 873,

daniel-romano-DD · 2025-08-11T14:47:11Z

tracer/src/Datadog.Trace/Configuration/ConfigurationSources/CompositeConfigurationSource.cs

-            => _sources
-              .Select(source => source.GetString(key, telemetry, validator, recordValue))
-              .FirstOrDefault(value => value.IsValid, ConfigurationResult<string>.NotFound());
+        {


The only difference in all these GetXXX is the value retrieving function, which could be a lambda, to avoid the n-plicated code

Yeah, I've gone back and forth on the lambda thing. I extracted these to lambdas in ConfigurationBuilder and it makes everything seem that much more complicated, that I'm really not sure which is best. Add to that the fact that there's slightly different behaviour between each of the cases and it ends up really not being clear-cut which approach is better tbh.

daniel-romano-DD

Nice work

pr-commenter · 2025-08-11T18:10:16Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7360 compared to master:

1 benchmarks are slower, with geometric mean 1.137
14 benchmarks have fewer allocations
5 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1	5.68 KB	5.72 KB	37 B	0.65%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	10.5μs	57.7ns	331ns	0	0	0	5.49 KB
master	`StartStopWithChild`	netcoreapp3.1	14.2μs	57.1ns	206ns	0	0	0	5.68 KB
master	`StartStopWithChild`	net472	21.7μs	121ns	734ns	0.987	0.329	0.11	6.03 KB
#7360	`StartStopWithChild`	net6.0	10.8μs	60.9ns	395ns	0	0	0	5.5 KB
#7360	`StartStopWithChild`	netcoreapp3.1	13.4μs	69.5ns	354ns	0	0	0	5.72 KB
#7360	`StartStopWithChild`	net472	21.8μs	124ns	841ns	0.972	0.324	0.108	6.03 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	3.35 KB	3.31 KB	-46 B	-1.37%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	932μs	103ns	355ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.03ms	67.4ns	261ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.21ms	92.5ns	358ns	3.35 KB
#7360	`WriteAndFlushEnrichedTraces`	net6.0	950μs	132ns	512ns	2.7 KB
#7360	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.04ms	1.33μs	4.97μs	2.7 KB
#7360	`WriteAndFlushEnrichedTraces`	net472	1.2ms	134ns	500ns	3.31 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0	182.92 KB	176.95 KB	-5.97 KB	-3.26%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0	179.42 KB	173.45 KB	-5.97 KB	-3.33%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑netcoreapp3.1	186.66 KB	179.06 KB	-7.6 KB	-4.07%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑netcoreapp3.1	183.24 KB	175.64 KB	-7.6 KB	-4.15%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472	213.49 KB	199.31 KB	-14.18 KB	-6.64%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472	209.98 KB	195.79 KB	-14.18 KB	-6.75%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	324μs	1.72μs	8.94μs	0	0	179.42 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	478μs	1.52μs	5.46μs	0	0	183.24 KB
master	`AllCycleSimpleBody`	net472	424μs	112ns	420ns	33.3	2.08	209.98 KB
master	`AllCycleMoreComplexBody`	net6.0	340μs	619ns	2.4μs	0	0	182.92 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	479μs	1.38μs	5.35μs	0	0	186.66 KB
master	`AllCycleMoreComplexBody`	net472	432μs	175ns	679ns	32.3	0	213.49 KB
master	`ObjectExtractorSimpleBody`	net6.0	328ns	1.74ns	8.5ns	0	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	399ns	1.22ns	4.71ns	0	0	272 B
master	`ObjectExtractorSimpleBody`	net472	314ns	0.951ns	3.68ns	0.0432	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.4μs	32.1ns	147ns	0	0	3.79 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	8.3μs	29.7ns	115ns	0	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.71μs	1.44ns	5.39ns	0.571	0	3.8 KB
#7360	`AllCycleSimpleBody`	net6.0	322μs	560ns	2.17μs	0	0	173.45 KB
#7360	`AllCycleSimpleBody`	netcoreapp3.1	464μs	1.34μs	5.21μs	0	0	175.64 KB
#7360	`AllCycleSimpleBody`	net472	427μs	77.4ns	300ns	29.2	0	195.79 KB
#7360	`AllCycleMoreComplexBody`	net6.0	326μs	1.6μs	6.79μs	0	0	176.95 KB
#7360	`AllCycleMoreComplexBody`	netcoreapp3.1	492μs	1.1μs	4.27μs	0	0	179.06 KB
#7360	`AllCycleMoreComplexBody`	net472	436μs	103ns	386ns	30.2	0	199.31 KB
#7360	`ObjectExtractorSimpleBody`	net6.0	330ns	0.0729ns	0.282ns	0	0	280 B
#7360	`ObjectExtractorSimpleBody`	netcoreapp3.1	401ns	1.93ns	8.17ns	0	0	272 B
#7360	`ObjectExtractorSimpleBody`	net472	303ns	0.0212ns	0.0822ns	0.0443	0	281 B
#7360	`ObjectExtractorMoreComplexBody`	net6.0	6.35μs	31.8ns	149ns	0	0	3.78 KB
#7360	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.89μs	30.8ns	119ns	0	0	3.69 KB
#7360	`ObjectExtractorMoreComplexBody`	net472	6.89μs	8.56ns	33.1ns	0.582	0	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	76.4μs	209ns	810ns	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	97.1μs	126ns	488ns	0	32.4 KB
master	`EncodeArgs`	net472	114μs	210ns	812ns	4.6	32.51 KB
master	`EncodeLegacyArgs`	net6.0	143μs	49.5ns	179ns	0	2.15 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	200μs	144ns	560ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	264μs	57.8ns	216ns	0	2.16 KB
#7360	`EncodeArgs`	net6.0	78μs	256ns	991ns	0	32.4 KB
#7360	`EncodeArgs`	netcoreapp3.1	97.6μs	44.8ns	173ns	0	32.4 KB
#7360	`EncodeArgs`	net472	109μs	9.8ns	35.3ns	4.93	32.5 KB
#7360	`EncodeLegacyArgs`	net6.0	142μs	20.1ns	75.2ns	0	2.15 KB
#7360	`EncodeLegacyArgs`	netcoreapp3.1	198μs	146ns	526ns	0	2.14 KB
#7360	`EncodeLegacyArgs`	net472	263μs	35.3ns	137ns	0	2.16 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	396μs	84.6ns	328ns	4.55 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	821μs	10.7μs	106μs	4.48 KB
master	`RunWafRealisticBenchmark`	net472	434μs	125ns	483ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	291μs	126ns	490ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	304μs	66.7ns	241ns	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	314μs	35.3ns	137ns	2.29 KB
#7360	`RunWafRealisticBenchmark`	net6.0	402μs	175ns	656ns	4.56 KB
#7360	`RunWafRealisticBenchmark`	netcoreapp3.1	819μs	12.2μs	121μs	4.48 KB
#7360	`RunWafRealisticBenchmark`	net472	434μs	58.9ns	228ns	4.66 KB
#7360	`RunWafRealisticBenchmarkWithAttack`	net6.0	286μs	153ns	574ns	2.24 KB
#7360	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	299μs	47.3ns	177ns	2.22 KB
#7360	`RunWafRealisticBenchmarkWithAttack`	net472	314μs	65.1ns	244ns	2.29 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	62.1μs	79.1ns	306ns	14.52 KB
master	`SendRequest`	netcoreapp3.1	71.2μs	93.9ns	325ns	17.42 KB
master	`SendRequest`	net472	0.011ns	0.00208ns	0.00805ns	0 b
#7360	`SendRequest`	net6.0	61.3μs	47.9ns	186ns	14.52 KB
#7360	`SendRequest`	netcoreapp3.1	70.1μs	104ns	389ns	17.42 KB
#7360	`SendRequest`	net472	0.0128ns	0.00244ns	0.00943ns	0 b

Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0	6 B	2 B	-4 B	-66.67%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	4 B	1 B	-3 B	-75.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472	73 B	0 b	-73 B	-100.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472	47 B	0 b	-47 B	-100.00%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`OriginalCharSlice`	net6.0	1.9ms	7.69μs	29.8μs	0	640.01 KB
master	`OriginalCharSlice`	netcoreapp3.1	2.13ms	4.49μs	16.8μs	0	640 KB
master	`OriginalCharSlice`	net472	2.69ms	280ns	1.01μs	100	641.95 KB
master	`OptimizedCharSlice`	net6.0	1.4ms	298ns	1.15μs	0	6 B
master	`OptimizedCharSlice`	netcoreapp3.1	1.68ms	149ns	576ns	0	1 B
master	`OptimizedCharSlice`	net472	1.98ms	388ns	1.5μs	0	73 B
master	`OptimizedCharSliceWithPool`	net6.0	839μs	41ns	159ns	0	4 B
master	`OptimizedCharSliceWithPool`	netcoreapp3.1	837μs	66.2ns	256ns	0	1 B
master	`OptimizedCharSliceWithPool`	net472	1.15ms	51.9ns	201ns	0	47 B
#7360	`OriginalCharSlice`	net6.0	1.88ms	4.16μs	15μs	0	640.01 KB
#7360	`OriginalCharSlice`	netcoreapp3.1	2.22ms	8.28μs	32.1μs	0	640 KB
#7360	`OriginalCharSlice`	net472	2.71ms	393ns	1.42μs	100	641.95 KB
#7360	`OptimizedCharSlice`	net6.0	1.41ms	299ns	1.16μs	0	2 B
#7360	`OptimizedCharSlice`	netcoreapp3.1	1.74ms	88ns	341ns	0	1 B
#7360	`OptimizedCharSlice`	net472	2.02ms	139ns	521ns	0	0 b
#7360	`OptimizedCharSliceWithPool`	net6.0	838μs	77.6ns	300ns	0	1 B
#7360	`OptimizedCharSliceWithPool`	netcoreapp3.1	810μs	234ns	908ns	0	1 B
#7360	`OptimizedCharSliceWithPool`	net472	1.13ms	107ns	413ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	56.02 KB	55.71 KB	-306 B	-0.55%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	42.05 KB	41.78 KB	-271 B	-0.64%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	681μs	2.49μs	9.66μs	0	41.68 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	688μs	982ns	3.8μs	0	42.05 KB
master	`WriteAndFlushEnrichedTraces`	net472	978μs	2.32μs	8.67μs	4.46	56.02 KB
#7360	`WriteAndFlushEnrichedTraces`	net6.0	675μs	985ns	3.69μs	0	41.71 KB
#7360	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	760μs	4.33μs	30.6μs	0	41.78 KB
#7360	`WriteAndFlushEnrichedTraces`	net472	937μs	4.32μs	16.2μs	4.46	55.71 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	2μs	1.23ns	4.61ns	0	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.71μs	8.49ns	32.9ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.92μs	2.33ns	9.02ns	0.145	0.0145	987 B
#7360	`ExecuteNonQuery`	net6.0	1.87μs	7.71ns	28.8ns	0	0	1.02 KB
#7360	`ExecuteNonQuery`	netcoreapp3.1	2.48μs	5.98ns	23.2ns	0	0	1.02 KB
#7360	`ExecuteNonQuery`	net472	2.85μs	6.05ns	23.4ns	0.156	0.0142	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.7μs	7.92ns	30.7ns	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	2.27μs	8.65ns	32.4ns	0	1.03 KB
master	`CallElasticsearch`	net472	3.56μs	1.41ns	5.29ns	0.159	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.78μs	8.74ns	37.1ns	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.49μs	6.66ns	25.8ns	0	1.08 KB
master	`CallElasticsearchAsync`	net472	3.73μs	1.97ns	7.63ns	0.169	1.1 KB
#7360	`CallElasticsearch`	net6.0	1.77μs	2.03ns	7.87ns	0	1.03 KB
#7360	`CallElasticsearch`	netcoreapp3.1	2.29μs	8.67ns	33.6ns	0	1.03 KB
#7360	`CallElasticsearch`	net472	3.72μs	2.89ns	10.8ns	0.147	1.04 KB
#7360	`CallElasticsearchAsync`	net6.0	1.91μs	10.1ns	52.3ns	0	1.01 KB
#7360	`CallElasticsearchAsync`	netcoreapp3.1	2.44μs	11ns	42.5ns	0	1.08 KB
#7360	`CallElasticsearchAsync`	net472	4.05μs	6.14ns	23.8ns	0.16	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.8μs	1.67ns	6.47ns	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	2.38μs	11.5ns	50.2ns	0	952 B
master	`ExecuteAsync`	net472	2.53μs	3.08ns	11.9ns	0.14	915 B
#7360	`ExecuteAsync`	net6.0	1.88μs	6.01ns	23.3ns	0	952 B
#7360	`ExecuteAsync`	netcoreapp3.1	2.48μs	3.41ns	13.2ns	0	952 B
#7360	`ExecuteAsync`	net472	2.6μs	3.83ns	14.8ns	0.142	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	6.85μs	8.6ns	33.3ns	0	2.36 KB
master	`SendAsync`	netcoreapp3.1	8.35μs	28.3ns	110ns	0	2.9 KB
master	`SendAsync`	net472	12.6μs	12.9ns	50.1ns	0.503	3.18 KB
#7360	`SendAsync`	net6.0	6.93μs	3.82ns	13.8ns	0	2.36 KB
#7360	`SendAsync`	netcoreapp3.1	8.67μs	17.1ns	66.3ns	0	2.9 KB
#7360	`SendAsync`	net472	12.3μs	9.4ns	36.4ns	0.494	3.18 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7360

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	1.137	457,900.00	520,550.00

More allocations ⚠️ in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	57.34 KB	65.54 KB	8.19 KB	14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	254.85 KB	271.87 KB	17.02 KB	6.68%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	43.15 KB	45.51 KB	2.36 KB	5.47%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	43.78 KB	44.06 KB	280 B	0.64%

Fewer allocations 🎉 in #7360

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472	287.58 KB	278.53 KB	-9.05 KB	-3.15%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	48.2μs	270ns	1.71μs	43.78 KB
master	`StringConcatBenchmark`	netcoreapp3.1	55.1μs	751ns	7.47μs	43.15 KB
master	`StringConcatBenchmark`	net472	58.7μs	137ns	496ns	57.34 KB
master	`StringConcatAspectBenchmark`	net6.0	459μs	1.82μs	6.58μs	257.62 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	483μs	1.85μs	6.68μs	254.85 KB
master	`StringConcatAspectBenchmark`	net472	404μs	1.84μs	9.22μs	287.58 KB
#7360	`StringConcatBenchmark`	net6.0	48.6μs	225ns	1.35μs	44.06 KB
#7360	`StringConcatBenchmark`	netcoreapp3.1	50.3μs	317ns	3.02μs	45.51 KB
#7360	`StringConcatBenchmark`	net472	57.5μs	197ns	762ns	65.54 KB
#7360	`StringConcatAspectBenchmark`	net6.0	522μs	2.09μs	8.88μs	257.7 KB
#7360	`StringConcatAspectBenchmark`	netcoreapp3.1	503μs	2.45μs	11μs	271.87 KB
#7360	`StringConcatAspectBenchmark`	net472	414μs	2.32μs	14.6μs	278.53 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.53μs	13.2ns	59.2ns	0	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	3.37μs	13.9ns	54ns	0	1.7 KB
master	`EnrichedLog`	net472	3.94μs	5.49ns	21.3ns	0.257	1.64 KB
#7360	`EnrichedLog`	net6.0	2.57μs	11.9ns	49.2ns	0	1.7 KB
#7360	`EnrichedLog`	netcoreapp3.1	3.45μs	16ns	63.8ns	0	1.7 KB
#7360	`EnrichedLog`	net472	3.86μs	2.99ns	11.6ns	0.251	1.64 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	122μs	119ns	462ns	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	127μs	68.9ns	258ns	4.31 KB
master	`EnrichedLog`	net472	168μs	101ns	390ns	4.52 KB
#7360	`EnrichedLog`	net6.0	122μs	114ns	413ns	4.31 KB
#7360	`EnrichedLog`	netcoreapp3.1	128μs	159ns	596ns	4.31 KB
#7360	`EnrichedLog`	net472	166μs	119ns	447ns	4.52 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.93μs	7.71ns	28.9ns	0	2.26 KB
master	`EnrichedLog`	netcoreapp3.1	6.57μs	17.5ns	67.8ns	0	2.26 KB
master	`EnrichedLog`	net472	7.45μs	6.72ns	26ns	0.297	2.08 KB
#7360	`EnrichedLog`	net6.0	5.02μs	24.7ns	98.9ns	0	2.26 KB
#7360	`EnrichedLog`	netcoreapp3.1	6.9μs	11.6ns	45ns	0	2.26 KB
#7360	`EnrichedLog`	net472	7.63μs	6.23ns	24.1ns	0.304	2.08 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	2.05μs	9.2ns	35.6ns	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	2.62μs	9.75ns	36.5ns	0	1.2 KB
master	`SendReceive`	net472	3.14μs	2.47ns	9.58ns	0.189	1.2 KB
#7360	`SendReceive`	net6.0	2.03μs	9.96ns	43.4ns	0	1.2 KB
#7360	`SendReceive`	netcoreapp3.1	2.68μs	8.97ns	34.7ns	0	1.2 KB
#7360	`SendReceive`	net472	3.2μs	5.68ns	22ns	0.19	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.2μs	7.17ns	26.8ns	0	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	5.64μs	16ns	61.9ns	0	1.63 KB
master	`EnrichedLog`	net472	6.77μs	5.73ns	22.2ns	0.303	2.03 KB
#7360	`EnrichedLog`	net6.0	4.25μs	10.1ns	37.7ns	0	1.58 KB
#7360	`EnrichedLog`	netcoreapp3.1	5.61μs	14.4ns	55.7ns	0	1.63 KB
#7360	`EnrichedLog`	net472	6.73μs	7.54ns	29.2ns	0.303	2.03 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	737ns	3.89ns	20.6ns	0	576 B
master	`StartFinishSpan`	netcoreapp3.1	929ns	4.85ns	25.7ns	0	576 B
master	`StartFinishSpan`	net472	929ns	0.903ns	3.5ns	0.0888	578 B
master	`StartFinishScope`	net6.0	901ns	4.31ns	17.2ns	0	696 B
master	`StartFinishScope`	netcoreapp3.1	1.17μs	6.39ns	34.4ns	0	696 B
master	`StartFinishScope`	net472	1.12μs	0.669ns	2.59ns	0.102	658 B
#7360	`StartFinishSpan`	net6.0	760ns	3.74ns	15ns	0	576 B
#7360	`StartFinishSpan`	netcoreapp3.1	937ns	4.96ns	25.7ns	0	576 B
#7360	`StartFinishSpan`	net472	903ns	0.455ns	1.7ns	0.0907	578 B
#7360	`StartFinishScope`	net6.0	891ns	0.407ns	1.52ns	0	696 B
#7360	`StartFinishScope`	netcoreapp3.1	1.12μs	5.69ns	24.8ns	0	696 B
#7360	`StartFinishScope`	net472	1.1μs	1.37ns	5.29ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.04μs	0.615ns	2.3ns	0	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.41μs	6.88ns	27.5ns	0	696 B
master	`RunOnMethodBegin`	net472	1.42μs	0.845ns	3.27ns	0.0996	658 B
#7360	`RunOnMethodBegin`	net6.0	1.01μs	5.1ns	21.6ns	0	696 B
#7360	`RunOnMethodBegin`	netcoreapp3.1	1.35μs	6.71ns	27.7ns	0	696 B
#7360	`RunOnMethodBegin`	net472	1.41μs	0.576ns	2.15ns	0.0988	658 B

This is necessary so that we can "correct" telemetry that is overridden in the CompositeConfigurationSource

…ig for all of them

andrewlock marked this pull request as ready for review August 7, 2025 14:01

andrewlock requested a review from a team as a code owner August 7, 2025 14:01

daniel-romano-DD reviewed Aug 11, 2025

View reviewed changes

daniel-romano-DD approved these changes Aug 11, 2025

View reviewed changes

andrewlock requested a review from a team as a code owner August 11, 2025 17:09

link04 approved these changes Aug 11, 2025

View reviewed changes

Base automatically changed from andrew/more-telemetry-refactor to master August 12, 2025 14:24

andrewlock requested a review from a team as a code owner August 12, 2025 14:24

andrewlock added 6 commits August 12, 2025 15:26

Expose Origin on IConfigurationSource

0a4a77e

Add "telemetry override" parameter to ConfigurationResult.

3be43ec

This is necessary so that we can "correct" telemetry that is overridden in the CompositeConfigurationSource

Search the configuration sources in reverse, to ensure we record conf…

4864538

…ig for all of them

"Fix" ordering of recorded telemetry

f810638

TEMP test against system tests

af3765d

Fix incorrect use of lambda in bool accessor

7071dcb

andrewlock force-pushed the andrew/config-chaining branch from ce0516c to 7071dcb Compare August 12, 2025 14:26

Add additional integration test

bb84c83

Implement "enahanced telemetry reporting" #7360

Are you sure you want to change the base?

Implement "enahanced telemetry reporting" #7360

Uh oh!

Conversation

andrewlock commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

dd-trace-dotnet-ci-bot bot commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

daniel-romano-DD Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

daniel-romano-DD left a comment

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

More allocations ⚠️ in #7360

Raw results

Fewer allocations 🎉 in #7360

Raw results

Fewer allocations 🎉 in #7360

Raw results

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #7360

Raw results

Fewer allocations 🎉 in #7360

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7360

More allocations ⚠️ in #7360

Fewer allocations 🎉 in #7360

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

Uh oh!

andrewlock commented Aug 7, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot bot commented Aug 7, 2025 •

edited

Loading

pr-commenter bot commented Aug 11, 2025 •

edited

Loading