Add instrumentor for Azure Kusto #3591

MattKotsenas · 2025-12-08T19:37:43Z

Changes

Adds an initial implementation of traces and metrics for any client built on top of the Microsoft.Azure.Kusto.Cloud.Platform package (namely Microsoft.Azure.Kusto.Data and Microsoft.Azure.Kusto.Ingest).

The library generally follows the semantic conventions for dbs for both traces and metrics, importantly, it does both query summarization and sanitization. It does not support adding query parameters as tags, as those aren't available in Kusto's internal custom tracing framework. Test coverage includes unit tests, benchmarks both for end-to-end instrumentation and query processing specific, and integration tests using the existing pattern of Testcontainers.

The package is not strong-name signed because it uses Microsoft.Azure.Kusto.Language for query processing, which also is not strong-name signed.

Merge requirement checklist

CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
Unit tests added/updated
Appropriate CHANGELOG.md files updated for non-trivial changes
Changes in public API reviewed (if applicable)

Add an option to disable query summarization

…by HttpClient)

…are made

codecov · 2025-12-08T19:38:54Z

Codecov Report

❌ Patch coverage is 91.04478% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.41%. Comparing base (48d18f6) to head (0256577).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...n.Kusto/Implementation/KustoTraceRecordListener.cs	83.78%	18 Missing ⚠️
.../Implementation/KustoInstrumentationEventSource.cs	0.00%	11 Missing ⚠️
...on.Kusto/Implementation/TruncatingStringBuilder.cs	97.36%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3591      +/-   ##
==========================================
+ Coverage   71.22%   71.41%   +0.19%     
==========================================
  Files         442      467      +25     
  Lines       17516    17902     +386     
==========================================
+ Hits        12475    12785     +310     
- Misses       5041     5117      +76

Flag	Coverage Δ
unittests-Contrib.Shared.Tests	`86.54% <ø> (ø)`
unittests-Exporter.Geneva	`53.64% <ø> (-0.14%)`	⬇️
unittests-Exporter.InfluxDB	`95.14% <ø> (ø)`
unittests-Exporter.Instana	`74.86% <ø> (ø)`
unittests-Exporter.OneCollector	`94.62% <ø> (ø)`
unittests-Extensions	`90.65% <ø> (ø)`
unittests-Extensions.Enrichment	`100.00% <ø> (ø)`
unittests-Extensions.Enrichment.AspNetCore	`86.27% <ø> (ø)`
unittests-Extensions.Enrichment.Http	`94.33% <ø> (ø)`
unittests-Instrumentation.AWS	`83.80% <ø> (ø)`
unittests-Instrumentation.AspNet	`78.14% <ø> (ø)`
unittests-Instrumentation.AspNetCore	`71.69% <ø> (ø)`
unittests-Instrumentation.Cassandra	`23.52% <ø> (?)`
unittests-Instrumentation.ConfluentKafka	`14.10% <ø> (ø)`
unittests-Instrumentation.ElasticsearchClient	`80.12% <ø> (ø)`
unittests-Instrumentation.EntityFrameworkCore	`80.80% <ø> (ø)`
unittests-Instrumentation.EventCounters	`77.27% <ø> (ø)`
unittests-Instrumentation.GrpcCore	`91.42% <ø> (ø)`
unittests-Instrumentation.GrpcNetClient	`79.61% <ø> (ø)`
unittests-Instrumentation.Hangfire	`86.05% <ø> (ø)`
unittests-Instrumentation.Http	`74.22% <ø> (ø)`
unittests-Instrumentation.Kusto	`91.04% <91.04%> (?)`
unittests-Instrumentation.Owin	`88.62% <ø> (ø)`
unittests-Instrumentation.Process	`100.00% <ø> (ø)`
unittests-Instrumentation.Quartz	`78.76% <ø> (ø)`
unittests-Instrumentation.Runtime	`100.00% <ø> (ø)`
unittests-Instrumentation.ServiceFabricRemoting	`34.54% <ø> (ø)`
unittests-Instrumentation.SqlClient	`87.29% <ø> (ø)`
unittests-Instrumentation.StackExchangeRedis	`71.80% <ø> (ø)`
unittests-Instrumentation.Wcf	`79.57% <ø> (ø)`
unittests-OpAmp.Client	`71.55% <ø> (ø)`
unittests-PersistentStorage	`76.58% <ø> (ø)`
unittests-Resources.AWS	`74.42% <ø> (ø)`
unittests-Resources.Azure	`85.31% <ø> (ø)`
unittests-Resources.Container	`67.34% <ø> (ø)`
unittests-Resources.Gcp	`71.42% <ø> (ø)`
unittests-Resources.Host	`71.85% <ø> (ø)`
unittests-Resources.OperatingSystem	`76.98% <ø> (ø)`
unittests-Resources.Process	`100.00% <ø> (ø)`
unittests-Resources.ProcessRuntime	`79.59% <ø> (ø)`
unittests-Sampler.AWS	`93.23% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...entation/InstrumentationHandleManagerExtensions.cs	`100.00% <100.00%> (ø)`
....Kusto/Implementation/KustoActivitySourceHelper.cs	`100.00% <100.00%> (ø)`
...ation.Kusto/Implementation/KustoInstrumentation.cs	`100.00% <100.00%> (ø)`
...rumentation.Kusto/Implementation/KustoProcessor.cs	`100.00% <100.00%> (ø)`
...ntation.Kusto/Implementation/KustoStatementInfo.cs	`100.00% <100.00%> (ø)`
...rumentation.Kusto/Implementation/SpanExtensions.cs	`100.00% <100.00%> (ø)`
...tion.Kusto/Implementation/TraceRecordExtensions.cs	`100.00% <100.00%> (ø)`
...entation.Kusto/Implementation/TraceRecordParser.cs	`100.00% <100.00%> (ø)`
...entation.Kusto/KustoMeterInstrumentationOptions.cs	`100.00% <100.00%> (ø)`
...entation.Kusto/KustoTraceInstrumentationOptions.cs	`100.00% <100.00%> (ø)`
... and 5 more

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

thompson-tomo · 2025-12-10T13:41:03Z

src/OpenTelemetry.Instrumentation.Kusto/Implementation/KustoTraceRecordListener.cs

+            activity?.DisplayName = operationName;
+            activity?.AddTag(KustoActivitySourceHelper.ClientRequestIdTagKey, record.Activity.ClientRequestId.ToString());
+
+            activity?.AddTag(SemanticConventions.AttributeDbSystemName, KustoActivitySourceHelper.DbSystem);


Are we sure about the value here? Kusto seems to be the query language and not what is being called. azure.data_explorer would appear to be more accurate.

I think it's worth raising to the group. I could go either way, but lean a bit towards retaining "Kusto" because it feels less ambiguous to me. The Saas product is called "Azure Data Explorer", but the client library is "Microsoft.Azure.Kusto.Data". The rich client UI is called "Kusto Explorer", but the web product is called "Azure Data Explorer Web UI". The provisioning library is "Azure.Provisioning.Kusto"

It's also worth calling out that you can query sources that are backed by Kusto clusters, but are not themselves the Azure Data Explorer Saas product, like Azure Log Analytics.

Also a reminder for myself, if we do change the name, we need to update a few other attributes as well to use the same prefix (namely the client request id).

thompson-tomo · 2025-12-10T13:41:52Z

src/OpenTelemetry.Instrumentation.Kusto/Implementation/KustoTraceRecordListener.cs

+
+            activity?.AddTag(SemanticConventions.AttributeDbSystemName, KustoActivitySourceHelper.DbSystem);
+            activity?.AddTag(SemanticConventions.AttributeDbOperationName, operationName);
+            meterTags.Add(SemanticConventions.AttributeDbSystemName, KustoActivitySourceHelper.DbSystem);


Are we sure about the value here? Kusto seems to be the query language and not what is being called. azure.data_explorer would appear to be more accurate.

MattKotsenas added 30 commits December 5, 2025 15:23

Scaffolding

e37dd06

Add initial implementation

06a6cf0

Add basic tests

350f9ef

Add option for query text

8cb5db7

Refactor and clean up

d1e1916

Fix removal and clean up tests

eed9aa2

Clean up README

d533315

Add tests for exceptions

577510f

Refactor to use a single registration

8738c18

Add test to verify when not registered

c7637de

Clean up tests

55d3a46

Update db.system.name to follow conventions

07c4bdd

Add query summarization and sanitization

2a39a24

Add Benchmarks project

8476dd6

Use record structs

294f5c6

Perf: Use a truncating string builder

06afa83

Perf: Simplify text edits

d071901

Perf: Reuse GlobalState

12d1655

Add option to disable summarization

aa68fcf

Add an option to disable query summarization

Do not sanitize parameterized queries

aa1555c

Extrace TraceRecord parsing to own class and add tests

4f09e1f

Handle embedded delimiters in query text

9c3ef2f

Reduce depedency to Kusto.Cloud.Platform and bump version

cecc02c

Add database to attributes

020acb3

Add test coverage for TraceRecordParser

e5b9a4e

Add enrichment callback

6012e7a

Update readme to document options

872b06d

Separate server.address from server.port

898ad32

Replace custom Uri parsing with Uri.TryCreate

756421a

Add full instrumentation benchmark

7873ff6

MattKotsenas added 14 commits December 5, 2025 15:49

Update README with metrics

d17d403

Add error.type as attribute on trace

552e7fc

Fix semantic convention for metric name

c3d2923

Refactor so that metrics attributes match spans

c709cf3

Remove url.full attribute which is against spec (and will be covered …

dd15076

…by HttpClient)

Fix instrumentation benchmarks

5ea3578

Clean up registration

6facd6a

Clean up package versions

2e3a3cf

Add comments and remove dead code

efa14dc

Add metrics-only and trace-only integration tests

42cbf91

Add additional logging for errors

8ad211e

Move initialization earlier to ensure it is before any kusto clients …

eb74496

…are made

Fix naming of ActivitySource and Meter

e36e1c4

Split InstrumentationOptions for Trace and Meter

524f04d

MattKotsenas requested a review from a team as a code owner December 8, 2025 19:37

github-actions bot added infra Infra work - CI/CD, code coverage, linters dependencies Pull requests that update a dependency file perf Performance related labels Dec 8, 2025

MattKotsenas force-pushed the feature/kusto-instrumentor branch 2 times, most recently from 57b046e to 0aef4ac Compare December 8, 2025 20:30

Update docs

77819be

MattKotsenas force-pushed the feature/kusto-instrumentor branch from 0aef4ac to 77819be Compare December 8, 2025 21:28

Update build files to add new project

0256577

thompson-tomo reviewed Dec 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add instrumentor for Azure Kusto #3591

Add instrumentor for Azure Kusto #3591

Uh oh!

MattKotsenas commented Dec 8, 2025

Uh oh!

codecov bot commented Dec 8, 2025 •

edited

Loading

Uh oh!

thompson-tomo Dec 10, 2025

Uh oh!

MattKotsenas Dec 10, 2025

Uh oh!

MattKotsenas Dec 10, 2025

Uh oh!

thompson-tomo Dec 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add instrumentor for Azure Kusto #3591

Are you sure you want to change the base?

Add instrumentor for Azure Kusto #3591

Uh oh!

Conversation

MattKotsenas commented Dec 8, 2025

Changes

Merge requirement checklist

Uh oh!

codecov bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

thompson-tomo Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

MattKotsenas Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

MattKotsenas Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

thompson-tomo Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Dec 8, 2025 •

edited

Loading