Skip to content

[db] Add docs for context info propagation #2236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 24 commits into from
Closed

[db] Add docs for context info propagation #2236

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1529979
[db] Add docs for context info propagation
May 6, 2025
ab6e386
Update docs/database/database-spans.md
sincejune May 7, 2025
08b02a8
Update docs/database/sql-server.md
sincejune May 7, 2025
2c26648
Merge branch 'main' into db-propagate-context-info
May 7, 2025
0ccfda5
feedback
May 7, 2025
a6e65f4
feedback
May 7, 2025
886aa37
feedback
May 7, 2025
f633fa2
feedback
May 7, 2025
5b835f8
Update docs/database/database-spans.md
sincejune May 7, 2025
e021083
Update docs/database/database-spans.md
sincejune May 7, 2025
1bcbf0d
feedback
May 7, 2025
e6803f5
Update docs/database/database-spans.md
sincejune May 8, 2025
f986ee2
add single quotes
May 8, 2025
8794c1d
Add co-author
XSAM May 13, 2025
ec00e3c
feedback
May 19, 2025
fa6cf6d
fix format
May 19, 2025
2c0e01a
update baggage.service.name example
May 19, 2025
430957b
Keep only `service.name` for general databases
XSAM May 21, 2025
7f05f5e
Update sql-server doc
XSAM May 21, 2025
5534272
Adjust font format
XSAM May 21, 2025
9876582
Add an example for sql server
XSAM May 21, 2025
89a4baf
Merge remote-tracking branch 'upstream/main' into db-propagate-contex…
XSAM May 21, 2025
0e7ff53
Fix CI
XSAM May 21, 2025
8effebc
Merge remote-tracking branch 'upstream/main' into db-propagate-contex…
XSAM May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .chloggen/db-propagate-context-info.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: db

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add doc for context information propagation

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [2162]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
39 changes: 39 additions & 0 deletions docs/database/database-spans.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ linkTitle: Spans
- [Notes and well-known identifiers for `db.system.name`](#notes-and-well-known-identifiers-for-dbsystemname)
- [Sanitization of `db.query.text`](#sanitization-of-dbquerytext)
- [Generating a summary of the query](#generating-a-summary-of-the-query)
- [Propagating context to databases](#propagating-context-to-databases)
- [Recommended sqlcommenter attributes](#recommended-sqlcommenter-attributes)
- [Semantic conventions for specific database technologies](#semantic-conventions-for-specific-database-technologies)

<!-- tocstop -->
Expand Down Expand Up @@ -478,6 +480,43 @@ Semantic conventions for individual database systems or specialized instrumentat
MAY specify a different `db.query.summary` format as long as produced summary remains
relatively short and its cardinality remains low comparing to the `db.query.text`.

## Propagating context to databases

**Status**: [Development][DocumentStatus]

Instrumentations SHOULD propagate the context information to the SQL queries following [sqlcommenter](https://google.github.io/sqlcommenter/spec/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to share a learning from my org. We've implemented a sqlcommenter-like approach for our instrumentations and we found it more beneficial to prepend the sql comment rather than append to the query text. This runs counter to the sqlcommenter spec, but we found that most databases truncate the query in views like pg_stat_activity, so prepending the attributes offered a better guarantee that they wouldn't be truncated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zacharycmontoya Thank you for pointing this out!

I think it makes sense to prepend the sql comment instead. I'll update this PR accordingly if there are no objections. cc @open-telemetry/semconv-db-approvers @XSAM

Copy link
Member

@XSAM XSAM May 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using prepend conflicts the spec of sql commenter. We might need to change the spec of sql commenter if prepend approach is superior than append approach.

@zacharycmontoya Could you elaborate more about this? How serious and often the truncate will happen to certain databases if we use append.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres limits query text to 1024 characters by default, but you can increase it.

Copy link
Member

@nenadnoveljic nenadnoveljic May 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prepended comments break pg_hint_plan functionality:

EXPLAIN /*+ IndexScan(t) */  /* a */ SELECT * FROM t WHERE n = 1;
                         QUERY PLAN                          
-------------------------------------------------------------
 Index Scan using i on t  (cost=0.15..36.38 rows=13 width=4)
   Index Cond: (n = 1)
(2 rows)

EXPLAIN /* a */ /*+ IndexScan(t) */  SELECT * FROM t WHERE n = 1;
                           QUERY PLAN                            
-----------------------------------------------------------------
 Bitmap Heap Scan on t  (cost=4.26..14.95 rows=13 width=4)
   Recheck Cond: (n = 1)
   ->  Bitmap Index Scan on i  (cost=0.00..4.25 rows=13 width=0)
         Index Cond: (n = 1)
(4 rows)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea is to make it flexible by allowing the implementation of propagation for a specific database to choose between append or prepend.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified this PR to incorporate the idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about pushing this as a standard in light of google/sqlcommenter#284

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to use sqlcommenter only with attributes that do not change frequently. In the SQL Server documentation, I have added the section below. @trask Does this plan seem good to you?

Instrumentations SHOULD make use of SET CONTEXT_INFO to add high-cardinality information(e.g. traceparent) as adding sql comments to queries changes their unique identifier in SQL Server.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this PR is still about the context propagation, right? trace-id + span-id are pretty much guaranteed to be unique

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmolkova The traceparent format is more widely used, so I prefer adopting that. What are your thoughts?


### Recommended sqlcommenter attributes

| Attribute | Type | Description | Require level | Stability |
|------------------------|--------|---------------------------------------|------------------------------|----------------------------------------------------------------|
| `baggage.service.name` | string | Logical name of the service [1] | `Conditionally Required` [2] | ![Development](https://img.shields.io/badge/-development-blue) |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

related: open-telemetry/oteps#247

| `traceparent` | string | The trace context of current span [3] | `Recommended` [4] | ![Development](https://img.shields.io/badge/-development-blue) |

**[1] `baggage.service.name`:** MUST be the same for all instances of horizontally scaled services. If the value was not specified, SDKs MUST fall back to `unknown_service:` concatenated with [process.executable.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/), e.g. `unknown_service:bash`. If `process.executable.name` is not available, the value MUST be set to `unknown_service`.

[Baggage](https://www.w3.org/TR/baggage/) means you can pass data across services and processes, making it available to add to traces, metrics, or logs in those services.

**[2] `baggage.service.name`:** SHOULD be set if `traceparent` cannot be injected due to performance issues.

**[3] `traceparent`:** MUST be in the [text format](https://www.w3.org/TR/trace-context/#traceparent-header).

**[4] `traceparent`:** `traceparent` has extremely high-cardinality. It's RECOMMENDED to propagate this info if the high-cardinality is safe for the database observabilty engine.

**Examples:**

- Query with `baggage.service.name`:

```sql
SELECT * FROM songs /* baggage.service.name=music-player%3Aplay */
```

- Query with `baggage.service.name` and `traceparent`

```sql
SELECT * FROM songs /* baggage.service.name=music-player%3Aplay, traceparent=00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01 */
```

## Semantic conventions for specific database technologies

More specific Semantic Conventions are defined for the following database technologies:
Expand Down
25 changes: 25 additions & 0 deletions docs/database/sql-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ linkTitle: SQL Server
<!-- toc -->

- [Spans](#spans)
- [Propagating context to SQL Server](#propagating-context-to-sql-server)
- [Recommended sqlcommenter attributes](#recommended-sqlcommenter-attributes)
- [Metrics](#metrics)

<!-- tocstop -->
Expand Down Expand Up @@ -152,6 +154,29 @@ and SHOULD be provided **at span creation time** (if provided at all):
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->

### Propagating context to SQL Server

**Status**: [Development][DocumentStatus]

Instrumentations SHOULD propagate the context information to the SQL queries following [sqlcommenter](https://google.github.io/sqlcommenter/spec/).

#### Recommended sqlcommenter attributes

| Attribute | Type | Description | Require level | Stability |
|------------------------|--------|---------------------------------------|-----------------------------|----------------------------------------------------------------|
| `baggage.service.name` | string | Logical name of the service [1] | `Conditionally Required`[2] | ![Development](https://img.shields.io/badge/-development-blue) |
| `traceparent` | string | The trace context of current span [3] | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |

**[1] `baggage.service.name`:** MUST be the same for all instances of horizontally scaled services. If the value was not specified, SDKs MUST fall back to `unknown_service:` concatenated with [process.executable.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/), e.g. `unknown_service:bash`. If `process.executable.name` is not available, the value MUST be set to `unknown_service`.

[Baggage](https://www.w3.org/TR/baggage/) means you can pass data across services and processes, making it available to add to traces, metrics, or logs in those services.

**[2] `baggage.service.name`:** SHOULD be set if `traceparent` cannot be injected due to performance issues.

**[3] `traceparent`:** MUST be in the [text format](https://www.w3.org/TR/trace-context/#traceparent-header).

Instrumentations SHOULD make use of [SET CONTEXT_INFO](https://learn.microsoft.com/en-us/sql/t-sql/statements/set-context-info-transact-sql?view=sql-server-ver16) to add high-cardinality information(e.g. `traceparent`) as adding sql comments to queries changes their unique identifier in SQL Server.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add to the guidance here that instrumentations will need to copy over the transaction (if present) to the new set context_info command. When we didn't do this, we ran into customer issues when there was an active transaction

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate more about what copy over the transaction is actually doing? Could we just run set context_info before every query?

Copy link
Contributor

@zacharycmontoya zacharycmontoya May 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the LOC I'm referencing for our C# client : https://github.com/DataDog/dd-trace-dotnet/pull/6233/files#diff-29ead36b0177265c725705ce75f5a441417f4c22e99f7080bc8ef0f486b723a7R145-R146

When creating a new set context_info command that we run before the original query, we simply copy the Transaction from the original DbCommand object to the new one. Otherwise, when executing the set context_info command we encountered a InvalidOperationException

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SET context_info must run on the same physical connection as the SQL statement it’s meant to instrument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated


## Metrics

Microsoft SQL Server client instrumentations SHOULD collect metrics according to the general
Expand Down
Loading