feat(contrib/gomongodb.org/mongo-driver): add optional query tag truncation #3679

colinking · 2025-06-20T17:32:11Z

What does this PR do?

This PR adds support for a WithMaxQuerySize option to the MongoDB module that configures truncation on the mongodb.query span tag.

Ideally, we default to enabling truncation. However, this PR currently maintains backwards-compatibility by defaulting to no truncation. I'll defer to the DataDog team on whether we can change this default.

I've also ported this behavior to the mongo-driver.v2 module.

Fixes #3672

Motivation

See: #3672

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
New code is free of linting errors. You can check this by running golangci-lint run locally.
Add an appropriate team label so this PR gets put in the right place for the release notes.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

### What does this PR do? This PR adds support for a `WithMaxQuerySize` method to the MongoDB package that enables truncation on the `mongodb.query` span tag. I also ported this behavior to the new `mongo-driver.v2` package which supports the v2 release of the Mongo drivers. Opening this PR here first for feedback from our team and then will open on the upstream repo. The default is to not truncate query span tags, to avoid breaking changes. I'll ask in the upstream PR about whether we could truncate by default instead. ### Motivation This reduces the memory footprint of tracing Mongo applications that write large documents.

This PR updates the behavior of `WithMaxQuerySize` when `max=0` to avoid attaching the query tag entirely. This is more intuitive ("max query size of zero") and gives folks a way to disable serializing the command entirely.

mtoffl01 · 2025-06-30T20:36:51Z

Hey @colinking ,

Thanks for your contribution!

The Datadog Agent has logic to manipulate db queries and I am concerned that truncation could break the agent logic.

That said, 100MB (referenced in #3672) is quite large, and I imagine the agent is flat-out rejecting these payloads.

So that we can find the best solution to your problem, can you tell me a little bit more about your user experience? What is the actual impact of these [very large] span tags?

Thanks!

contrib/go.mongodb.org/mongo-driver.v2/mongo/mongo.go

contrib/go.mongodb.org/mongo-driver.v2/mongo/mongo_test.go

darccio · 2025-07-15T11:08:24Z

contrib/go.mongodb.org/mongo-driver.v2/mongo/option.go

+// If max is < 0, the query is never truncated.
+//
+// Defaults to -1.


Following with my previous comments, I'd rather to make 0 value something more meaningful, and default to a higher max length instead of not truncating. As long as it's configurable, it's ok to do this kind of changes.

The question is "is it possible to have an empty query?". If not, I'd say that 0 should mean the same as a negative value. If it's possible, then 0 should set "" as tag value. WDYT?

The 0 value is currently meaningful, as that's the only way to specify you want to drop the query tag entirely.

In other words:

Negative value: do not truncate

Zero: do not attach the query tag

Positive value: attach the query tag, but truncate it

If y'all want to avoid truncation here (e.g. due to the obfuscation logic mentioned above), then another option is to drop the query tag if the length is beyond the configurable limit. Therefore, you always get a well-formed query.

darccio · 2025-08-26T14:14:06Z

@rarguelloF WDYT about this PR? Thanks!

colinking · 2025-12-12T22:02:03Z

Sorry for the delayed responses here y'all.

@mtoffl01: In our case, the process that was generating these spans was OOMing. You're probably right that the agent would reject such a large span, but in our case the span was never getting sent since the process OOMed first.

For context, we had some code that would generate these large bulk MongoDB writes. This code could get retried multiple times, all of which was occurring within a parent span. Therefore, the total memory required to hold this trace in memory was on the order of GBs.

We rolled out this change to production and haven't encountered any OOMs since and overall memory usage dropped significantly.

colinking · 2026-01-22T17:37:44Z

Just gently bumping this PR. Thanks in advance y'all. @darccio @rarguelloF @mtoffl01

darccio · 2026-01-26T15:45:23Z

@colinking Thanks for pinging us. I reviewed it again and I'd suggest a comment to maxQuerySize so it makes clear that -1 is the default value, and that 0 truncates the query entirely.

I'm approving this. Once the comment is added, I'll merge this.

colinking · 2026-01-26T17:06:35Z

Thanks @darccio. I've updated the WithMaxQuerySize comment in both versions of this library: https://github.com/DataDog/dd-trace-go/pull/3679/files#diff-c1317684bee762a6cc5ea40dc97501738e3db51d73464426db1203d18469669dR43-R52

contrib/go.mongodb.org/mongo-driver.v2/mongo/mongo.go

darccio · 2026-01-27T08:19:10Z

@codex review

chatgpt-codex-connector · 2026-01-27T08:22:28Z

Codex Review: Didn't find any major issues. Another round soon, please!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

darccio · 2026-01-27T09:17:17Z

@rarguelloF The PR is ready for merging. Are we good from @DataDog/apm-idm-go?

darccio · 2026-01-27T15:04:50Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-01-27T15:04:56Z

View all feedbacks in Devflow UI.

2026-01-27 15:04:56 UTC ℹ️ Start processing command /merge

2026-01-27 15:06:02 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 43m (p90).

2026-01-27 15:07:10 UTC 🚨 MergeQueue: This merge request is in error

There was an unexpected error while creating the working branch: cannot create empty commit

This could indicate that something doesn't work properly with the build system or that this one has reached its maximum capacity.
You can try to wait a bit and then re-add your pull request to the queue!

Details

Error: cannot create empty commit

FullStacktrace:
child workflow execution error (type: mergequeue_private.MergeQueue_BuildWorkingBranch, workflowID: 7d92ba44-7a19-4d53-98c1-1ad6dc47914a_66, runID: 6d769d1b-094c-4628-a853-76fe6fd19563, initiatedEventID: 66, startedEventID: 67): cannot create empty commit (type: MergeQueueError, retryable: false): activity error (type: github.GithubService_CreateEmptyCommit, scheduledEventID: 14, startedEventID: 15, identity: 1@github-worker-7587f6b96c-gg7n9@): GET https://api.github.com/repos/DataDog/dd-trace-go/git/commits/fa72b69e5371c19505a674205beb4e63f0dd9c0e: 401 Bad credentials [] (Request ID: A172:166343:151E0E2:5EF5ED5:6978D48E) (type: GitFailure, retryable: false): GET https://api.github.com/repos/DataDog/dd-trace-go/git/commits/fa72b69e5371c19505a674205beb4e63f0dd9c0e: 401 Bad credentials [] (type: ErrorResponse, retryable: true)

darccio · 2026-01-27T15:07:57Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-01-27T15:08:11Z

View all feedbacks in Devflow UI.

2026-01-27 15:08:10 UTC ℹ️ Start processing command /merge

2026-01-27 15:09:13 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 43m (p90).

2026-01-27 16:04:35 UTC ℹ️ MergeQueue: This merge request was merged

colinking marked this pull request as ready for review June 20, 2025 17:33

colinking requested review from a team as code owners June 20, 2025 17:33

colinking mentioned this pull request Jun 20, 2025

[BUG]: MongoDB query span tag should be truncated #3672

Open

Update WithMaxQuerySize(0) to not attach query (#2)

6a7995c

This PR updates the behavior of `WithMaxQuerySize` when `max=0` to avoid attaching the query tag entirely. This is more intuitive ("max query size of zero") and gives folks a way to disable serializing the command entirely.

mtoffl01 self-assigned this Jun 30, 2025

darccio reviewed Jul 15, 2025

View reviewed changes

Merge branch 'DataDog:main' into main

23ec97e

colinking requested a review from darccio December 12, 2025 22:32

Clarify WithMaxQuerySize comment

298e6cd

darccio approved these changes Jan 26, 2026

View reviewed changes

darccio reviewed Jan 26, 2026

View reviewed changes

contrib/go.mongodb.org/mongo-driver.v2/mongo/mongo.go Show resolved Hide resolved

darccio changed the title ~~contrib/gomongodb.org/mongo-driver: add optional query tag truncation~~ feat(contrib/gomongodb.org/mongo-driver): add optional query tag truncation Jan 27, 2026

darccio added the apm:ecosystem contrib/* related feature requests or bugs label Jan 27, 2026

rarguelloF approved these changes Jan 27, 2026

View reviewed changes

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Jan 27, 2026

dd-devflow bot added mergequeue-status: error and removed mergequeue-status: in_progress labels Jan 27, 2026

dd-devflow bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: error mergequeue-status: queued labels Jan 27, 2026

gh-worker-dd-mergequeue-cf854d bot merged commit 437d647 into DataDog:main Jan 27, 2026
65 of 67 checks passed

dd-devflow bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Jan 27, 2026

feat(contrib/gomongodb.org/mongo-driver): add optional query tag truncation #3679

feat(contrib/gomongodb.org/mongo-driver): add optional query tag truncation #3679

Uh oh!

Conversation

colinking commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Reviewer's Checklist

Uh oh!

mtoffl01 commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

darccio Jul 15, 2025

Choose a reason for hiding this comment

Uh oh!

colinking Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

colinking Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

darccio commented Aug 26, 2025

Uh oh!

colinking commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

colinking commented Jan 22, 2026

Uh oh!

darccio commented Jan 26, 2026

Uh oh!

colinking commented Jan 26, 2026

Uh oh!

Uh oh!

darccio commented Jan 27, 2026

Uh oh!

chatgpt-codex-connector bot commented Jan 27, 2026

Uh oh!

darccio commented Jan 27, 2026

Uh oh!

darccio commented Jan 27, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

darccio commented Jan 27, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

colinking commented Jun 20, 2025 •

edited

Loading

mtoffl01 commented Jun 30, 2025 •

edited

Loading

colinking commented Dec 12, 2025 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Jan 27, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Jan 27, 2026 •

edited

Loading