Skip to content

Conversation

nina9753
Copy link

@nina9753 nina9753 commented Aug 11, 2025

What does this PR do?

Extends the official PubSub plugin capabilities and optimizes Datadog tracing instrumentation for Google Cloud Pub/Sub push messages and Cloud Events across all Node.js web frameworks.

  • Automatic Detection: No configuration needed
  • Large Payloads: Supports up to 10MB efficiently
  • All Frameworks: Works with Express, Fastify, Hapi, Koa, etc.
  • Distributed Tracing: Maintains trace context across services
  • Production Ready: Comprehensive error handling

GCP sends push → HTTP server receives POST request
Plugin detects → "This is a Pub/Sub push!" (via User-Agent: APIs-Google)
Plugin creates → HTTP span as child of PubSub span
Express inherits → Express spans inherit via scope activation
Your app processes → Business logic with automatic tracing

This makes the spans have this hierarchy :
http.request (manually created as child)
└── express.request (inherits via scope activation)
└── express.middleware (inherits via scope activation)
└── your-business-logic (main.js)

Follow-up PR will be opened to create the synthetic span in the flame graph

Motivation

An inferred span for the HTTP push post to the Cloud Run service from a pub/sub topic
Example full Push Direct Span of a cloud run service triggering another service using a direct push subscription
Example full Eventarc pubsub trigger span of a cloud run service triggering another service using an Eventarc cloud event trigger

Plugin Checklist

Additional Notes

The changes had to be made in server.js because in Push Subscription, the Cloud Run Service receives HTTP Requests.

  • Google Cloud Pub/Sub acts as the HTTP client
  • The cloud run service acts as the HTTP server
  • GCP topic pushes TO the cloud run service via HTTP POST requests
  • Then the cloud run service receives the pushed messages

Additional information can be found in this doc

image

Copy link

github-actions bot commented Aug 11, 2025

Overall package size

Self size: 11.92 MB
Deduped: 111.53 MB
No deduping: 111.87 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.1.0 | 20.37 MB | 20.37 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.1 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.4 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | source-map | 0.7.6 | 185.63 kB | 185.63 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Copy link

codecov bot commented Aug 11, 2025

Codecov Report

❌ Patch coverage is 28.49162% with 128 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.25%. Comparing base (0502c95) to head (1e9b42b).

Files with missing lines Patch % Lines
...-google-cloud-pubsub/src/pubsub-transit-handler.js 14.39% 113 Missing ⚠️
...ckages/datadog-instrumentations/src/body-parser.js 14.28% 12 Missing ⚠️
...ckages/datadog-instrumentations/src/http/server.js 84.21% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6260      +/-   ##
==========================================
- Coverage   84.06%   83.25%   -0.82%     
==========================================
  Files         382      478      +96     
  Lines       17269    20177    +2908     
==========================================
+ Hits        14518    16798    +2280     
- Misses       2751     3379     +628     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nina9753 nina9753 marked this pull request as draft August 11, 2025 17:43
@pr-commenter
Copy link

pr-commenter bot commented Aug 11, 2025

Benchmarks

Benchmark execution time: 2025-08-20 21:22:34

Comparing candidate commit 1e9b42b in PR branch nina.rei/SVLS-7168/gcp-push-pubsub-plugin with baseline commit 0502c95 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1268 metrics, 55 unstable metrics.

Copy link
Collaborator

@BridgeAR BridgeAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite a big PR, if you like, we could set up a sync call to go through the changes to review the PR :)

if (abortController.signal.aborted) return
}
return next()
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand this change.
I don't think we should ever skip that functionality, since it would also interfere with user's code, if I am not mistaken.

@nina9753 nina9753 marked this pull request as draft August 27, 2025 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants