Skip to content

Comments

Hybrid agent context management#1589

Merged
lrafeei merged 16 commits intodevelop-hybrid-core-tracingfrom
hybrid-agent-context-management
Dec 17, 2025
Merged

Hybrid agent context management#1589
lrafeei merged 16 commits intodevelop-hybrid-core-tracingfrom
hybrid-agent-context-management

Conversation

@lrafeei
Copy link
Contributor

@lrafeei lrafeei commented Nov 21, 2025

This PR contains a barebones implementation for Distributed Tracing/context propagation using OTel and NR APIs

@github-actions
Copy link

github-actions bot commented Nov 21, 2025

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 7 0 0 0.97s
✅ MARKDOWN markdownlint 7 0 0 0 1.37s
✅ PYTHON ruff 971 0 0 0 1.02s
✅ PYTHON ruff-format 971 0 0 0 0.36s
✅ YAML prettier 15 0 0 0 1.51s
✅ YAML v8r 15 0 0 5.72s
✅ YAML yamllint 15 0 0 0.68s

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security

@mergify mergify bot added the tests-failing Tests failing in CI. label Nov 21, 2025
@codecov-commenter
Copy link

codecov-commenter commented Nov 21, 2025

Codecov Report

❌ Patch coverage is 59.72222% with 29 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop-hybrid-core-tracing@378c2c6). Learn more about missing BASE report.

Files with missing lines Patch % Lines
newrelic/hooks/hybridagent_opentelemetry.py 48.64% 17 Missing and 2 partials ⚠️
newrelic/api/opentelemetry.py 69.69% 6 Missing and 4 partials ⚠️
Additional details and impacted files
@@                      Coverage Diff                       @@
##             develop-hybrid-core-tracing    #1589   +/-   ##
==============================================================
  Coverage                               ?   79.88%           
==============================================================
  Files                                  ?      213           
  Lines                                  ?    25060           
  Branches                               ?     3980           
==============================================================
  Hits                                   ?    20020           
  Misses                                 ?     3617           
  Partials                               ?     1423           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lrafeei lrafeei force-pushed the hybrid-agent-context-management branch from 7b8ade6 to d61ac06 Compare December 2, 2025 20:49
@lrafeei lrafeei marked this pull request as ready for review December 2, 2025 21:11
@lrafeei lrafeei requested a review from a team as a code owner December 2, 2025 21:11
@mergify mergify bot added the merge-conflicts Merge conflicts detected. label Dec 8, 2025
@lrafeei lrafeei force-pushed the hybrid-agent-context-management branch from e8ab06d to 63d7803 Compare December 10, 2025 04:48
@mergify mergify bot removed the merge-conflicts Merge conflicts detected. label Dec 10, 2025
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just gonna post this anyway if there's anything that is still relevant after the changes we discussed.

@lrafeei lrafeei mentioned this pull request Dec 12, 2025
@lrafeei lrafeei requested a review from hmstepanek December 12, 2025 20:54
@lrafeei lrafeei requested a review from hmstepanek December 15, 2025 18:17
Copy link
Contributor

@hmstepanek hmstepanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of time but posting a partial review-I'll look at the rest tomorrow.

# precedence over whatever sampling algorithm is
# enabled.

if self.otel_parent:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand it from Chris - we need to always return sampled=True in order to fake OTEL into thinking the span is being sampled and collect data on it. Then we use our internal sampling decision later to decide whether we are actually going to sample it (aka add it to our reservoir).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's actually a fair point. Either way, it's going to get there...

lrafeei and others added 2 commits December 16, 2025 23:17
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
@lrafeei lrafeei requested a review from hmstepanek December 17, 2025 07:28
# This is an HTTP request (WSGI, ASGI, or otherwise)
nr_environ = context_carrier.copy()
attributes["nr.http.headers"] = nr_environ
attributes["nr.http.headers"] = context_carrier
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering why we had/got rid of the copy here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized I was not actually manipulating the header data anymore, so I could just pass in the contents of the carrier directly as an attribute

@lrafeei lrafeei requested a review from hmstepanek December 17, 2025 23:42
@lrafeei lrafeei merged commit 5577ffd into develop-hybrid-core-tracing Dec 17, 2025
49 of 58 checks passed
@lrafeei lrafeei deleted the hybrid-agent-context-management branch December 17, 2025 23:57
@mergify mergify bot removed the tests-failing Tests failing in CI. label Dec 17, 2025
hmstepanek added a commit that referenced this pull request Feb 9, 2026
* Context propagation/DT enabling

* Add accept/extract DT tests

* Propagation extract/accept tests & remote tracestate parsing

* Remove extra spaces

* Explanation for update flags

* Modify carrier and sampling logic

* Change test name

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Apply reviewer suggestions

* Default parent_span_trace_id to None

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Reviewer suggestions & cleanup

* [MegaLinter] Apply linters fixes

* Apply review suggestions (for real this time)

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
TimPansino added a commit that referenced this pull request Feb 20, 2026
* Reorder inheritance on span nodes (#1539)

* Reorder inheritance on span nodes

Core tracing will be adding a bunch of logic to the span_event method which is called
via super on inheriting classes before those inheriting classes add additional
attributes. In order for the core tracing logic to work correctly, this inheritance
call order has to be reversed; the inheriting classes must first add the attributes
and THEN call super.

* Remove *args, **kwargs

* [MegaLinter] Apply linters fixes

---------

Co-authored-by: hmstepanek <30059933+hmstepanek@users.noreply.github.com>

* Fix removal of process attr call (#1550)

* Add `adaptive_sampling_target` setting & fix logic (#1549)

* Fix removal of process attr call

* Add adaptive_sampling_target setting & fix logic

* Add a new setting: `distributed_tracing.sampler.adaptive_sampling_target` that can be
used to configure the sampling target.
* Previously the logic for remote parent sampled was slightly incorrect. The sampling
value from the tracestate header was used to determine remote parent sampled, instead of
from the parent header-this has been fixed. Additionally, the sampling value was only
grabbed from the trace state and newrelic headers if there was a priority-this has been
fixed too.
* Log statements have been added to assist with sampling debug.

---------

Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

* Add support for partial granularity type (#1532)

* Fix error when shutdown_agent called from harvest thread (#1552)

* fix(aiomysql): avoid wrapping pooled connections multiple times (#1553)

* fix(aiomysql): avoid wrapping pooled connections multiple times

* Move and rewrite regression test

* Tweak implementation of fix

---------

Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Fix structlog tests (#1556)

* Bump the github_actions group with 4 updates (#1555)

Bumps the github_actions group with 4 updates: [actions/upload-artifact](https://github.com/actions/upload-artifact), [actions/download-artifact](https://github.com/actions/download-artifact), [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) and [github/codeql-action](https://github.com/github/codeql-action).


Updates `actions/upload-artifact` from 4.6.2 to 5.0.0
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@ea165f8...330a01c)

Updates `actions/download-artifact` from 5.0.0 to 6.0.0
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@634f93c...018cc2c)

Updates `astral-sh/setup-uv` from 7.1.1 to 7.1.2
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@2ddd2b9...8585678)

Updates `github/codeql-action` from 4.30.9 to 4.31.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@16140ae...4e94bd1)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github_actions
- dependency-name: actions/download-artifact
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github_actions
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
- dependency-name: github/codeql-action
  dependency-version: 4.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* Add instrumentation for new kinesis method (#1557)

* Add free-threaded Python to CI (#1562)

* Bump github/codeql-action in the github_actions group (#1566)

Bumps the github_actions group with 1 update: [github/codeql-action](https://github.com/github/codeql-action).


Updates `github/codeql-action` from 4.31.0 to 4.31.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@4e94bd1...0499de3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.31.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Region aware/ Claude 3+ bedrock support (#1561)

* Modify extractor logic.

* Add support for Claude Sonnet 3+ and region aware models.

* Update claude content extraction logic.

* Add support for Claude Sonnet 3+ and region aware models.

* Update claude content extraction logic.

* Add testing for aiobotocore.

* Restore newline.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix notice_error logic for non-iterable exceptions. (#1564)

* Revert "Fix notice_error logic for non-iterable exceptions. (#1564)" (#1568)

This reverts commit b9d9d3b.

* Add additional trace points for AWS Kinesis (#1569)

* Enable environment variables for attribute filters (#1558)

* Enable env vars for attribute filters

* [MegaLinter] Apply linters fixes

* Trigger tests

* Change attribute filters to space delimited

* Fix test assertion

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Update version of cibuildwheel to latest (#1570)

* Force uv to use non-emulated Python on windows arm64 (#1567)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add support 4 partial granularity tracing

* Move config consolidation into global settings

* [MegaLinter] Apply linters fixes

* Move inifinte tracing override to server side config

* [MegaLinter] Apply linters fixes

* Fix failing event loop tests

* Fixup: linter

* [MegaLinter] Apply linters fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: canonrock16 <35710450+canonrock16@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
Co-authored-by: hmstepanek <30059933+hmstepanek@users.noreply.github.com>

* Add metrics for tracking partial granularity (#1560)

* Add SamplerProxy (#1559)

* Add adaptive sampling target config options & instances (#1577)

* Fix error when shutdown_agent called from harvest thread (#1552)

* fix(aiomysql): avoid wrapping pooled connections multiple times (#1553)

* fix(aiomysql): avoid wrapping pooled connections multiple times

* Move and rewrite regression test

* Tweak implementation of fix

---------

Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Fix structlog tests (#1556)

* Bump the github_actions group with 4 updates (#1555)

Bumps the github_actions group with 4 updates: [actions/upload-artifact](https://github.com/actions/upload-artifact), [actions/download-artifact](https://github.com/actions/download-artifact), [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) and [github/codeql-action](https://github.com/github/codeql-action).

Updates `actions/upload-artifact` from 4.6.2 to 5.0.0
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@ea165f8...330a01c)

Updates `actions/download-artifact` from 5.0.0 to 6.0.0
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@634f93c...018cc2c)

Updates `astral-sh/setup-uv` from 7.1.1 to 7.1.2
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@2ddd2b9...8585678)

Updates `github/codeql-action` from 4.30.9 to 4.31.0
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@16140ae...4e94bd1)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github_actions
- dependency-name: actions/download-artifact
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github_actions
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
- dependency-name: github/codeql-action
  dependency-version: 4.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* Add instrumentation for new kinesis method (#1557)

* Add free-threaded Python to CI (#1562)

* Bump github/codeql-action in the github_actions group (#1566)

Bumps the github_actions group with 1 update: [github/codeql-action](https://github.com/github/codeql-action).

Updates `github/codeql-action` from 4.31.0 to 4.31.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@4e94bd1...0499de3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.31.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Region aware/ Claude 3+ bedrock support (#1561)

* Modify extractor logic.

* Add support for Claude Sonnet 3+ and region aware models.

* Update claude content extraction logic.

* Add support for Claude Sonnet 3+ and region aware models.

* Update claude content extraction logic.

* Add testing for aiobotocore.

* Restore newline.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix notice_error logic for non-iterable exceptions. (#1564)

* Revert "Fix notice_error logic for non-iterable exceptions. (#1564)" (#1568)

This reverts commit b9d9d3b.

* Add additional trace points for AWS Kinesis (#1569)

* Enable environment variables for attribute filters (#1558)

* Enable env vars for attribute filters

* [MegaLinter] Apply linters fixes

* Trigger tests

* Change attribute filters to space delimited

* Fix test assertion

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Update version of cibuildwheel to latest (#1570)

* Force uv to use non-emulated Python on windows arm64 (#1567)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Skip hypercorn tests for v0.18 (#1579)

* Skip hypercorn tests for v0.18

* Remove tornadomaster for 3.14

* Add support for *.adaptive.sampling_target

* Add adaptive sampler instances to SamplerProxy

* Fix instability in CI caused by health check tests (#1584)

* Bump the github_actions group across 1 directory with 5 updates (#1582)

Bumps the github_actions group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [actions/checkout](https://github.com/actions/checkout) | `5.0.0` | `5.0.1` |
| [docker/metadata-action](https://github.com/docker/metadata-action) | `5.8.0` | `5.9.0` |
| [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) | `3.6.0` | `3.7.0` |
| [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `7.1.2` | `7.1.3` |
| [github/codeql-action](https://github.com/github/codeql-action) | `4.31.2` | `4.31.3` |

Updates `actions/checkout` from 5.0.0 to 5.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@08c6903...93cb6ef)

Updates `docker/metadata-action` from 5.8.0 to 5.9.0
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](docker/metadata-action@c1e5197...318604b)

Updates `docker/setup-qemu-action` from 3.6.0 to 3.7.0
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](docker/setup-qemu-action@2910929...c7c5346)

Updates `astral-sh/setup-uv` from 7.1.2 to 7.1.3
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@8585678...5a7eac6)

Updates `github/codeql-action` from 4.31.2 to 4.31.3
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@0499de3...014f16e)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 5.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
- dependency-name: docker/metadata-action
  dependency-version: 5.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: docker/setup-qemu-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
- dependency-name: github/codeql-action
  dependency-version: 4.31.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Asyncio loop_factory fix (#1576)

* Runner instrumentation in asyncio

* Clean up asyncio instrumentation

* Add asyncio tests for loop_factory

* Modify uvicorn test for loop_factory

* Fix linter errors

* [MegaLinter] Apply linters fixes

* Apply suggestions from code review

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Fix issue in ASGI header consumption (#1578)

* Correct code for Sanic instrumentation

* Correct handling of headers in ASGIWebTransaction

* Correct handling of headers in ASGIBrowserMiddleware

* Add regression test for ASGI headers issues

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: canonrock16 <35710450+canonrock16@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

* Add distributed_tracing.sampler.*root.* config support (#1586)

* Add root config

* [MegaLinter] Apply linters fixes

---------

Co-authored-by: hmstepanek <30059933+hmstepanek@users.noreply.github.com>

* Initial Hybrid Agent Trace implementation (#1587)

* Initial otel trace/span implementation

* Add some hybrid cross agent tests

* Tweak some formatting and merge issues

* [MegaLinter] Apply linters fixes

* Fix application initialization bug

* [MegaLinter] Apply linters fixes

* Reviewer suggestions, part 1

* [MegaLinter] Apply linters fixes

* Fix hasattr syntax

* More reviewer suggestions/syntax fixes

* [MegaLinter] Apply linters fixes

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Reviewer suggestions, part 2

* Add fixture & enable/disable testing

* Log potential error instead of raising error

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Update newrelic/api/opentelemetry.py

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Add TraceIdRatioBasedSampler, config, & tests

* Fixup: put args before kwargs

* LangChain: Fix message timestamps, add default role assignment, and Bedrock support (#1580)

* Record the request message as the time the request started for LangChain.

* Tracking the original timestamp of the request for input messages that are
recorded as LlmChatCompletionMessage event types.

* First pass at preserving LlmChatCompletionMessage timestamp for the request
with Bedrock methods.

* the `kwargs` was being mapped directly to the OpenAI client
and having timestamp in there caused a problem. As a quick test,
only add the request timestamp after the wrapped function has been
invoked.

* Moved the request timestamp to its own variable instead of part of kwargs.

* OpenAI async request messages were not being assigned the correct timestamp.

* Trying to improve the passing of the request timestamp through for Bedrock.

* Passing too many parameters.

* Set a default role on input/output messages within LangChain.

* [MegaLinter] Apply linters fixes

* Fix request_timestamp for LlmChatCompletionSummary table

* Fix request_timestamp for LlmChatCompletionSummary table

* [MegaLinter] Apply linters fixes

* Bedrock Converse Streaming Support (#1565)

* Add more formatting to custom event validatators

* Add streamed responses to converse mock server

* Add streaming fixtures for testing for converse

* Rename other bedrock test files

* Add tests for converse streaming

* Instrument converse streaming

* Move GeneratorProxy adjacent functions to mixin

* Fix checking of supported models

* Reorganize converse error tests

* Port new converse botocore tests to aiobotocore

* Instrument response streaming in aiobotocore converse

* Fix suggestions from code review

* Port in converse changes from strands PR

* Delete commented code

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Bedrock Converse Streaming Support (#1565)

* Add more formatting to custom event validatators

* Add streamed responses to converse mock server

* Add streaming fixtures for testing for converse

* Rename other bedrock test files

* Add tests for converse streaming

* Instrument converse streaming

* Move GeneratorProxy adjacent functions to mixin

* Fix checking of supported models

* Reorganize converse error tests

* Port new converse botocore tests to aiobotocore

* Instrument response streaming in aiobotocore converse

* Fix suggestions from code review

* Port in converse changes from strands PR

* Delete commented code

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* [MegaLinter] Apply linters fixes

* request_timestamp is now passed across different method

* Fixed gemini model kwargs issue

* [MegaLinter] Apply linters fixes

* Update tests to validate presence of timestamp/ role and fix bugs in instrumentation.

* Update aiobotocore instrumentation to receive request timestamp.

---------

Co-authored-by: Josh Bonczkowski <jbonczkowski@newrelic.com>
Co-authored-by: sgoel-nr <236423107+sgoel-nr@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>

* Update newrelic/core/config.py

Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

* Fixup: unpacking args after kwargs

* [MegaLinter] Apply linters fixes

* Fixup: duplicated test names

* Prioritize full over partial granularity

* priority +=2 when full granularity and sampled=true
* priority +=1 when partial granularity and sampled=true
* always_on sampler for full granularity sets priority=3 instead of 2

* Increment priority when only sampled is sent in headers

* [MegaLinter] Apply linters fixes

* Fixup: test name

* Merge main into develop-hybrid-core-tracing branch (#1605)

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add Strands tools and agents instrumentation. (#1563)

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* [MegaLinter] Apply linters fixes

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* Handle additional args in mock model.

* Add test to force exception and exercise _handle_tool_streaming_completion_error.

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* Handle additional args in mock model.

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* [MegaLinter] Apply linters fixes

* Add test to force exception and exercise _handle_tool_streaming_completion_error.

* Implement strands context passing instrumentation.

* Address review feedback.

* [MegaLinter] Apply linters fixes

* Remove test_simple.py file.

---------

Co-authored-by: Tim Pansino <tpansino@newrelic.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Strands MultiAgent Instrumentation (#1590)

* Rename strands instrument functions

* Add instrumentation for strands multiagent

* Reorganize strands tests

* Strands multiagent tests

* Remove timestamp from test expected events.

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add Strands tools and agents instrumentation. (#1563)

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* [MegaLinter] Apply linters fixes

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* Handle additional args in mock model.

* Add test to force exception and exercise _handle_tool_streaming_completion_error.

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Add tests file.

* Cleanup instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* Handle additional args in mock model.

* Strands Mock Model (#1551)

* Add strands to tox.ini

* Add mock models for strands testing

* Add simple test file to validate strands mocking

* Add baseline instrumentation.

* Add tool and agent instrumentation.

* Cleanup.

Co-authored-by: Tim Pansino <tpansino@newrelic.com>

* [MegaLinter] Apply linters fixes

* Add test to force exception and exercise _handle_tool_streaming_completion_error.

* Implement strands context passing instrumentation.

* Address review feedback.

* [MegaLinter] Apply linters fixes

* Remove test_simple.py file.

---------

Co-authored-by: Tim Pansino <tpansino@newrelic.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>

* Strands MultiAgent Instrumentation (#1590)

* Rename strands instrument functions

* Add instrumentation for strands multiagent

* Reorganize strands tests

* Strands multiagent tests

* Remove timestamp from test expected events.

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>

* Fixed tool type bug  for strands

* Pin langchain & langchain_core (#1604)

* Add safeguarding to converse attr extraction. (#1603)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Pin scikit-learn

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: Tim Pansino <tpansino@newrelic.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shubham Goel <sgoel@newrelic.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Hybrid agent context management (#1589)

* Context propagation/DT enabling

* Add accept/extract DT tests

* Propagation extract/accept tests & remote tracestate parsing

* Remove extra spaces

* Explanation for update flags

* Modify carrier and sampling logic

* Change test name

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Apply reviewer suggestions

* Default parent_span_trace_id to None

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Reviewer suggestions & cleanup

* [MegaLinter] Apply linters fixes

* Apply review suggestions (for real this time)

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Pretty-ify logic for improved readability

* Add tracking metric, nr.pg & error attrs

* Only record metric if pg transaction are sent

* Check for nr.pg attr in partial gran tests

* Keep error attributes & test

* Add test for metrics & fix bugs

* [MegaLinter] Apply linters fixes

* [MegaLinter] Apply linters fixes

* Add set_status() API

* Megalinter fixes

* [MegaLinter] Apply linters fixes

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Add tests for description behavior

* [MegaLinter] Apply linters fixes

* Refactor span_event into separate functions

* Cache spans before harvesting in compact mode

* Ignore RUF012 & Apply linters fixes

* Fix failing test

* Fixup: pr header value test

* Hybrid agent WSGI traces (#1607)

* WSGI support with Flask tests

* Supportability metric & setting name change

* Megalinter and merge fixes

* [MegaLinter] Apply linters fixes

* Tweaks

* [MegaLinter] Apply linters fixes

* Reviewer suggestions and cleanup

* [MegaLinter] Apply linters fixes

* Add timer wait for application start up

* [MegaLinter] Apply linters fixes

* Change function name

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* ASGI (server) support

* DynamoDB (Client) support

* More DB framework tests

* Requests (Client) support

* Tweak cross agent tests

* Refactor database attr logic

* Reviewer suggestions and syntax

* Oops, all syntax

* Remove full_granularity.<section> config options

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Reviewer suggestions and linter fixes

* Fix supportability metric merge resolution

* Change span logic to use sentinels

* Kafka-python support (PRODUCER/CONSUMER)

* Add sampler supportability metrics

* Remove conversion to CSV

* [MegaLinter] Apply linters fixes

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Reviewer suggestions

* Use json.loads instead of eval

* [MegaLinter] Apply linters fixes

* Add support and tests for RabbitMQ (#1622)

* Add support and tests for RabbitMQ

* Remove py3.8 from pika tests

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Revert library names to lowercase

---------

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Add cross agent tests for core tracing

* Fix lint issues

  Error: tests/cross_agent/test_distributed_tracing_trace_context.py:123:38: C418 Unnecessary dict comprehension passed to `dict()` (remove the outer call to `dict()`)
  Error: tests/cross_agent/test_distributed_tracing_trace_context.py:124:37: C418 Unnecessary dict comprehension passed to `dict()` (remove the outer call to `dict()`)
  Error: tests/cross_agent/test_harvest_sampling_rates.py:64:27: FLY002 Consider `f"{setting}.{key}"` instead of string join
  Error: tests/cross_agent/test_harvest_sampling_rates.py:160:13: B007 Loop control variable `n` not used within loop body
  Error: tests/cross_agent/test_harvest_sampling_rates.py:163:13: B007 Loop control variable `n` not used within loop body
  Error: tests/cross_agent/test_harvest_sampling_rates.py:172:13: B007 Loop control variable `n` not used within loop body
  Error: tests/cross_agent/test_harvest_sampling_rates.py:181:13: B007 Loop control variable `n` not used within loop body
  Error: tests/cross_agent/test_harvest_sampling_rates.py:190:13: B007 Loop control variable `n` not used within loop body
  Error: tests/cross_agent/test_sampler_configuration.py:85:27: FLY002 Consider `f"{setting}.{key}"` instead of string join
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:69:13: B007 Loop control variable `method_frame` not used within loop body
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:88:13: B007 Loop control variable `method_frame` not used within loop body
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:222:9: RUF059 Unpacked variable `method` is never used
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:260:17: B007 Loop control variable `method_frame` not used within loop body
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:286:13: RUF059 Unpacked variable `method_frame` is never used
  Error: tests/hybridagent_pika/test_pika_blocking_connection_consume_generator.py:323:9: RUF059 Unpacked variable `method` is never used

* Update partial tracing attributes to be intrinsics (#1635)

* move attrs from agent to intrinsics

* Fix linter errors

* [MegaLinter] Apply linters fixes

* Trigger tests

* Hybrid Agent SpanLink and SpanEvent Events Implementation (#1628)

* SpanLink & SpanEvent implementation

* Supportability metrics for dropped events

* Reviewer suggestions

* [MegaLinter] Apply linters fixes

* Fix LazySpan logic/Add spanlink/spanevent tests for different trace types

* [MegaLinter] Apply linters fixes

* Remove duplicated files & fix invalid ratio

* Remove special handling for ratio=0

* Load instrumentors in configuration

* traces.enabled setting, removing instrumentors in tests, & linter fixes

* Set TracerProvider to be singleton

* Tracer/library attributes and test

* Test response attributes for web transactions

* Test framework that only OTel supports

* Test native (graphql) instrumentation

* Add attribute parsing for native elasticsearch

* Add tests for disabled (for OTel) framework

* Obfuscate DBs and miscellaneous

* Application activation retry logic & reviewer suggestions

* Apply suggestions from code review

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* [MegaLinter] Apply linters fixes

* Add safety guard

* Fixup bug in ct (#1634)

* Handle attrs set to None

* [MegaLinter] Apply linters fixes

---------

Co-authored-by: hmstepanek <30059933+hmstepanek@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

* Fixup: megalinter

* [MegaLinter] Apply linters fixes

* Trigger tests

* Bump the github_actions group across 1 directory with 7 updates (#1657)

Bumps the github_actions group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [actions/checkout](https://github.com/actions/checkout) | `6.0.1` | `6.0.2` |
| [actions/setup-python](https://github.com/actions/setup-python) | `6.1.0` | `6.2.0` |
| [docker/login-action](https://github.com/docker/login-action) | `3.6.0` | `3.7.0` |
| [docker/build-push-action](https://github.com/docker/build-push-action) | `6.18.0` | `6.19.1` |
| [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) | `3.1.0` | `3.2.0` |
| [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `7.2.0` | `7.3.0` |
| [github/codeql-action](https://github.com/github/codeql-action) | `4.31.10` | `4.32.2` |



Updates `actions/checkout` from 6.0.1 to 6.0.2
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@8e8c483...de0fac2)

Updates `actions/setup-python` from 6.1.0 to 6.2.0
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@83679a8...a309ff8)

Updates `docker/login-action` from 3.6.0 to 3.7.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@5e57cd1...c94ce9f)

Updates `docker/build-push-action` from 6.18.0 to 6.19.1
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@2634353...601a80b)

Updates `actions/attest-build-provenance` from 3.1.0 to 3.2.0
- [Release notes](https://github.com/actions/attest-build-provenance/releases)
- [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md)
- [Commits](actions/attest-build-provenance@00014ed...96278af)

Updates `astral-sh/setup-uv` from 7.2.0 to 7.3.0
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](astral-sh/setup-uv@61cb8a9...eac588a)

Updates `github/codeql-action` from 4.31.10 to 4.32.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@cdefb33...45cbd0c)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github_actions
- dependency-name: actions/setup-python
  dependency-version: 6.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: docker/login-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: docker/build-push-action
  dependency-version: 6.19.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: actions/attest-build-provenance
  dependency-version: 3.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
- dependency-name: github/codeql-action
  dependency-version: 4.32.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github_actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix timestamp merge conflicts in AI

* Reviewer suggestions, part 1

* Revert setting transport type when empty payload

* Add span links and span events as params

* Call tracer provider once for instrumentors

* Add try/except logic for opentelemetry imports

* Change application activation logic & otel naming

* [MegaLinter] Apply linters fixes

* Remove duplicate older version of trace_context tests

* Fix sampling rates cross agent test fails

* Refactor pytest parametrize with Claude

* Apply suggestions from code review

Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* Reviewer suggestions, part 3

* Apply suggestions from code review

Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* Reviewer suggestions, part 3

* Add test case for unknown partial gran type

* [MegaLinter] Apply linters fixes

* Trigger tests

* Add newrelic.core.samplers to packages

* Explicitly check description

* More reviewer suggestions

* [MegaLinter] Apply linters fixes

* Trigger tests

* [MegaLinter] Apply linters fixes

* Update newrelic/hooks/hybridagent_opentelemetry.py

Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* [MegaLinter] Apply linters fixes

* Trigger tests

* Revert "Refactor pytest parametrize with Claude"

This reverts commit 9b5095d.

* Revert "Update newrelic/hooks/hybridagent_opentelemetry.py"

This reverts commit 3f8d15c.

* Fix comment formatting

* Import current_transaction

* Update newrelic/api/transaction.py

* Fix comment

* Add TRACE level logging & use to log sampling

* Rename (kept, instr..) stats to be more explicit

* [MegaLinter] Apply linters fixes

* Add comment

* Test benchmark

* Revert "Test benchmark"

This reverts commit ba23cfa.

* Call .log instead of ._log

* Comment out trace level logging

* Revert raise inside fixture if 0 events

* Add logging for commit shas

* Change sha on benchmark runner

* [MegaLinter] Apply linters fixes

* Add comment

* Revert raise inside fixture if 0 events

* Comment out trace level logging

* Tweak benchmark job

* Make non-opentelemetry spans more performant

* Uplevel partial gran checks outside of nodes

This is done to improve performance when there are many spans.

* Fix merge conflict

* Simplify sampling logic code to improve performance

* Simplify sampling logic code to improve performance

* [MegaLinter] Apply linters fixes

* Add env var check to tracer provider logic

* [MegaLinter] Apply linters fixes

* Fixup: delete commented out lines

* Remove added version in header val before compare

* [MegaLinter] Apply linters fixes

* Use _environ_as_bool

* [MegaLinter] Apply linters fixes

* Fix infinite tracing span generation

* Fixup: remove span link from function node

* Revert "Fixup: remove span link from function node"

This reverts commit 8341b66.

* Fix harvest loop tests

* Ensure ImportHookFinder is singleton

* Add extra overhead to max compressed span tests

* Increase number of linux runners

* Do not run harvest tests in CI

* [MegaLinter] Apply linters fixes

* Bump tests

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: hmstepanek <30059933+hmstepanek@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: canonrock16 <35710450+canonrock16@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: sgoel-nr <sgoel@newrelic.com>
Co-authored-by: Josh Bonczkowski <jbonczkowski@newrelic.com>
Co-authored-by: sgoel-nr <236423107+sgoel-nr@users.noreply.github.com>
Co-authored-by: Tim Pansino <tpansino@newrelic.com>
Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants