generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 68
Fix: Lambda Topology Issue #1016
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
yiyuan-he
merged 2 commits into
aws-observability:release/v1.33.x
from
yiyuan-he:lambda-topology-fix
Feb 11, 2025
Merged
Fix: Lambda Topology Issue #1016
yiyuan-he
merged 2 commits into
aws-observability:release/v1.33.x
from
yiyuan-he:lambda-topology-fix
Feb 11, 2025
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
b52de67 to
a8da0be
Compare
50ae35b to
666fa99
Compare
wangzlei
approved these changes
Feb 11, 2025
yiyuan-he
added a commit
to yiyuan-he/aws-otel-java-instrumentation
that referenced
this pull request
Mar 5, 2025
**Issue #, if available:** Lambda Topology issue -- more context in PRs for Python and JavaScript: - aws-observability/aws-otel-python-instrumentation#319 - aws-observability/aws-otel-js-instrumentation#149 **Description of changes:** - Apply fix for the Lambda Topology issue. The logic mimics the fix in our other ADOT SDKs. - Adding back AWS Resource support for Lambda. - aws-observability#907 - We previously removed support due to the Lambda Topology issue **Test plan:** Set up two Lambda functions with Java runtimes and tested with custom Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are screenshots of the topology for various configurations. **v1 Topology (lambdaA & lambdaB instrumented)** <img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM" src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7" /> **v2 Topology (lambdaA & lambdaB instrumented)** <img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM" src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5" /> We observe the following correct behaviors for topology above: - Service entity node for `Invoke` call to downstream lambda. - AWS Resource node for `GetFunction` call to downstream lambda. - AWS Resource node for `ListBuckets` call to downstream s3. **v1 Topology (lambdaB not instrumented)** <img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM" src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff" /> **v2 Topology (lambdaB not instrumented)** <img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM" src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff" /> We observe the following correct behaviors for topology above: - Downstream lambda called with `Invoke` is correctly treated as RemoteService entity when not instrumented Additionally, I generated the spans locally to verify the lambda instrumentation patch behaves correctly. **v1 Invoke** <img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM" src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5" /> **v2 Invoke** <img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM" src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2" /> **v1 GetFunction** <img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM" src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02" /> **v2 GetFunction** <img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM" src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2" /> We observe the following correct behaviors in the spans above: - For `Invoke` calls, we see `aws.remote.service` and `aws.remote.environment` correctly populated in the spans. - For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource attributes such as `aws.remote.resource.identifier` and `aws.cloudformation.primary.identifier` correctly populated. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
yiyuan-he
added a commit
to yiyuan-he/aws-otel-java-instrumentation
that referenced
this pull request
May 28, 2025
**Issue #, if available:** Lambda Topology issue -- more context in PRs for Python and JavaScript: - aws-observability/aws-otel-python-instrumentation#319 - aws-observability/aws-otel-js-instrumentation#149 **Description of changes:** - Apply fix for the Lambda Topology issue. The logic mimics the fix in our other ADOT SDKs. - Adding back AWS Resource support for Lambda. - aws-observability#907 - We previously removed support due to the Lambda Topology issue **Test plan:** Set up two Lambda functions with Java runtimes and tested with custom Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are screenshots of the topology for various configurations. **v1 Topology (lambdaA & lambdaB instrumented)** <img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM" src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7" /> **v2 Topology (lambdaA & lambdaB instrumented)** <img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM" src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5" /> We observe the following correct behaviors for topology above: - Service entity node for `Invoke` call to downstream lambda. - AWS Resource node for `GetFunction` call to downstream lambda. - AWS Resource node for `ListBuckets` call to downstream s3. **v1 Topology (lambdaB not instrumented)** <img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM" src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff" /> **v2 Topology (lambdaB not instrumented)** <img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM" src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff" /> We observe the following correct behaviors for topology above: - Downstream lambda called with `Invoke` is correctly treated as RemoteService entity when not instrumented Additionally, I generated the spans locally to verify the lambda instrumentation patch behaves correctly. **v1 Invoke** <img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM" src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5" /> **v2 Invoke** <img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM" src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2" /> **v1 GetFunction** <img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM" src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02" /> **v2 GetFunction** <img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM" src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2" /> We observe the following correct behaviors in the spans above: - For `Invoke` calls, we see `aws.remote.service` and `aws.remote.environment` correctly populated in the spans. - For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource attributes such as `aws.remote.resource.identifier` and `aws.cloudformation.primary.identifier` correctly populated. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
yiyuan-he
added a commit
that referenced
this pull request
Jun 3, 2025
## What does this pull request do? Cherry picks the Lambda Topology fix from our Lambda Layer v1 branch. Original PR: #1016 ## Test strategy Unit tests and followed e2e test strategy used in the [original PR](#1016) to sanity check correct behavior in service topology. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
lukeina2z
added a commit
that referenced
this pull request
Aug 7, 2025
… v2.11.x (#1114) This change merges all private Lambda Java updates from the v1.33 branch into the v2.11.x branch. I performed a 'git rebase v2.11' on the v1.33 branch, reviewed all changes, and completed the build and testing process. The resulting Lambda layer generated trace data identical to the version built directly from the v2.11.x branch (excluding this PR). Here is the list of all migrated PRs: Build layer during CI/CD workflows + some minor refactoring #989 support java11 runtime for lambda #1001 Unique artifact names for upload and merge for download #1014 Bug fixes] Lambda - duplicate lambda spans + appsignals from unsampled spans #1000 Fix: Lambda Topology Issue (#1016) Fix: Lambda Topology Issue (#1016) #1085 feat: Support microservice span in Lambda Java environment. #1053 Test Tested Java11, 17, and 21 Lambda functions. Manually tested PR-1000 and PR-1053. Both work as expected in the v2.11 branch. MicroService (SpringBoot) support works well. I verified attribute Trace.lambda.multiple server can be found in the Lambda server span, once we have Servlet instrumentation enabled with OTEL_INSTRUMENTATION_SERVLET_ENABLED. Note: The changes in the patch files are not included in this PR. They should have been reviewed and incorporated as part of this migration: Upgrade Java Lambda Layer to 2.x #1076 Lambda with SpringBoot MicroService: <img width="1367" alt="lambda" src="https://github.com/user-attachments/assets/5cf5be29-4986-454c-b61b-773d6cde3848" /> Service Map and added microservice attribute 'Trace.lambda.multiple server'. <img width="1864" alt="traceMap" src="https://github.com/user-attachments/assets/f7ff1771-61f0-4013-b571-90370a726aa9" /> AppSignals <img width="1875" alt="appSignals" src="https://github.com/user-attachments/assets/24f1b3a8-851c-4c97-bb50-087ee275b86d" /> By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Lambda Topology issue -- more context in PRs for Python and JavaScript:
Description of changes:
Test plan:
Set up two Lambda functions with Java runtimes and tested with custom Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are screenshots of the topology for various configurations.
v1 Topology (lambdaA & lambdaB instrumented)

v2 Topology (lambdaA & lambdaB instrumented)

We observe the following correct behaviors for topology above:
Invokecall to downstream lambda.GetFunctioncall to downstream lambda.ListBucketscall to downstream s3.v1 Topology (lambdaB not instrumented)

v2 Topology (lambdaB not instrumented)

We observe the following correct behaviors for topology above:
Invokeis correctly treated as RemoteService entity when not instrumentedAdditionally, I generated the spans locally to verify the lambda instrumentation patch behaves correctly.
v1 Invoke

v2 Invoke

v1 GetFunction

v2 GetFunction

We observe the following correct behaviors in the spans above:
Invokecalls, we seeaws.remote.serviceandaws.remote.environmentcorrectly populated in the spans.Invokecalls (i.e.GetFunction), we see AWS Resource attributes such asaws.remote.resource.identifierandaws.cloudformation.primary.identifiercorrectly populated.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.