Skip to content

Conversation

@yiyuan-he
Copy link
Contributor

@yiyuan-he yiyuan-he commented Feb 7, 2025

Issue #, if available:

Lambda Topology issue -- more context in PRs for Python and JavaScript:

Description of changes:

Test plan:

Set up two Lambda functions with Java runtimes and tested with custom Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are screenshots of the topology for various configurations.

v1 Topology (lambdaA & lambdaB instrumented)
Screenshot 2025-02-07 at 11 48 51 AM

v2 Topology (lambdaA & lambdaB instrumented)
Screenshot 2025-02-07 at 11 26 34 AM

We observe the following correct behaviors for topology above:

  • Service entity node for Invoke call to downstream lambda.
  • AWS Resource node for GetFunction call to downstream lambda.
  • AWS Resource node for ListBuckets call to downstream s3.

v1 Topology (lambdaB not instrumented)
Screenshot 2025-02-07 at 12 17 59 PM

v2 Topology (lambdaB not instrumented)
Screenshot 2025-02-07 at 12 17 59 PM

We observe the following correct behaviors for topology above:

  • Downstream lambda called with Invoke is correctly treated as RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda instrumentation patch behaves correctly.

v1 Invoke
Screenshot 2025-02-06 at 10 10 23 PM

v2 Invoke
Screenshot 2025-02-06 at 10 11 49 PM

v1 GetFunction
Screenshot 2025-02-06 at 10 08 53 PM

v2 GetFunction
Screenshot 2025-02-06 at 10 10 05 PM

We observe the following correct behaviors in the spans above:

  • For Invoke calls, we see aws.remote.service and aws.remote.environment correctly populated in the spans.
  • For non-Invoke calls (i.e. GetFunction), we see AWS Resource attributes such as aws.remote.resource.identifier and aws.cloudformation.primary.identifier correctly populated.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@yiyuan-he yiyuan-he requested a review from a team as a code owner February 7, 2025 02:22
@yiyuan-he yiyuan-he force-pushed the lambda-topology-fix branch from b52de67 to a8da0be Compare February 7, 2025 06:04
@yiyuan-he yiyuan-he changed the title Fix: Lambda Topology Issue [WIP] Fix: Lambda Topology Issue Feb 7, 2025
@yiyuan-he yiyuan-he merged commit cd3c2c5 into aws-observability:release/v1.33.x Feb 11, 2025
4 checks passed
yiyuan-he added a commit to yiyuan-he/aws-otel-java-instrumentation that referenced this pull request Mar 5, 2025
**Issue #, if available:**

Lambda Topology issue -- more context in PRs for Python and JavaScript:
-
aws-observability/aws-otel-python-instrumentation#319
-
aws-observability/aws-otel-js-instrumentation#149

**Description of changes:**

- Apply fix for the Lambda Topology issue. The logic mimics the fix in
our other ADOT SDKs.
- Adding back AWS Resource support for Lambda.
-
aws-observability#907
  - We previously removed support due to the Lambda Topology issue

**Test plan:**

Set up two Lambda functions with Java runtimes and tested with custom
Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are
screenshots of the topology for various configurations.

**v1 Topology (lambdaA & lambdaB instrumented)**
<img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM"
src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7"
/>

**v2 Topology (lambdaA & lambdaB instrumented)**
<img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM"
src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5"
/>

We observe the following correct behaviors for topology above:
- Service entity node for `Invoke` call to downstream lambda.
- AWS Resource node for `GetFunction` call to downstream lambda.
- AWS Resource node for `ListBuckets` call to downstream s3.

**v1 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

**v2 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

We observe the following correct behaviors for topology above:
- Downstream lambda called with `Invoke` is correctly treated as
RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda
instrumentation patch behaves correctly.

**v1 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM"
src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5"
/>

**v2 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM"
src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2"
/>

**v1 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM"
src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02"
/>

**v2 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM"
src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2"
/>

We observe the following correct behaviors in the spans above:
- For `Invoke` calls, we see `aws.remote.service` and
`aws.remote.environment` correctly populated in the spans.
- For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource
attributes such as `aws.remote.resource.identifier` and
`aws.cloudformation.primary.identifier` correctly populated.


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
@yiyuan-he yiyuan-he deleted the lambda-topology-fix branch March 5, 2025 18:05
yiyuan-he added a commit to yiyuan-he/aws-otel-java-instrumentation that referenced this pull request May 28, 2025
**Issue #, if available:**

Lambda Topology issue -- more context in PRs for Python and JavaScript:
-
aws-observability/aws-otel-python-instrumentation#319
-
aws-observability/aws-otel-js-instrumentation#149

**Description of changes:**

- Apply fix for the Lambda Topology issue. The logic mimics the fix in
our other ADOT SDKs.
- Adding back AWS Resource support for Lambda.
-
aws-observability#907
  - We previously removed support due to the Lambda Topology issue

**Test plan:**

Set up two Lambda functions with Java runtimes and tested with custom
Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are
screenshots of the topology for various configurations.

**v1 Topology (lambdaA & lambdaB instrumented)**
<img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM"
src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7"
/>

**v2 Topology (lambdaA & lambdaB instrumented)**
<img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM"
src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5"
/>

We observe the following correct behaviors for topology above:
- Service entity node for `Invoke` call to downstream lambda.
- AWS Resource node for `GetFunction` call to downstream lambda.
- AWS Resource node for `ListBuckets` call to downstream s3.

**v1 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

**v2 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

We observe the following correct behaviors for topology above:
- Downstream lambda called with `Invoke` is correctly treated as
RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda
instrumentation patch behaves correctly.

**v1 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM"
src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5"
/>

**v2 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM"
src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2"
/>

**v1 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM"
src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02"
/>

**v2 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM"
src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2"
/>

We observe the following correct behaviors in the spans above:
- For `Invoke` calls, we see `aws.remote.service` and
`aws.remote.environment` correctly populated in the spans.
- For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource
attributes such as `aws.remote.resource.identifier` and
`aws.cloudformation.primary.identifier` correctly populated.


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
yiyuan-he added a commit that referenced this pull request Jun 3, 2025
## What does this pull request do?
Cherry picks the Lambda Topology fix from our Lambda Layer v1 branch.

Original PR:
#1016

## Test strategy
Unit tests and followed e2e test strategy used in the [original
PR](#1016)
to sanity check correct behavior in service topology.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
lukeina2z added a commit that referenced this pull request Aug 7, 2025
… v2.11.x (#1114)

This change merges all private Lambda Java updates from the v1.33 branch
into the v2.11.x branch. I performed a 'git rebase v2.11' on the v1.33
branch, reviewed all changes, and completed the build and testing
process. The resulting Lambda layer generated trace data identical to
the version built directly from the v2.11.x branch (excluding this PR).

Here is the list of all migrated PRs:

Build layer during CI/CD workflows + some minor refactoring #989 

support java11 runtime for lambda #1001 

Unique artifact names for upload and merge for download #1014 

Bug fixes] Lambda - duplicate lambda spans + appsignals from unsampled
spans #1000

Fix: Lambda Topology Issue (#1016)
 
Fix: Lambda Topology Issue (#1016) #1085
 
feat: Support microservice span in Lambda Java environment. #1053

Test
Tested Java11, 17, and 21 Lambda functions. Manually tested PR-1000 and
PR-1053. Both work as expected in the v2.11 branch. MicroService
(SpringBoot) support works well. I verified attribute
Trace.lambda.multiple server can be found in the Lambda server span,
once we have Servlet instrumentation enabled with
OTEL_INSTRUMENTATION_SERVLET_ENABLED.

Note: The changes in the patch files are not included in this PR. They
should have been reviewed and incorporated as part of this migration:

Upgrade Java Lambda Layer to 2.x #1076 

Lambda with SpringBoot MicroService:
<img width="1367" alt="lambda"
src="https://github.com/user-attachments/assets/5cf5be29-4986-454c-b61b-773d6cde3848"
/>

Service Map and added microservice attribute 'Trace.lambda.multiple
server'.
<img width="1864" alt="traceMap"
src="https://github.com/user-attachments/assets/f7ff1771-61f0-4013-b571-90370a726aa9"
/>

AppSignals
<img width="1875" alt="appSignals"
src="https://github.com/user-attachments/assets/24f1b3a8-851c-4c97-bb50-087ee275b86d"
/>



By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants