Skip to content

Conversation

@liustve
Copy link
Contributor

@liustve liustve commented Apr 10, 2025

Issue description:

  • Added input for language version runtime
  • Added a workflow to automatically performance test and gather the results for the unreleased JavaScript Lambda Layer.

The GitHub workflow does the following:

  1. Deploy and setup the JavaScript Lambda sample application instrumented with the Application Signals Lambda Layer on the AppSignals e2e testing aws account.
  2. Record the start time of the test and run the specified amount (default: 20) Lambda cold start iterations on the sample application
  3. Record the end time of the test
  4. Query CW Logs Insights for the aggregated performance testing metrics with the following query:
fields @timestamp, @message, @initDuration
| filter @message like "REPORT RequestId"
| stats
    count(@initDuration) as Sample_Count,
    avg(@initDuration) as Average_Init_Duration,
    min(@initDuration) as Min_Init_Duration,
    max(@initDuration) as Max_Init_Duration,
    pct(@initDuration, 50) as P50_Init_Duration,
    pct(@initDuration, 90) as P90_Init_Duration,
    pct(@initDuration, 99) as P99_Init_Duration'
  1. Parse and flatten the result from the query call and save it in a file.
  2. Clean and destroy the deployed lambda resources upon completion, cancellation, or failure.

Testing
Testing was done on a personal aws dev account with the exact IAM permissions as the e2e testing aws account.
Example of a successful job:

https://github.com/liustve/aws-application-signals-test-framework/actions/runs/14387543293

@liustve liustve requested review from wangzlei and yiyuan-he April 10, 2025 18:32
Copy link
Contributor

@yiyuan-he yiyuan-he left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Curious why did we decide to add language runtime version as input? What will our guidance to MCM executor be for which version to use (i.e. latest Lambda runtime)?

@liustve
Copy link
Contributor Author

liustve commented Apr 10, 2025

LGTM!

Curious why did we decide to add language runtime version as input? What will our guidance to MCM executor be for which version to use (i.e. latest Lambda runtime)?

Not sure if this workflow might be used in cases other than the MCM. I was thinking of making it more flexible for other usecases by allowing the runner to use any versions that they needed.

Open to getting rid of this change if it doesn't seem necessary.

Yes, will probably update MCM preflight steps to ask the executor to set the workflow run to the latest language version.

@liustve liustve merged commit 7c9f852 into aws-observability:main Apr 10, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants