test: publish to gamma and run smoke test on merge #725
Conversation
|
|
||
| on: | ||
| push: | ||
| branches: [main, release/*.*.*] |
There was a problem hiding this comment.
question: do you want this on merges to release branch as well? I understand merging to main.
in our release workflow, we only merge stuff to release branch when we are getting the release ready. We typically cut a branch from main. After we do that, we run the release workflow which would end up running the smoke tests anyhow.
Are we duplicating test runs and making the release flow longer?
There was a problem hiding this comment.
These only run in the background when we merge to main, and give us checkmarks for each commit. We are not blocking release activity or pull requests, so this doesn't restrict our work in any way.
I think it's worthwhile because it is still possible to make rebasing errors on release branches, which is why we also run unit and integration tests against release branches
| INSTALL_METHOD: 'CDN' | ||
| run: | | ||
| curl -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL" | jq -r '.value' > $AWS_WEB_IDENTITY_TOKEN_FILE | ||
| npm run smoke:headless |
There was a problem hiding this comment.
nit/suggestion: Given there's some overlap with the cd.yaml workflow, we could explore if we could simplify the workflows to reduce redundancy across the workflows and only have to maintain the workflow for gamma in one place.
There was a problem hiding this comment.
I would like the release workflow cd.yml to re-use smoke.yml, but I need to do some more testing for that change by doing a test release to aws-rum-web-experimental
Yes from description, I would like to do this but it's risky to edit cd.yaml without doing tons of manual testing
Summary
We are only running smoke tests during release, which is too late. Instead, we should be running this test more frequently on merges to main to proactively detect regressions. In addition, this will also keep Gamma CDN release up to date for manual testing.
Implementation
Created
smoke.yml, which uses secrets from new environmentsmoke-test.Future
I would like the release workflow
cd.ymlto re-usesmoke.yml, but I need to do some more testing for that change by doing a test release toaws-rum-web-experimentalTests
https://github.com/aws-observability/aws-rum-web/actions/runs/20116775237
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.