ci: Add instrumentation test log parsing and reporting to our test automation scripts #3003
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A
Description of changes: This change adds a python script which parses the instrumentation logs from Android Test Orchestrator that are retrieved from a Device Farm test run, improves the error reporting in the CodeBuild report UI, generates a Junit report (for CodeBuild report UI), and sends relevant metrics to CloudWatch. This change also modifies the Device Farm
schedule-runcommand to pass in the custom Device Farm TestSpec ARN that we want to use which uses the Android Test Orchestrator rather than the default test runner.nota bene: This is the Amplify part of a larger change to our internal CodeBuild project in an effort to better understand the state of our integration tests. This dry run for this PR will fail until the changes in the CodeBuild project are done.
How did you test these changes?
I've tested this change in a test AWS account that runs the build E2E on this integ-test-reporting branch. The tests run as expected, the CodeBuild test report is generated and can be read in the UI.
Documentation update required?
General Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.