Skip to content

Commit 50ecd4d

Browse files
Add workflow to Validate E2E Tests Are Accounted For in a Repo (#440)
*Issue description:* *Description of changes:* Add new validation workflow: - This validation is to ensure that all ApplicationSignals e2e test workflows relevant to this repo are actually being used in this repo. - This validation supports excluding specific tests via comma-separated input parameter - Validation is done between `TESTS_EXPECTED` and `TESTS_ACTUALLY_USED` - TESTS_EXPECTED: Generated by filtering test framework workflows in `.github/workflows/` by language prefix, excluding perf/lambda tests, and applying exclusions - TESTS_ACTUALLY_USED: Generated by parsing `application-signals-e2e-test.yml` to extract workflow references, filtering by test framework, and applying exclusions *Testing:* CWAgent: <img width="3138" height="1872" alt="image" src="https://github.com/user-attachments/assets/37633b33-7798-4d45-825b-62e46b022965" /> ADOT Java: <img width="3138" height="1712" alt="image" src="https://github.com/user-attachments/assets/1dfab8a9-362f-4d10-b206-e9ed2aa8ac54" /> ADOT Python: <img width="3138" height="1632" alt="image" src="https://github.com/user-attachments/assets/74251329-857c-47c4-880c-60423d536b2d" /> ADOT JS: <img width="3138" height="1632" alt="image" src="https://github.com/user-attachments/assets/4325f942-e6d9-4668-a693-a30d34cc089d" /> ADOT .NET: <img width="3138" height="1712" alt="image" src="https://github.com/user-attachments/assets/fdbb3f31-207a-4950-9c52-140eb16629de" /> By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Mahad Janjua <[email protected]>
1 parent 4a3437d commit 50ecd4d

File tree

1 file changed

+101
-0
lines changed

1 file changed

+101
-0
lines changed
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
name: Validate E2E Tests Are Accounted For
2+
on:
3+
workflow_call:
4+
inputs:
5+
exclusions:
6+
description: 'Comma-separated list of test files to exclude from validation'
7+
type: string
8+
required: false
9+
default: ''
10+
11+
env:
12+
TEST_RESOURCES_FOLDER: ${GITHUB_WORKSPACE}
13+
EXCLUSIONS: ${{ inputs.exclusions }}
14+
15+
jobs:
16+
validate-e2e-tests-are-accounted-for:
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Install Ruby
20+
uses: ruby/[email protected]
21+
with:
22+
ruby-version: "3.4"
23+
24+
- name: Checkout aws-application-signals-test-framework
25+
run: |
26+
cd ${{ env.TEST_RESOURCES_FOLDER }}
27+
git clone https://github.com/aws-observability/aws-application-signals-test-framework.git
28+
29+
- name: Determine workflow files expected to be used by ADOT Java
30+
if: contains(github.repository, 'aws-otel-java-instrumentation')
31+
run: |
32+
cd ${{ env.TEST_RESOURCES_FOLDER }}/aws-application-signals-test-framework/.github/workflows/
33+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
34+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_EXPECTED: $EXCLUSIONS"
35+
echo "TESTS_EXPECTED=$(ls | grep -E '(^java-)|(^metric-limiter-)' | grep -Ev '(lambda-layer-perf-test|otlp-ocb)' | grep -E 'test\.ya?ml$' | { [[ -n "$EXCLUSIONS" ]] && grep -v -F -f <(echo "$EXCLUSIONS" | tr ',' '\n') || cat; } | sort | tr '\n' ' ' | sed 's/ $//')" >> $GITHUB_ENV
36+
37+
- name: Determine workflow files expected to be used by ADOT JavaScript
38+
if: contains(github.repository, 'aws-otel-js-instrumentation')
39+
run: |
40+
cd ${{ env.TEST_RESOURCES_FOLDER }}/aws-application-signals-test-framework/.github/workflows/
41+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
42+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_EXPECTED: $EXCLUSIONS"
43+
echo "TESTS_EXPECTED=$(ls | grep -E '^node-' | grep -v 'lambda-layer-perf-test' | grep -E 'test\.ya?ml$' | { [[ -n "$EXCLUSIONS" ]] && grep -v -F -f <(echo "$EXCLUSIONS" | tr ',' '\n') || cat; } | sort | tr '\n' ' ' | sed 's/ $//')" >> $GITHUB_ENV
44+
45+
- name: Determine workflow files expected to be used by ADOT Python
46+
if: contains(github.repository, 'aws-otel-python-instrumentation')
47+
run: |
48+
cd ${{ env.TEST_RESOURCES_FOLDER }}/aws-application-signals-test-framework/.github/workflows/
49+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
50+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_EXPECTED: $EXCLUSIONS"
51+
echo "TESTS_EXPECTED=$(ls | grep -E '^python-' | grep -v 'lambda-layer-perf-test' | grep -E 'test\.ya?ml$' | { [[ -n "$EXCLUSIONS" ]] && grep -v -F -f <(echo "$EXCLUSIONS" | tr ',' '\n') || cat; } | sort | tr '\n' ' ' | sed 's/ $//')" >> $GITHUB_ENV
52+
53+
- name: Determine workflow files expected to be used by ADOT .NET
54+
if: contains(github.repository, 'aws-otel-dotnet-instrumentation')
55+
run: |
56+
cd ${{ env.TEST_RESOURCES_FOLDER }}/aws-application-signals-test-framework/.github/workflows/
57+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
58+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_EXPECTED: $EXCLUSIONS"
59+
echo "TESTS_EXPECTED=$(ls | grep -E '^dotnet-' | grep -v 'lambda-layer-perf-test' | grep -E 'test\.ya?ml$' | { [[ -n "$EXCLUSIONS" ]] && grep -v -F -f <(echo "$EXCLUSIONS" | tr ',' '\n') || cat; } | sort | tr '\n' ' ' | sed 's/ $//')" >> $GITHUB_ENV
60+
61+
- name: Determine workflow files expected to be used by CloudWatch Agent
62+
if: contains(github.repository, 'amazon-cloudwatch-agent')
63+
run: |
64+
cd ${{ env.TEST_RESOURCES_FOLDER }}/aws-application-signals-test-framework/.github/workflows/
65+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
66+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_EXPECTED: $EXCLUSIONS"
67+
echo "TESTS_EXPECTED=$(ls | grep -E '(^java-)|(^node-)|(^python-)|(^dotnet-)|(^metric-limiter-)' | grep -Ev '(sigv4|lambda-test|lambda-layer-perf-test|ocb|genesis)' | grep -E 'test\.ya?ml$' | { [[ -n "$EXCLUSIONS" ]] && grep -v -F -f <(echo "$EXCLUSIONS" | tr ',' '\n') || cat; } | sort | tr '\n' ' ' | sed 's/ $//')" >> $GITHUB_ENV
68+
69+
- name: Determine workflow files actually being used by ${{ github.repository }}
70+
run: |
71+
cd ${{ env.TEST_RESOURCES_FOLDER }}
72+
git clone -b ${{ github.ref_name }} https://github.com/${{ github.repository }}.git ${{ github.repository }}
73+
cd ${{ github.repository }}/.github/workflows/
74+
75+
# Get all test framework workflows actually used in application-signals-e2e-test.yml, with exclusions applied
76+
EXCLUSIONS="${{ env.EXCLUSIONS }}"
77+
[[ -n "$EXCLUSIONS" ]] && echo "Applying exclusions to TESTS_ACTUALLY_USED: $EXCLUSIONS"
78+
echo "TESTS_ACTUALLY_USED=$(ruby -ryaml -e '
79+
exclusions = "${{ env.EXCLUSIONS }}".split(",").map(&:strip).reject(&:empty?)
80+
workflows = YAML.load_file("application-signals-e2e-test.yml")["jobs"].values
81+
.map { |v| v["uses"] }
82+
.compact
83+
.uniq
84+
# Ignore the Job that triggers this "validate-e2e-tests-are-accounted-for" validation
85+
.select { |u| u.include?("aws-application-signals-test-framework/.github/workflows") && !u.include?("validate-e2e-tests-are-accounted-for") }
86+
.map { |u| u.split("/").last.split("@").first }
87+
.reject { |w| exclusions.include?(w) }
88+
.sort
89+
puts workflows.join(" ")
90+
')" >> $GITHUB_ENV
91+
92+
- name: Compare TESTS_ACTUALLY_USED and TESTS_EXPECTED
93+
run: |
94+
echo "Tests being used:"
95+
echo "- $TESTS_ACTUALLY_USED"
96+
echo "Tests expected to be used:"
97+
echo "- $TESTS_EXPECTED"
98+
if [[ -z "$TESTS_ACTUALLY_USED" || -z "$TESTS_EXPECTED" || "$TESTS_ACTUALLY_USED" != "$TESTS_EXPECTED" ]]; then
99+
echo "Mismatch found!"
100+
exit 1
101+
fi

0 commit comments

Comments
 (0)