Skip to content

Commit 96155c1

Browse files
mooc9988mooc9988
authored
feat(e2e): use cloud vms for e2e tests; move wal path (#1862)
* refactor(e2e): test running on cloud provider Signed-off-by: Curtis Wan <[email protected]> * wip: write /etc/docker/daemon.json if needed Signed-off-by: Curtis Wan <[email protected]> * wip: write /etc/docker/daemon.json if needed Signed-off-by: Curtis Wan <[email protected]> * wip: remove docker setup Signed-off-by: Curtis Wan <[email protected]> * wip: change test py Signed-off-by: Curtis Wan <[email protected]> * wip: test artifacts Signed-off-by: Curtis Wan <[email protected]> * wip: fix path Signed-off-by: Curtis Wan <[email protected]> * wip: fix path Signed-off-by: Curtis Wan <[email protected]> * wip: report results Signed-off-by: Curtis Wan <[email protected]> * wip: copy only valid files Signed-off-by: Curtis Wan <[email protected]> * wip: sum in e2e-run Signed-off-by: Curtis Wan <[email protected]> * wip: fix space Signed-off-by: Curtis Wan <[email protected]> * wip: fix NULL Signed-off-by: Curtis Wan <[email protected]> * fix json Signed-off-by: Curtis Wan <[email protected]> * remove unneeded inputs Signed-off-by: Curtis Wan <[email protected]> * move wal location; enhance upload-artifact; roll back input options Signed-off-by: Curtis Wan <[email protected]> * add java-setup Signed-off-by: Curtis Wan <[email protected]> * remove test yml Signed-off-by: Curtis Wan <[email protected]> * only retain reports Signed-off-by: Curtis Wan <[email protected]> * filter archive files Signed-off-by: Curtis Wan <[email protected]> * filter archive files Signed-off-by: Curtis Wan <[email protected]> * remove test yml Signed-off-by: Curtis Wan <[email protected]> --------- Signed-off-by: Curtis Wan <[email protected]>
1 parent e69770b commit 96155c1

File tree

5 files changed

+56
-42
lines changed

5 files changed

+56
-42
lines changed

‎.github/workflows/e2e-run.yml‎

Lines changed: 40 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,25 +14,44 @@ on:
1414
test-path:
1515
required: false
1616
type: string
17-
storage-path:
18-
required: true
19-
type: string
17+
outputs:
18+
artifact-id:
19+
description: "Artifact ID of the test results"
20+
value: ${{ jobs.run_e2e.outputs.artifact-id }}
21+
success-num:
22+
description: "Number of successful tests"
23+
value: ${{ jobs.run_e2e.outputs.success-num }}
24+
failure-num:
25+
description: "Number of failed tests"
26+
value: ${{ jobs.run_e2e.outputs.failure-num }}
27+
run-time-secs:
28+
description: "Total run time in seconds"
29+
value: ${{ jobs.run_e2e.outputs.run-time-secs }}
30+
2031

2132
jobs:
2233
run_e2e:
2334
name: "Run E2E tests"
2435
runs-on: ${{ inputs.runner }}
25-
env:
26-
TC_GENERAL_MIRROR_URL: "mirrors.ustc.edu.cn"
36+
# Map the job outputs to step outputs
37+
outputs:
38+
artifact-id: ${{ steps.archive-artifacts.outputs.artifact-id }}
39+
success-num: ${{ steps.extract-results.outputs.success-num }}
40+
failure-num: ${{ steps.extract-results.outputs.failure-num }}
41+
run-time-secs: ${{ steps.extract-results.outputs.run-time-secs }}
2742
steps:
2843
- name: Checkout
2944
uses: actions/checkout@v4
3045
- name: Clean last running results
3146
run: |
3247
rm -rf results
33-
rm -rf "${{ inputs.storage-path }}/${{ inputs.suite-id }}"
48+
- uses: actions/setup-java@v4
49+
with:
50+
distribution: 'temurin'
51+
java-version: '17'
52+
cache: 'gradle'
3453
- name: Setup Gradle
35-
uses: gradle/[email protected]
54+
uses: gradle/actions/setup-gradle@v3
3655
- name: Run E2E tests with yaml
3756
if: ${{ inputs.test-path == '' }}
3857
run: ./tests/docker/run_tests.sh
@@ -45,13 +64,24 @@ jobs:
4564
env:
4665
TC_PATHS: ${{ inputs.test-path }}
4766
shell: bash
48-
- name: Move results
67+
- name: Extract results
68+
id: extract-results
4969
run: |
5070
results_path="$(pwd)/results/$(readlink results/latest | cut -d'/' -f5)"
51-
mkdir -p "${{ inputs.storage-path }}/${{ inputs.suite-id }}"
52-
mv "${results_path}" "${{ inputs.storage-path }}/${{ inputs.suite-id }}"
71+
echo "success-num=$(jq .num_passed $results_path/report.json)" >> $GITHUB_OUTPUT
72+
echo "failure-num=$(jq .num_failed $results_path/report.json)" >> $GITHUB_OUTPUT
73+
echo "run-time-secs=$(jq .run_time_seconds $results_path/report.json)" >> $GITHUB_OUTPUT
5374
if: ${{ always() }}
5475
shell: bash
76+
- name: Archive result artifacts
77+
id: archive-artifacts
78+
uses: actions/upload-artifact@v4
79+
with:
80+
name: ${{ inputs.suite-id }}
81+
retention-days: 3
82+
compression-level: 1
83+
path: |
84+
results/*/report*
5585
- name: Bring down docker containers
5686
run: ./tests/docker/ducker-ak down
5787
shell: bash

‎.github/workflows/nightly-extra-e2e.yml‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@ jobs:
2121
with:
2222
suite-id: "benchmarks"
2323
test-path: "tests/kafkatest/benchmarks"
24-
storage-path: "/data/github-actions/reports"
2524
runner: ${{ inputs.runs_on || 'extra' }}
2625
connect_e2e_1:
2726
name: "Run connect E2E Tests 1"
@@ -30,7 +29,6 @@ jobs:
3029
with:
3130
suite-id: "connect1"
3231
test-yaml: "tests/suites/connect_test_suite1.yml"
33-
storage-path: "/data/github-actions/reports"
3432
runner: ${{ inputs.runs_on || 'extra' }}
3533
connect_e2e_2:
3634
name: "Run connect E2E Tests 2"
@@ -39,7 +37,6 @@ jobs:
3937
with:
4038
suite-id: "connect2"
4139
test-yaml: "tests/suites/connect_test_suite2.yml"
42-
storage-path: "/data/github-actions/reports"
4340
runner: ${{ inputs.runs_on || 'extra' }}
4441
connect_e2e_3:
4542
name: "Run connect E2E Tests 3"
@@ -48,7 +45,6 @@ jobs:
4845
with:
4946
suite-id: "connect3"
5047
test-yaml: "tests/suites/connect_test_suite3.yml"
51-
storage-path: "/data/github-actions/reports"
5248
runner: ${{ inputs.runs_on || 'extra' }}
5349
streams_e2e:
5450
name: "Run streams E2E Tests"
@@ -57,7 +53,6 @@ jobs:
5753
with:
5854
suite-id: "streams"
5955
test-path: "tests/kafkatest/tests/streams"
60-
storage-path: "/data/github-actions/reports"
6156
runner: ${{ inputs.runs_on || 'extra' }}
6257
e2e_summary:
6358
name: "E2E Tests Summary"
@@ -68,7 +63,8 @@ jobs:
6863
- name: Report results
6964
run: python3 tests/report_e2e_results.py
7065
env:
66+
CURRENT_REPO: ${{ github.repository }}
67+
RUN_ID: ${{ github.run_id }}
7168
WEB_HOOK_URL: ${{ secrets.E2E_REPORT_WEB_HOOK_URL }}
72-
SHOW_RESULTS_URL: ${{ secrets.E2E_REPORT_SHOW_RESULTS_URL2 }}
73-
STORAGE_PATH: "/data/github-actions/reports"
69+
DATA_MAP: "{\"benchmarks_e2e\": ${{ toJSON(needs.benchmarks_e2e.outputs) }}, \"connect_e2e_1\": ${{ toJSON(needs.connect_e2e_1.outputs) }}}, \"connect_e2e_2\": ${{ toJSON(needs.connect_e2e_2.outputs) }}, \"connect_e2e_3\": ${{ toJSON(needs.connect_e2e_3.outputs) }}, \"streams_e2e\": ${{ toJSON(needs.streams_e2e.outputs) }}"
7470
REPORT_TITLE_PREFIX: "Extra"

‎.github/workflows/nightly-main-e2e.yml‎

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@ jobs:
2121
with:
2222
suite-id: "main1"
2323
test-yaml: "tests/suites/main_kos_test_suite1.yml"
24-
storage-path: "/data/github-actions/reports"
2524
runner: ${{ inputs.runs_on || 'main' }}
2625
main_e2e_2:
2726
name: "Run Main E2E Tests 2"
@@ -30,7 +29,6 @@ jobs:
3029
with:
3130
suite-id: "main2"
3231
test-yaml: "tests/suites/main_kos_test_suite2.yml"
33-
storage-path: "/data/github-actions/reports"
3432
runner: ${{ inputs.runs_on || 'main' }}
3533
main_e2e_3:
3634
name: "Run Main E2E Tests 3"
@@ -39,7 +37,6 @@ jobs:
3937
with:
4038
suite-id: "main3"
4139
test-yaml: "tests/suites/main_kos_test_suite3.yml"
42-
storage-path: "/data/github-actions/reports"
4340
runner: ${{ inputs.runs_on || 'main' }}
4441
main_e2e_4:
4542
name: "Run Main E2E Tests 4"
@@ -48,7 +45,6 @@ jobs:
4845
with:
4946
suite-id: "main4"
5047
test-yaml: "tests/suites/main_kos_test_suite4.yml"
51-
storage-path: "/data/github-actions/reports"
5248
runner: ${{ inputs.runs_on || 'main' }}
5349
main_e2e_5:
5450
name: "Run Main E2E Tests 5"
@@ -57,7 +53,6 @@ jobs:
5753
with:
5854
suite-id: "main5"
5955
test-yaml: "tests/suites/main_kos_test_suite5.yml"
60-
storage-path: "/data/github-actions/reports"
6156
runner: ${{ inputs.runs_on || 'main' }}
6257
main_e2e_6:
6358
name: "Run Main E2E Tests 6"
@@ -66,7 +61,6 @@ jobs:
6661
with:
6762
suite-id: "main6"
6863
test-yaml: "tests/suites/main_kos_test_suite6.yml"
69-
storage-path: "/data/github-actions/reports"
7064
runner: ${{ inputs.runs_on || 'main' }}
7165
e2e_summary:
7266
runs-on: ${{ inputs.runs_on || 'main' }}
@@ -77,7 +71,8 @@ jobs:
7771
- name: Report results
7872
run: python3 tests/report_e2e_results.py
7973
env:
74+
CURRENT_REPO: ${{ github.repository }}
75+
RUN_ID: ${{ github.run_id }}
8076
WEB_HOOK_URL: ${{ secrets.E2E_REPORT_WEB_HOOK_URL }}
81-
SHOW_RESULTS_URL: ${{ secrets.E2E_REPORT_SHOW_RESULTS_URL }}
82-
STORAGE_PATH: "/data/github-actions/reports"
77+
DATA_MAP: "{\"main_e2e_1\": ${{ toJSON(needs.main_e2e_1.outputs) }}, \"main_e2e_2\": ${{ toJSON(needs.main_e2e_2.outputs) }}}, \"main_e2e_3\": ${{ toJSON(needs.main_e2e_3.outputs) }}, \"main_e2e_4\": ${{ toJSON(needs.main_e2e_4.outputs) }}, \"main_e2e_5\": ${{ toJSON(needs.main_e2e_5.outputs) }}, \"main_e2e_6\": ${{ toJSON(needs.main_e2e_6.outputs) }}"
8378
REPORT_TITLE_PREFIX: "Main"

‎tests/kafkatest/services/kafka/templates/kafka.properties‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ elasticstream.endpoint=s3://
141141
s3.endpoint=http://10.5.0.2:4566
142142
s3.region=us-east-1
143143
s3.bucket=ko3
144-
s3.wal.path=/mnt/kafka/kafka-data-logs-1/s3wal
144+
s3.wal.path=/mnt/kafka/s3wal
145145
s3.wal.capacity=209715200
146146
s3.wal.cache.size=104857600
147147
s3.wal.upload.threshold=52428800

‎tests/report_e2e_results.py‎

Lines changed: 9 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -22,32 +22,25 @@
2222

2323
if __name__ == '__main__':
2424
web_hook_url = os.getenv('WEB_HOOK_URL')
25-
show_results_url = os.getenv('SHOW_RESULTS_URL')
26-
storage_path = os.getenv('STORAGE_PATH')
2725
title_prefix = os.getenv('REPORT_TITLE_PREFIX')
2826
title_prefix = title_prefix if title_prefix else ""
2927

30-
# iterate all the folders in the storage path
31-
base_path = Path(storage_path)
32-
all_tests_folder = sorted(base_path.iterdir(), key=os.path.getmtime)
28+
data_map = json.loads(os.getenv('DATA_MAP'))
29+
artifact_url_prefix = "https://github.com/%s/actions/runs/%s/artifacts/" % (os.getenv('CURRENT_REPO'), os.getenv('RUN_ID'))
3330

3431
total_passed = 0
3532
total_failed = 0
3633
reports_dict = {}
3734

38-
for path in all_tests_folder:
39-
if not path.is_dir():
35+
for key, value in data_map.items():
36+
if not value:
4037
continue
41-
suite_id = path.name
42-
with open(os.path.join(path, "report.json")) as f:
43-
data = json.load(f)
44-
total_failed += data['num_failed']
45-
total_passed += data['num_passed']
46-
reports_dict[suite_id] = {'num_passed': data['num_passed'], 'num_failed': data['num_failed'],
47-
'run_time_seconds': data['run_time_seconds']}
38+
total_failed += int(value['failure-num'])
39+
total_passed += int(value['success-num'])
40+
reports_dict[key] = {'num_passed': int(value['success-num']), 'num_failed': int(value['failure-num']), 'run_time_seconds': float(value['run-time-secs']), 'artifact-id': value['artifact-id']}
4841

4942
if not reports_dict:
50-
print("No reports found in %s" % storage_path)
43+
print("No reports found.")
5144
exit(0)
5245

5346
post_data = {"msg_type": "interactive", "card": {
@@ -85,7 +78,7 @@
8578
"content": "See more details for %s" % key,
8679
"tag": "lark_md"
8780
},
88-
"url": "http://%s/kafka/%s" % (show_results_url, key),
81+
"url": artifact_url_prefix + value['artifact-id'],
8982
"type": "default",
9083
"value": {}
9184
}],

0 commit comments

Comments
 (0)