Skip to content

Commit 741ab38

Browse files
reactive-firewallreactive-firewall
committed
[CONFIG] Initial implementation of CI/CD round 2 improvements (- WIP #338 -)
Changes in file .github/workflows/CI-BUILD.yml: * implemented uploading build info file for triggered builds * added more build details to the BUILD-info.txt artifact * improved readability of logs slightly * moved commit build details to build action * added logic to post commit comment with build details * added default token for `gh api` usage * Refactored to allow summary comment with updated permissions * comment via action if build success * related work Changes in file .github/workflows/CI-MATs.yml: * updated check_build to fetch the build info file * added cleanup prototype after fetching build info * removed debug check for fetched build-info * improved cleanup by moving the artifact into place * added logic to load the build-info artifact * separated status and summary jobs (summaries will be built in new template action) * updated to use new generated summaries and concatenate them * corrected overlooked logic error in summary output * added logic to post commit comment with MATs details * minor refactor for quoting * added default token for `gh api` usage * added job to comment with summary on relevant commit * minor changes to format * related work Changes in file .github/workflows/Tests.yml: * added logic to get triggering workflow id * implement downloading and loading info from MATs * test for coverage job to use triggering commit * fix for style inconsistency * updated most jobs to need the check_mats job for build_info values * use trigger artifact info * refactor for template actions a little * expand env values with trigger artifact info * related work Changes in file .github/actions/checkout-and-rebuild/action.yml: * updated artifact pattern to align with recent changes to CI-MATs.yml * moved summary to re-usable-action * use env for values instead of API * added missing shell value for template action * reformat build details so they can be hidden unless needed * added logic to handle windows pip upgrade workaround * related work Additions with file .github/actions/run-minimal-acceptance-tests/action.yml: * Initial implementation of template action to run more detailed MATs in CI/CD * minor improvement for coverage reports * minor performance tweaks (eg. printf is faster than echo) * improved outputs a bit * related work Changes in file .github/workflows/makefile-lint.yml: * version bump (cherry-picked from 0bb6794)
1 parent 0eb6118 commit 741ab38

File tree

7 files changed

+796
-166
lines changed

7 files changed

+796
-166
lines changed

‎.github/actions/checkout-and-rebuild/action.yml‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: 'Checkout and use Build'
33
description: 'checks-out the given commit and fetches the build artifact'
44
author: 'Mr. Walls'
55
branding:
6-
icon: 'chevron-down'
6+
icon: 'download-cloud'
77
color: 'blue'
88
inputs:
99
sha:
@@ -88,25 +88,32 @@ runs:
8888
python-version: ${{ inputs.python-version }}
8989
cache: 'pip' # caching pip dependencies
9090
if: ${{ !cancelled() }}
91+
- run: python -m pip install --upgrade pip
92+
shell: bash
93+
if: ${{ !cancelled() && runner.os == 'Windows' }}
9194
- name: "Install Test Dependencies"
9295
shell: bash
93-
run: make -j1 -f Makefile test-reqs ;
96+
run: |
97+
printf "%s\n" "::group::prep-test-reqs"
98+
make -j1 -f Makefile test-reqs || exit 1
99+
printf "%s\n" "::endgroup::"
94100
- id: output_artifact_name
95101
if: ${{ success() }}
96102
shell: bash
97-
run: printf "artifact-name=%s\n" multicast-build-${{ steps.output_sha.outputs.sha }}.zip >> "$GITHUB_OUTPUT"
103+
run: printf "artifact-name=%s\n" multicast-build-${{ steps.output_sha.outputs.sha }} >> "$GITHUB_OUTPUT"
98104
- name: "Fetch Build Files"
99105
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
100106
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
101107
with:
102108
path: ${{ inputs.path }}/dist
103-
pattern: multicast-build-${{ steps.output_sha.outputs.sha }}.zip
109+
pattern: multicast-build-${{ steps.output_sha.outputs.sha }}
104110
merge-multiple: true
105111
repository: reactive-firewall/multicast
106112
github-token: ${{ inputs.token }}
107113
run-id: ${{ inputs.build-run-id }}
108114
- name: "Enumerate Fetched Files"
109115
id: output_artifact_files
116+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
110117
env:
111118
BUILD_MATCH_PATTERN: "dist/multicast-*-*.whl dist/multicast-*.tar.gz"
112119
SCRIPT_NAME: ".github/actions/checkout-and-rebuild/action.yml"
@@ -116,11 +123,12 @@ runs:
116123
if [ -z "$FILES" ]; then
117124
printf "::warning file=%s:: %s\n" "${SCRIPT_NAME}" "No Built files found."
118125
printf "%s\n" "files=" >> "$GITHUB_OUTPUT"
126+
exit 1
119127
else
120128
printf "%s\n" "Built files found:"
121129
printf "%s\n" "$FILES"
122-
# Replace line breaks with commas for GitHub Action Output
130+
# Replace line breaks with spaces for GitHub Action Output
123131
FILES="${FILES//$'\n'/ }"
124132
printf "%s\n" "files=$FILES" >> "$GITHUB_OUTPUT"
133+
exit 0
125134
fi
126-
if: ${{ success() }}

‎.github/actions/run-minimal-acceptance-tests/action.yml‎

Lines changed: 310 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,310 @@
1+
---
2+
name: 'Multicast Minimal Acceptance Tests'
3+
description: 'Performs various Minimal Acceptance Testing for the Multicast Project'
4+
author: 'Mr. Walls'
5+
branding:
6+
icon: 'bar-chart-2'
7+
color: 'black'
8+
inputs:
9+
sha:
10+
description: |
11+
The commit to report mats for. Should already be cloned and checked-out beforehand.
12+
required: true
13+
default: ${{ github.server_url == 'https://github.com' && github.sha || 'HEAD' }}
14+
token:
15+
description: |
16+
The token used to authenticate when uploading results from the Minimal Accceptance Testing.
17+
When running this action on github.com, the default value is sufficient. When running on
18+
GHES, you can pass a personal access token for github.com if you are experiencing
19+
rate limiting.
20+
default: ${{ github.server_url == 'https://github.com' && github.token || '' }}
21+
required: true
22+
python-version:
23+
description: |
24+
The python version to use. The default is to use the value of the environment
25+
variable 'PYTHON_VERSION'.
26+
default: '3.12'
27+
required: true
28+
outputs:
29+
sha:
30+
description: "The SHA of the commit checked-out"
31+
value: ${{ steps.output_sha.outputs.sha || 'HEAD' }}
32+
python-version:
33+
description: "The python version that was used in the run"
34+
value: ${{ steps.output_python.outputs.python-version || '' }}
35+
mats_status:
36+
description: "The outcome of evaluating the MATs steps"
37+
value: ${{ steps.mats_outcome.outcome || 'cancelled' }}
38+
mats_build_status:
39+
description: "The outcome of the MATs Build Group"
40+
value: ${{ steps.mats_build.outcome || 'cancelled' }}
41+
mats_bootstrap_status:
42+
description: "The outcome of the MATs Bootstrap Group"
43+
value: ${{ steps.mats_bootstrap.outcome || 'cancelled' }}
44+
mats_basic_status:
45+
description: "The outcome of the MATs Basic Group"
46+
value: ${{ steps.mats_basic.outcome || 'cancelled' }}
47+
mats_say_status:
48+
description: "The outcome of the MATs SAY Group"
49+
value: ${{ steps.mats_say.outcome || 'cancelled' }}
50+
mats_hear_status:
51+
description: "The outcome of the MATs HEAR Group"
52+
value: ${{ steps.mats_hear.outcome || 'cancelled' }}
53+
mats_usage_status:
54+
description: "The outcome of the MATs Usage Group"
55+
value: ${{ steps.mats_usage.outcome || 'cancelled' }}
56+
mats_coverage_status:
57+
description: "The outcome of the MATs Coverage Group"
58+
value: ${{ steps.mats_coverage.outcome || 'cancelled' }}
59+
mats_doctests_status:
60+
description: "The outcome of the MATs DocTests Group"
61+
value: ${{ steps.mats_doctests.outcome || 'cancelled' }}
62+
artifact-name:
63+
description: "The uploaded artifact-name"
64+
value: ${{ steps.output_artifact_name.outputs.artifact-name }}
65+
artifact-files:
66+
description: "The uploaded artifact-files"
67+
value: ${{ steps.output_artifact_files.outputs.files }}
68+
artifact-id:
69+
description: "The uploaded artifact-id"
70+
value: ${{ steps.upload.outputs.artifact-id }}
71+
artifact-url:
72+
description: "The uploaded artifact-url"
73+
value: ${{ steps.upload.outputs.artifact-url }}
74+
artifact-digest:
75+
description: "The uploaded artifact-digest"
76+
value: ${{ steps.upload.outputs.artifact-digest }}
77+
78+
runs:
79+
using: composite
80+
steps:
81+
- name: "Calculate Commit SHA"
82+
id: output_sha
83+
shell: bash
84+
run: |
85+
printf "sha=%s\n" $(git rev-parse --verify HEAD) >> "$GITHUB_OUTPUT"
86+
printf "BUILD_SHA=%s\n" $(git rev-parse --verify HEAD) >> "$GITHUB_ENV"
87+
- name: "Setup Python"
88+
id: output_python
89+
if: ${{ !cancelled() }}
90+
env:
91+
PYTHON_VERSION_INPUT: ${{ inputs.python-version }}
92+
shell: bash
93+
run: |
94+
if [[ -z $PYTHON_VERSION_INPUT ]]; then
95+
printf "python-version=%s\n" "${PYTHON_VERSION_INPUT}" >> "$GITHUB_OUTPUT"
96+
PYTHON_VERSION=${PYTHON_VERSION_INPUT}
97+
else
98+
printf "python-version=%s\n" "${PYTHON_VERSION}" >> "$GITHUB_OUTPUT"
99+
fi
100+
printf "%s\n" "PYTHON_VERSION=${PYTHON_VERSION}" >> "$GITHUB_ENV"
101+
- name: "Prepare Artifact Name"
102+
id: output_artifact_name
103+
if: ${{ !cancelled() }}
104+
shell: bash
105+
run: |
106+
printf "artifact-name=%s\n" multicast-mats-${{ steps.output_sha.outputs.sha }}-part-$(uuidgen) >> "$GITHUB_OUTPUT"
107+
printf "%s\n" "MATS_STEP_SUMMARY=MATs-Summary-Artifact-${PYTHON_VERSION}.txt" >> "$GITHUB_ENV"
108+
- name: "Run Build MATs"
109+
id: mats_build
110+
if: ${{ (github.repository == 'reactive-firewall/multicast') && !cancelled() }}
111+
shell: bash
112+
run: |
113+
printf "%s\n" "::group::Build-MATs"
114+
make -f Makefile test-mat-build || exit 1
115+
printf "%s\n" "::endgroup::"
116+
- name: "Run bootstrap MATs"
117+
id: mats_bootstrap
118+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
119+
shell: bash
120+
run: |
121+
printf "%s\n" "::group::Bootstrap-MATs"
122+
make -f Makefile test-mat-bootstrap || exit 1
123+
printf "%s\n" "::endgroup::"
124+
- name: "Run basic MATs"
125+
id: mats_basic
126+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
127+
shell: bash
128+
run: |
129+
printf "%s\n" "::group::Basic-MATs"
130+
make -f Makefile test-mat-basic || exit 1
131+
printf "%s\n" "::endgroup::"
132+
- name: "Run SAY MATs"
133+
id: mats_say
134+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
135+
shell: bash
136+
run: |
137+
printf "%s\n" "::group::SAY-MATs"
138+
make -f Makefile test-mat-say || exit 1
139+
printf "%s\n" "::endgroup::"
140+
- name: "Run HEAR MATs"
141+
id: mats_hear
142+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
143+
shell: bash
144+
run: |
145+
printf "%s\n" "::group::HEAR-MATs"
146+
make -f Makefile test-mat-hear || exit 1
147+
printf "%s\n" "::endgroup::"
148+
- name: "Run Usage MATs"
149+
id: mats_usage
150+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
151+
shell: bash
152+
run: |
153+
printf "%s\n" "::group::Usage-MATs"
154+
make -f Makefile test-mat-usage || exit 1
155+
printf "%s\n" "::endgroup::"
156+
- name: "Run Extra Coverage tests"
157+
id: mats_coverage
158+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
159+
shell: bash
160+
run: |
161+
printf "%s\n" "::group::Coverage-MATs"
162+
make -f Makefile test-extra-coverage || exit 1
163+
printf "%s\n" "::endgroup::"
164+
- name: "Run doctest MATs"
165+
id: mats_doctests
166+
if: ${{ (github.repository == 'reactive-firewall/multicast') && success() }}
167+
shell: bash
168+
run: |
169+
printf "%s\n" "::group::DocTests-MATs"
170+
make -f Makefile test-mat-doctests || exit 1
171+
printf "%s\n" "::endgroup::"
172+
- name: "Evaluate MATs"
173+
id: mats_outcome
174+
if: ${{ !cancelled() }}
175+
shell: bash
176+
run: |
177+
if [[ "${{ steps.mats_build.outcome }}" == "success" ]] ; then
178+
THE_RESULT="success"
179+
else
180+
THE_RESULT="failure"
181+
fi
182+
if [[ "${{ steps.mats_bootstrap.outcome }}" == "success" && "${THE_RESULT}" == "success" ]] ; then
183+
THE_RESULT="success"
184+
else
185+
THE_RESULT="failure"
186+
fi
187+
if [[ "${{ steps.mats_basic.outcome }}" == "success" && "${THE_RESULT}" == "success" ]] ; then
188+
THE_RESULT="success"
189+
else
190+
THE_RESULT="failure"
191+
fi
192+
if [[ "${{ steps.mats_say.outcome }}" == "success" && "${THE_RESULT}" == "success" ]] ; then
193+
THE_RESULT="success"
194+
else
195+
THE_RESULT="failure"
196+
fi
197+
if [[ "${{ steps.mats_hear.outcome }}" == "success" && "${THE_RESULT}" == "success" ]] ; then
198+
THE_RESULT="success"
199+
else
200+
THE_RESULT="failure"
201+
fi
202+
if [[ "${{ steps.mats_usage.outcome }}" == "success" && "${THE_RESULT}" == "success" ]] ; then
203+
THE_RESULT="success"
204+
else
205+
THE_RESULT="failure"
206+
fi
207+
if [[ "${THE_RESULT}" == "success" ]] ; then
208+
exit 0
209+
else
210+
exit 1
211+
fi
212+
- name: "Summarize MATs"
213+
id: mats_report
214+
if: ${{ always() }}
215+
shell: bash
216+
run: |
217+
if [[ "${{ steps.mats_outcome.outcome }}" == "success" ]] ; then
218+
printf "%s\n" " * :ballot_box_with_check: MATS succeeded with python version \`${PYTHON_VERSION}\` for [${BUILD_SHA}](https://github.com/reactive-firewall/multicast/commit/${BUILD_SHA})" > "$MATS_STEP_SUMMARY"
219+
printf "%s\n" " :ballot_box_with_check: MATS succeeded with python version \`${PYTHON_VERSION}\`" >> "$GITHUB_STEP_SUMMARY"
220+
else
221+
printf "%s\n" " * :x: ~MATS succeeded with python version \`${PYTHON_VERSION}\` for \`${BUILD_SHA}\`~" > "$MATS_STEP_SUMMARY"
222+
printf "%s\n" " :x: ~MATS succeeded with python version \`${PYTHON_VERSION}\`~" >> "$GITHUB_STEP_SUMMARY"
223+
fi
224+
if [[ "${{ steps.mats_build.outcome }}" == "success" ]] ; then
225+
printf "%s\n" " * :ballot_box_with_check: Minimal Build Testing succeeded" >> "$MATS_STEP_SUMMARY"
226+
else
227+
printf "%s\n" " * :x: Minimal Build Testing failed" >> "$MATS_STEP_SUMMARY"
228+
fi
229+
if [[ "${{ steps.mats_bootstrap.outcome }}" == "success" ]] ; then
230+
printf "%s\n" " * :ballot_box_with_check: Minimal Bootstrap Testing succeeded" >> "$MATS_STEP_SUMMARY"
231+
else
232+
printf "%s\n" " * :x: Minimal Bootstrap Testing failed" >> "$MATS_STEP_SUMMARY"
233+
fi
234+
if [[ "${{ steps.mats_basic.outcome }}" == "success" ]] ; then
235+
printf "%s\n" " * :ballot_box_with_check: Basic Minimal Acceptance Testing succeeded" >> "$MATS_STEP_SUMMARY"
236+
else
237+
printf "%s\n" " * :x: Basic Minimal Acceptance Testing failed" >> "$MATS_STEP_SUMMARY"
238+
fi
239+
if [[ "${{ steps.mats_say.outcome }}" == "success" ]] ; then
240+
printf "%s\n" " * :ballot_box_with_check: Minimal Acceptance Testing for \`SAY\` succeeded" >> "$MATS_STEP_SUMMARY"
241+
else
242+
printf "%s\n" " * :x: Minimal Acceptance Testing for \`SAY\` failed" >> "$MATS_STEP_SUMMARY"
243+
fi
244+
if [[ "${{ steps.mats_hear.outcome }}" == "success" ]] ; then
245+
printf "%s\n" " * :ballot_box_with_check: Minimal Acceptance Testing for \`HEAR\` and \`RECV\` succeeded" >> "$MATS_STEP_SUMMARY"
246+
else
247+
printf "%s\n" " * :x: Minimal Acceptance Testing for \`HEAR\` and/or \`RECV\` failed" >> "$MATS_STEP_SUMMARY"
248+
fi
249+
if [[ "${{ steps.mats_usage.outcome }}" == "success" ]] ; then
250+
printf "%s\n" " * :ballot_box_with_check: Minimal Acceptance Testing for the Command-line interface succeeded" >> "$MATS_STEP_SUMMARY"
251+
else
252+
printf "%s\n" " * :x: Basic Minimal Acceptance Testing failed" >> "$MATS_STEP_SUMMARY"
253+
fi
254+
if [[ "${{ steps.mats_doctests.outcome }}" == "success" ]] ; then
255+
printf "%s\n" " * :ballot_box_with_check: Minimal doctest Testing succeeded" >> "$MATS_STEP_SUMMARY"
256+
else
257+
printf "%s\n" " * :x: Minimal doctest Testing failed" >> "$MATS_STEP_SUMMARY"
258+
fi
259+
if [[ "${{ steps.mats_coverage.outcome }}" == "success" ]] ; then
260+
printf "%s\n" " * :ballot_box_with_check: Minimal Testing Coverage succeeded" >> "$MATS_STEP_SUMMARY"
261+
else
262+
printf "%s\n" " * :x: Minimal Testing Coverage failed" >> "$MATS_STEP_SUMMARY"
263+
fi
264+
- name: "Collect and Enumerate Generated Files"
265+
id: output_artifact_files
266+
if: ${{ !cancelled() && (github.repository == 'reactive-firewall/multicast') }}
267+
env:
268+
MATS_MATCH_PATTERN: "MATs-Summary-Artifact-*.txt test-reports/*"
269+
SCRIPT_NAME: ".github/actions/run-minimal-acceptance-tests/action.yml"
270+
shell: bash
271+
run: |
272+
FILES=$(git ls-files -o --exclude-standard -- ${{ env.MATS_MATCH_PATTERN }} )
273+
if [ -z "$FILES" ]; then
274+
printf "::warning file=%s:: %s\n" "${SCRIPT_NAME}" "No Built files found."
275+
printf "%s\n" "files=" >> "$GITHUB_OUTPUT"
276+
exit 1
277+
else
278+
printf "%s\n" "MATs files found:"
279+
printf "%s\n" "$FILES"
280+
mkdir MATS || :
281+
mkdir MATS/test-reports || :
282+
printf "%s\n" "$FILES" | xargs -I{} cp -f "{}" "MATS/{}" || :
283+
# Replace line breaks with spaces for GitHub Action Output
284+
FILES="${FILES//$'\n'/ }"
285+
printf "%s\n" "files=$FILES" >> "$GITHUB_OUTPUT"
286+
exit 0
287+
fi
288+
- name: "Upload Details"
289+
id: upload
290+
if: ${{ !cancelled() && (github.repository == 'reactive-firewall/multicast') }}
291+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
292+
with:
293+
path: MATS
294+
name: ${{ steps.output_artifact_name.outputs.artifact-name }}
295+
if-no-files-found: error
296+
compression-level: 9
297+
overwrite: true
298+
- name: "Cleanup from run"
299+
id: mats_cleanup_success
300+
if: ${{ success() }}
301+
shell: bash
302+
run: |
303+
rm -fRd ./MATs 2>/dev/null || :
304+
- name: "Cleanup from failed run"
305+
id: mats_cleanup_failure
306+
if: ${{ failure() }}
307+
shell: bash
308+
run: |
309+
rm -fRd ./MATs 2>/dev/null
310+
exit 1

0 commit comments

Comments
 (0)