Skip to content

Commit 13af820

Browse files
author
Treece Burgess
committed
Updating the PAPI GitHub CI, see README for more details on structure.
1 parent e378989 commit 13af820

26 files changed

+952
-133
lines changed

‎.github/workflows/README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
As of now, the GitHub CI is designed to run in three instances:
2+
1. An individual component basis, meaning if a component's codebase is updated, we will only run CI tests for that component. As an example, if we update `cupti_profiler.c` in `src/components/cuda`, we will only run CI tests for the cuda component. Note that this includes updates to subdirectories located in a component's directory (e.g. `src/components/cuda/tests`). See the **NOTE** in [Individual Component Basis](#individual-component-basis) for more info on the setup for the default components (`perf_event`, `perf_event_uncore`, and `sysdetect`).
3+
2. A change to the Counter Analysis Toolkit i.e. in the `src/counter_analysis_toolkit` directory and any subdirectories.
4+
3. A change in the PAPI framework i.e. in the `src/` directory or `src/` subdirectories (excluding individual components and the Counter Analysis Toolkit). If this occurs then we will run a full test suite.
5+
6+
7+
# Individual Component Basis
8+
All individual component basis tests have a `.yml` that is structured with `componentName_component_workflow.yml`. As
9+
an example for the `cuda` component we would have a `.yml` of `cuda_component_workflow.yml`. Therefore,
10+
if a new component is added to PAPI, you will need to create a `.yml` based on the aforementioned structure.
11+
12+
Upon creating the `.yml` file, you will need to add a workflow. Below is a skeleton that can
13+
be used as a starting point. As a reminder, make sure to change the necessary fields out for your component.
14+
15+
```yml
16+
name: cuda # replace cuda with your component name
17+
18+
on:
19+
pull_request:
20+
paths:
21+
- 'src/components/cuda/**' # replace the cuda path with your component path
22+
23+
jobs:
24+
component_tests:
25+
strategy:
26+
matrix:
27+
component: [cuda] # replace cuda with your component name
28+
debug: [yes, no]
29+
shlib: [with, without]
30+
fail-fast: false
31+
runs-on: [self-hosted, nvidia_gpu]
32+
timeout-minutes: 60
33+
steps:
34+
- uses: actions/checkout@v4
35+
- name: cuda component tests # replace cuda with your component name
36+
run: .github/workflows/ci_individual_component.sh ${{matrix.component}} ${{matrix.debug}} ${{matrix.shlib}}
37+
````
38+
39+
For each component `.yml`, there will be a single job with the options of:
40+
- `component`, this is the component we want to configure PAPI with (e.g. cuda)
41+
- `debug`, with the options of either `yes` (builds a debug version) or `no` (does not build a debug version)
42+
- `shlib`, with `--with-shlib-tools` or without `--with-shlib-tools`
43+
44+
These options will be used in the script `ci_individual_component.sh` to test configuring and building PAPI.
45+
46+
Besides configuring and building PAPI, `ci_individual_component.sh` will:
47+
- Check to make sure components are active that we expect
48+
- Run a test suite using either `run_tests.sh` (without `--with-shlib-tools`) or `run_tests_shlib.sh` (with `--with-shlib-tools`)
49+
50+
**NOTE**: The components `perf_event`, `perf_event_uncore`, and `sysdetect` do not follow the above outlined file structure.
51+
For these three components the files used are `default_components_workflow.yml` and `ci_default_components.sh`. Even though
52+
the file structure is different the workflow will still only run if a change is made to one of their directories or subdirectories.
53+
The reason for this is that these three components are compiled in by default and trying to pass one of them to `--with-components=`
54+
will result in an error during the build process. Therefore, any PAPI CI updates for one of these three components would need to be
55+
addressed in either of the two aforementioned files.
56+
57+
# Counter Analysis Toolkit
58+
The Counter Analysis Toolkit (CAT) CI uses the `cat_workflow.yml` and `ci_cat.sh` files. Any updates to the CI for CAT need to be done in these two files.
59+
60+
The `cat_workflow.yml` will have a single job with the options of:
61+
- `debug`, with the options of either `yes` (builds a debug version) or `no` (does not build a debug version)
62+
- `shlib`, with `--with-shlib-tools` or without `--with-shlib-tools`
63+
64+
These options will be used in the script `ci_cat.sh` to test configuring and building PAPI.
65+
66+
Besides configuring and building PAPI `ci_cat.sh` will:
67+
- Test building CAT
68+
- Check to see if CAT successfully detects the architecture we are on
69+
- Run CAT with a real event and a dummy event
70+
- For the real event, we expect the file to exist and values to be present
71+
- For the dummy event, we expect the file to exist, but values to not be present
72+
73+
# PAPI Framework
74+
The PAPI framework CI uses the `papi_framework_workflow.yml` along with the scripts `clang_analysis.sh`, `ci_papi_framework.sh`, and `spack.sh`. Any updates to the CI for the PAPI framework need to be done in these files.
75+
76+
`papi_framework_workflow.yml` will have a total of five jobs:
77+
1. papi_components_comprehensive
78+
- With the options of:
79+
- `components`, this is the components we want to configure PAPI with, i.e. `cuda nvml rocm rocm_smi powercap powercap_ppc rapl sensors_ppc infiniband net appio io lustre stealtime`
80+
- `debug`, with the options of either `yes` (builds a debug version) or `no` (does not build a debug version)
81+
- `shlib`, with `--with-shlib-tools` or without `--with-shlib-tools`
82+
83+
2. papi_components_amd
84+
- With the options of:
85+
- `components`, this is the components we want to configure PAPI with, i.e. `rocm rocm_smi`
86+
- `debug`, with the options of either `yes` (builds a debug version) or `no` (does not build a debug version)
87+
- `shlib`, with `--with-shlib-tools` or without `--with-shlib-tools`
88+
89+
3. papi_component_infiniband
90+
- With the options of:
91+
- `component`, this is the component we want to configure PAPI with, i.e. `infiniband`
92+
- `debug`, with the options of either `yes` (builds a debug version) or `no` (does not build a debug version)
93+
- `shlib`, with `--with-shlib-tools` or without `--with-shlib-tools`
94+
95+
4. papi_spack
96+
- The script `spack.sh` will be ran, which configures and builds PAPI from SPACK
97+
98+
5. papi_clang_analysis
99+
- The script `clang_analysis.sh` will be ran, which configures and builds PAPI with clang
100+
101+
For jobs 1, 2, and 3, the options listed will be used in the script `ci_papi_framework.sh` to test configuring and building PAPI.
102+
103+
Besides configuring and building PAPI, `ci_papi_framework.sh` will:
104+
- Check to make sure components are active that we expect
105+
- Run a test suite using either `run_tests.sh` (without `--with-shlib-tools`) or `run_tests_shlib.sh` (with `--with-shlib-tools`)

‎.github/workflows/appio_component_workflow.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: appio
2+
3+
on:
4+
pull_request:
5+
# run CI only if appio directory or appio sub-directories receive updates
6+
paths:
7+
- 'src/components/appio/**'
8+
9+
# allows you to run this workflow manually from the Actions tab
10+
workflow_dispatch:
11+
12+
jobs:
13+
component_tests:
14+
strategy:
15+
matrix:
16+
component: [appio]
17+
debug: [yes, no]
18+
shlib: [with, without]
19+
fail-fast: false
20+
runs-on: [self-hosted, cpu_intel]
21+
timeout-minutes: 60
22+
steps:
23+
- uses: actions/checkout@v4
24+
- name: appio component tests
25+
run: .github/workflows/ci_individual_component.sh ${{matrix.component}} ${{matrix.debug}} ${{matrix.shlib}}

‎.github/workflows/cat_workflow.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: counter analysis toolkit
2+
3+
on:
4+
pull_request:
5+
# run CI for updates to counter analysis toolkit
6+
paths:
7+
- 'src/counter_analysis_toolkit/**'
8+
# allows you to run this workflow manually from the Actions tab
9+
workflow_dispatch:
10+
11+
jobs:
12+
cat tests:
13+
strategy:
14+
matrix:
15+
debug: [yes, no]
16+
shlib: [with, without]
17+
fail-fast: false
18+
runs-on: [self-hosted, cpu_intel]
19+
timeout-minutes: 60
20+
steps:
21+
- uses: actions/checkout@v4
22+
- name: counter analysis toolkit tests
23+
run: .github/workflows/ci_cat.sh ${{matrix.debug}} ${{matrix.shlib}}

‎.github/workflows/ci.sh‎

Lines changed: 0 additions & 62 deletions
This file was deleted.

‎.github/workflows/ci_cat.sh‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/bin/bash -e
2+
3+
DEBUG=$1
4+
SHLIB=$2
5+
COMPILER=$3
6+
7+
[ -z "$COMPILER" ] && COMPILER=gcc@11
8+
9+
source /etc/profile
10+
set +x
11+
set -e
12+
trap 'echo "# $BASH_COMMAND"' DEBUG
13+
shopt -s expand_aliases
14+
15+
module load $COMPILER
16+
17+
cd src
18+
19+
# configuring and installing PAPI
20+
if [ "$SHLIB" = "without" ]; then
21+
./configure --prefix=$PWD/cat-ci --with-debug=$DEBUG --enable-warnings
22+
else
23+
./configure --prefix=$PWD/cat-ci --with-debug=$DEBUG --enable-warnings --with-shlib-tools
24+
fi
25+
make -j4 && make install
26+
27+
# set environment variables for CAT
28+
export PAPI_DIR=$PWD/cat-ci
29+
export LD_LIBRARY_PATH=${PAPI_DIR}/lib:$LD_LIBRARY_PATH
30+
cd counter_analysis_toolkit
31+
32+
# check detected architecture was correct
33+
# note that the make here will finish
34+
DETECTED_ARCH=$(make | grep -o 'ARCH.*' | head -n 1)
35+
if [ "$DETECTED_ARCH" != "ARCH=X86" ]; then
36+
echo "Failed to detect appropriate architecture."
37+
exit 1
38+
fi
39+
40+
# create output directory
41+
mkdir OUT_DIR
42+
# create real and fake events to monitor
43+
echo "BR_INST_RETIRED 0" > event_list.txt
44+
echo "PAPI_CI_FAKE_EVENT 0" >> event_list.txt
45+
./cat_collect -in event_list.txt -out OUT_DIR -branch
46+
47+
cd OUT_DIR
48+
# we expect this file to exist and have values
49+
[ -f BR_INST_RETIRED.branch ]
50+
[ -s BR_INST_RETIRED.branch ]
51+
# we expect this file to exist but be empty
52+
[ -f PAPI_CI_FAKE_EVENT.branch ]
53+
[ ! -s PAPI_CI_FAKE_EVENT.branch ]

‎.github/workflows/ci_default_components.sh‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/bin/bash -e
2+
3+
DEBUG=$1
4+
SHLIB=$2
5+
COMPILER=$3
6+
7+
[ -z "$COMPILER" ] && COMPILER=gcc@11
8+
9+
source /etc/profile
10+
set +x
11+
set -e
12+
trap 'echo "# $BASH_COMMAND"' DEBUG
13+
shopt -s expand_aliases
14+
15+
module load $COMPILER
16+
17+
cd src
18+
19+
# test linking with or without --with-shlib-tools
20+
if [ "$SHLIB" = "without" ]; then
21+
./configure --with-debug=$DEBUG --enable-warnings
22+
else
23+
./configure --with-debug=$DEBUG --enable-warnings --with-shlib-tools
24+
fi
25+
26+
make -j4
27+
28+
# run PAPI utilities
29+
utils/papi_component_avail
30+
31+
# active component check
32+
EXPECTED_ACTIVE_COMPONENTS="perf_event perf_event_uncore sysdetect"
33+
CURRENT_ACTIVE_COMPONENTS=$(utils/papi_component_avail | grep -A1000 'Active components' | grep "Name: " | awk '{printf "%s%s", sep, $2; sep=" "} END{print ""}')
34+
[ "$EXPECTED_ACTIVE_COMPONENTS" = "$CURRENT_ACTIVE_COMPONENTS" ]
35+
36+
# without '--with-shlib-tools' in ./configure
37+
if [ "$SHLIB" = "without" ]; then
38+
echo "Running full test suite for active components"
39+
./run_tests.sh TESTS_QUIET
40+
# with '--with-shlib-tools' in ./configure
41+
else
42+
echo "Running single component test for active components"
43+
./run_tests_shlib.sh TESTS_QUIET
44+
fi

‎.github/workflows/ci_individual_component.sh‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#!/bin/bash -e
2+
3+
COMPONENT=$1
4+
DEBUG=$2
5+
SHLIB=$3
6+
COMPILER=$4
7+
8+
[ -z "$COMPILER" ] && COMPILER=gcc@11
9+
10+
source /etc/profile
11+
set +x
12+
set -e
13+
trap 'echo "# $BASH_COMMAND"' DEBUG
14+
shopt -s expand_aliases
15+
16+
module load $COMPILER
17+
18+
cd src
19+
20+
# lmsensors environment variables
21+
if [ "$COMPONENT" = "lmsensors" ]; then
22+
wget https://github.com/groeck/lm-sensors/archive/V3-4-0.tar.gz
23+
tar -zxf V3-4-0.tar.gz
24+
cd lm-sensors-3-4-0
25+
make install PREFIX=../lm ETCDIR=../lm/etc
26+
cd ..
27+
export PAPI_LMSENSORS_ROOT=lm
28+
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PAPI_LMSENSORS_ROOT/lib
29+
fi
30+
31+
# rocm and rocm_smi environment variables
32+
if [ "$COMPONENT" = "rocm" ] || [ "$COMPONENT" = "rocm_smi" ]; then
33+
export PAPI_ROCM_ROOT=/apps/rocm/rocm-5.5.3
34+
export PAPI_ROCMSMI_ROOT=$PAPI_ROCM_ROOT/rocm_smi
35+
fi
36+
37+
# set necessary environemnt variables for cuda and nvml
38+
if [ "$COMPONENT" = "cuda" ] || [ "$COMPONENT" = "nvml" ]; then
39+
module load cuda
40+
export PAPI_CUDA_ROOT=$ICL_CUDA_ROOT
41+
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PAPI_CUDA_ROOT/extras/CUPTI/lib64
42+
fi
43+
44+
# test linking with or without --with-shlib-tools
45+
if [ "$SHLIB" = "without" ]; then
46+
./configure --with-debug=$DEBUG --enable-warnings --with-components="$COMPONENT"
47+
else
48+
./configure --with-debug=$DEBUG --enable-warnings --with-components="$COMPONENT" --with-shlib-tools
49+
fi
50+
51+
make -j4
52+
53+
# run PAPI utilities
54+
utils/papi_component_avail
55+
56+
# active component check
57+
EXPECTED_ACTIVE_COMPONENTS=$(echo "perf_event perf_event_uncore sysdetect" | sed "s/perf_event_uncore/& $COMPONENT/")
58+
CURRENT_ACTIVE_COMPONENTS=$(utils/papi_component_avail | grep -A1000 'Active components' | grep "Name: " | awk '{printf "%s%s", sep, $2; sep=" "} END{print ""}')
59+
[ "$EXPECTED_ACTIVE_COMPONENTS" = "$CURRENT_ACTIVE_COMPONENTS" ]
60+
61+
# without '--with-shlib-tools' in ./configure
62+
if [ "$SHLIB" = "without" ]; then
63+
echo "Running full test suite for active components"
64+
./run_tests.sh TESTS_QUIET --disable-cuda-events=yes
65+
# with '--with-shlib-tools' in ./configure
66+
else
67+
echo "Running single component test for active components"
68+
./run_tests_shlib.sh TESTS_QUIET
69+
fi

0 commit comments

Comments
 (0)