Skip to content

Commit bf03d25

Browse files
committed
Kselftest: Add enabled_tests.list and run.sh directory in selftest runner
- Added (or improved) validation to ensure that the enabled_tests.list whitelist file is present and non-empty before executing kernel selftests. - Script now also checks for the presence of the /kselftest directory and fails early with a clear error if missing. - This prevents silent failures and improves CI robustness by guaranteeing that test configura^Con and dependencies are always validated before running. - Logs explicit error messages for missing or empty enabled_tests.list and for missing /kselftest. - Follows community best practices for test automation, as seen in major kernel CI flows. Signed-off-by: Srikanth Muppandam <[email protected]>
1 parent 8ba9498 commit bf03d25

File tree

4 files changed

+298
-4
lines changed

4 files changed

+298
-4
lines changed
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Kernel Selftest Runner
2+
3+
This directory contains the automation scripts for running **Linux Kernel Selftests** on Qualcomm-based platforms (e.g., RB3GEN2) using the standardized `qcom-linux-testkit` framework.
4+
5+
## Overview
6+
7+
The selftest runner (`run.sh`) is designed to:
8+
- Automatically detect and run enabled kernel selftests using a whitelist file (`enabled_tests.list`)
9+
- Ensure all required dependencies and directories are present
10+
- Log detailed test results for CI/CD integration
11+
- Support both standalone execution and integration with LAVA/automated test infrastructure
12+
13+
## Files
14+
15+
- **run.sh**: Main test runner script. Executes all tests listed in `enabled_tests.list` and produces `.res` and `.run` result files.
16+
- **enabled_tests.list**: Whitelist of test suites or binaries to run (one per line). Supports comments and blank lines.
17+
- **Kernel_Selftests.res**: Detailed result file, capturing PASS/FAIL/SKIP for each test or subtest.
18+
- **Kernel_Selftests.run**: Cumulative PASS or FAIL summary, used by CI pipelines for quick parsing.
19+
20+
## Prerequisites
21+
22+
- `/kselftest` directory must be present on the device/target. This is where the selftest binaries are located (usually deployed by the build or CI).
23+
- All test scripts must have the executable bit set (`chmod +x run.sh`), enforced by GitHub Actions.
24+
- `enabled_tests.list` must exist and contain at least one non-comment, non-blank entry.
25+
26+
## Script Flow
27+
28+
1. **Environment Setup**:
29+
Dynamically locates and sources `init_env` and `functestlib.sh` to ensure robust, path-independent execution.
30+
31+
2. **Dependency Checks**:
32+
- Checks for required commands (e.g., `find`)
33+
- Ensures `/kselftest` directory exists
34+
- Validates `enabled_tests.list` is present and usable
35+
36+
3. **Test Discovery & Execution**:
37+
- Parses `enabled_tests.list`, ignoring comments and blanks
38+
- Supports both directory and subtest (e.g., `timers/thread_test`) entries
39+
- Executes test binaries or `run_test.sh` for each listed test
40+
- Logs individual and summary results
41+
42+
4. **Result Logging**:
43+
- Writes detailed results to `Kernel_Selftests.res`
44+
- Writes overall PASS/FAIL to `Kernel_Selftests.run`
45+
46+
5. **CI Compatibility**:
47+
- Designed for direct invocation by LAVA or any CI/CD harness
48+
- Fails early and logs meaningful errors if prerequisites are missing
49+
50+
## enabled_tests.list Format
51+
52+
```text
53+
# This is a comment
54+
timers
55+
thread
56+
timers/thread_test
57+
# Add more test names or subtests as needed
58+
```
59+
60+
- Each non-comment, non-blank line specifies a test directory or a test binary under `/kselftest`.
61+
62+
## Example Usage
63+
64+
```sh
65+
sh run.sh
66+
```
67+
68+
or from a higher-level testkit runner:
69+
70+
```sh
71+
./run-test.sh Kernel_Selftests
72+
```
73+
74+
## Troubleshooting
75+
76+
- **Missing executable bit**:
77+
If you see permission errors, ensure all scripts are `chmod +x`.
78+
79+
- **Missing `/kselftest` directory**:
80+
Ensure selftests are built and deployed to the target system.
81+
82+
- **Missing or empty `enabled_tests.list`**:
83+
Add test entries as needed. The runner will fail if this file is absent or empty.
84+
85+
## Contribution Guidelines
86+
87+
- Update `enabled_tests.list` as test coverage expands.
88+
- Follow the coding and structure conventions enforced by CI (see `CONTRIBUTING.md`).
89+
- All changes should pass permission checks and shellcheck lints in CI.
90+
91+
## License
92+
93+
SPDX-License-Identifier: BSD-3-Clause-Clear
94+
Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# enabled_tests.list for ARM64 selftests (mixed folder and binary)
2+
# Example: run only thread_test and settime_test in timers/
3+
# The line 'timers' will run all standard tests in timers/
4+
# The lines 'timers/thread_test' and 'timers/settime_test' run these binaries explicitly, even if not covered by timers/ directory run
5+
breakpoints
6+
capabilities
7+
cgroup
8+
clone3
9+
core
10+
timers
11+
timers/thread_test
12+
timers/settime_test
13+
futex
14+
kcmp
15+
net/nettest
16+
pidfd
17+
proc
18+
seccomp
19+
user
20+
vm
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
#!/bin/sh
2+
#
3+
# Kernel Selftests Validation Script for ARM64 (RB3GEN2)
4+
#
5+
# Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
6+
# SPDX-License-Identifier: BSD-3-Clause-Clear
7+
8+
# Dynamically locate and source init_env (robust for any repo structure)
9+
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
10+
INIT_ENV=""
11+
SEARCH="$SCRIPT_DIR"
12+
while [ "$SEARCH" != "/" ]; do
13+
if [ -f "$SEARCH/init_env" ]; then
14+
INIT_ENV="$SEARCH/init_env"
15+
break
16+
fi
17+
SEARCH=$(dirname "$SEARCH")
18+
done
19+
20+
if [ -z "$INIT_ENV" ]; then
21+
echo "[ERROR] Could not find init_env (starting at $SCRIPT_DIR)" >&2
22+
exit 1
23+
fi
24+
25+
# Only source if not already loaded (idempotent)
26+
if [ -z "$__INIT_ENV_LOADED" ]; then
27+
# shellcheck disable=SC1090
28+
. "$INIT_ENV"
29+
fi
30+
31+
# Always source functestlib.sh, using $TOOLS exported by init_env
32+
# shellcheck disable=SC1090
33+
. "$TOOLS/functestlib.sh"
34+
35+
TESTNAME="Kernel_Selftests"
36+
test_path=$(find_test_case_by_name "$TESTNAME")
37+
if [ -z "$test_path" ] || [ ! -d "$test_path" ]; then
38+
log_fail "$TESTNAME : Test directory not found."
39+
echo "FAIL $TESTNAME" > "./$TESTNAME.run"
40+
exit 1
41+
fi
42+
43+
cd "$test_path" || exit 1
44+
45+
res_file="./$TESTNAME.res"
46+
summary_file="./$TESTNAME.run"
47+
whitelist="./enabled_tests.list"
48+
selftest_dir="/kselftest"
49+
arch="$(uname -m)"
50+
skip_arch="x86 powerpc mips sparc"
51+
52+
pass=0
53+
fail=0
54+
skip=0
55+
56+
rm -f "$res_file" "$summary_file"
57+
echo "SUITE: $TESTNAME" > "$res_file"
58+
59+
log_info "Starting $TESTNAME..."
60+
61+
check_dependencies "find" "/kselftest"
62+
63+
if [ ! -f "$whitelist" ]; then
64+
log_fail "$TESTNAME: whitelist $whitelist not found"
65+
echo "FAIL $TESTNAME" > "$summary_file"
66+
exit 1
67+
fi
68+
69+
while IFS= read -r test || [ -n "$test" ]; do
70+
case "$test" in
71+
''|\#*) continue ;; # Skip blanks and comments
72+
esac
73+
74+
# If test is architecture-specific, skip if not supported
75+
for a in $skip_arch; do
76+
if [ "$test" = "$a" ]; then
77+
log_skip "$test skipped on $arch"
78+
echo "SKIP $test (unsupported arch)" >> "$res_file"
79+
skip=$((skip+1))
80+
continue 2
81+
fi
82+
done
83+
84+
# Check for directory/binary (e.g., timers/thread_test)
85+
case "$test" in
86+
*/*)
87+
test_dir="${test%%/*}"
88+
test_bin="${test#*/}"
89+
bin_path="$selftest_dir/$test_dir/$test_bin"
90+
if [ ! -d "$selftest_dir/$test_dir" ]; then
91+
log_skip "$test_dir not found"
92+
echo "SKIP $test (directory not found)" >> "$res_file"
93+
skip=$((skip+1))
94+
continue
95+
fi
96+
if [ -x "$bin_path" ]; then
97+
log_info "Running $test_dir/$test_bin"
98+
if timeout 300 "$bin_path" > "${test_dir}_${test_bin}.log" 2>&1; then
99+
log_pass "$test: passed"
100+
echo "PASS $test" >> "$res_file"
101+
pass=$((pass+1))
102+
else
103+
log_fail "$test: failed"
104+
echo "FAIL $test" >> "$res_file"
105+
fail=$((fail+1))
106+
fi
107+
else
108+
log_skip "$test: not found or not executable"
109+
echo "SKIP $test (not found or not executable)" >> "$res_file"
110+
skip=$((skip+1))
111+
fi
112+
continue
113+
;;
114+
esac
115+
116+
# Standard: just the directory (run run_test.sh or all *test bins)
117+
test_dir="$selftest_dir/$test"
118+
if [ ! -d "$test_dir" ]; then
119+
log_skip "$test not found"
120+
echo "SKIP $test (not found)" >> "$res_file"
121+
skip=$((skip+1))
122+
continue
123+
fi
124+
125+
if [ -x "$test_dir/run_test.sh" ]; then
126+
log_info "Running $test/run_test.sh"
127+
if timeout 300 "$test_dir/run_test.sh" > "$test.log" 2>&1; then
128+
log_pass "$test passed"
129+
echo "PASS $test" >> "$res_file"
130+
pass=$((pass+1))
131+
else
132+
log_fail "$test failed"
133+
echo "FAIL $test" >> "$res_file"
134+
fail=$((fail+1))
135+
fi
136+
else
137+
found_bin=0
138+
for bin in "$test_dir"/*test; do
139+
[ -f "$bin" ] && [ -x "$bin" ] || continue
140+
found_bin=1
141+
binname=$(basename "$bin")
142+
log_info "Running $test/$binname"
143+
if timeout 300 "$bin" > "${test}_${binname}.log" 2>&1; then
144+
log_pass "$test/$binname passed"
145+
echo "PASS $test/$binname" >> "$res_file"
146+
pass=$((pass+1))
147+
else
148+
log_fail "$test/$binname failed"
149+
echo "FAIL $test/$binname" >> "$res_file"
150+
fail=$((fail+1))
151+
fi
152+
done
153+
if [ "$found_bin" -eq 0 ]; then
154+
log_skip "$test: no test binaries"
155+
echo "SKIP $test (no test binaries)" >> "$res_file"
156+
skip=$((skip+1))
157+
fi
158+
fi
159+
done < "$whitelist"
160+
161+
echo "SUMMARY PASS=$pass FAIL=$fail SKIP=$skip" >> "$res_file"
162+
163+
if [ "$fail" -eq 0 ] && [ "$pass" -gt 0 ]; then
164+
echo "PASS $TESTNAME" > "$summary_file"
165+
log_pass "$TESTNAME: all tests passed"
166+
else
167+
echo "FAIL $TESTNAME" > "$summary_file"
168+
log_fail "$TESTNAME: one or more tests failed"
169+
fi
170+
171+
exit 0

Runner/utils/functestlib.sh

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,22 @@ log_skip() { log "SKIP" "$@"; }
1818
# --- Dependency check ---
1919
check_dependencies() {
2020
missing=0
21-
for cmd in "$@"; do
22-
if ! command -v "$cmd" >/dev/null 2>&1; then
23-
log_error "Required command '$cmd' not found in PATH."
21+
for item in "$@"; do
22+
if command -v "$item" >/dev/null 2>&1; then
23+
continue
24+
elif [ -d "$item" ]; then
25+
continue
26+
else
27+
log_error "ERROR: Required command or directory '$item' not found."
2428
missing=1
2529
fi
2630
done
27-
[ "$missing" -ne 0 ] && exit 1
31+
if [ "$missing" -ne 0 ]; then
32+
log_error "Exiting due to missing dependencies."
33+
exit 1
34+
else
35+
log_pass "All dependencies are present."
36+
fi
2837
}
2938

3039
# --- Test case directory lookup ---

0 commit comments

Comments
 (0)