Skip to content

Improve rngtest CI reliability with output-based validation and success ratio logging #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 9, 2025
Merged

Improve rngtest CI reliability with output-based validation and success ratio logging #109

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 18 additions & 27 deletions Runner/suites/Kernel/FunctionalArea/baseport/rngtest/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,60 +36,51 @@ log_info "----------------------------------------------------------------------
log_info "-------------------Starting $TESTNAME Testcase----------------------------"
log_info "=== Test Initialization ==="

log_info "Checking if dependency binary is available"
# Verifying the availability of the dependency binary
check_dependencies rngtest dd

TMP_BIN="/tmp/rngtest_input.bin"
TMP_OUT="/tmp/rngtest_output.txt"
ENTROPY_MB=10
RNG_SOURCE="/dev/urandom" # Use /dev/random if you want slow but highest entropy
COUNT=1000
PASS_THRESHOLD=997
RNG_SOURCE="/dev/urandom"
[ -e /dev/hwrng ] && RNG_SOURCE="/dev/hwrng"

log_info "Generating ${ENTROPY_MB}MB entropy input from $RNG_SOURCE using dd..."
log_info "Generating ${ENTROPY_MB}MB entropy input from $RNG_SOURCE"
if ! dd if="$RNG_SOURCE" of="$TMP_BIN" bs=1M count="$ENTROPY_MB" status=none 2>/dev/null; then
log_fail "$TESTNAME : Failed to read random data from $RNG_SOURCE"
echo "$TESTNAME FAIL" > "$res_file"
rm -f "$TMP_BIN"
exit 1
fi

log_info "Running rngtest -c 1000 < $TMP_BIN"
if ! rngtest -c 1000 < "$TMP_BIN" > "$TMP_OUT" 2>&1; then
log_fail "$TESTNAME : rngtest execution failed"
echo "$TESTNAME FAIL" > "$res_file"
rm -f "$TMP_BIN" "$TMP_OUT"
exit 1
fi
log_info "Running rngtest -c $COUNT < $TMP_BIN"
rngtest -c "$COUNT" < "$TMP_BIN" > "$TMP_OUT" 2>&1

# Check for entropy errors or source drained
if grep -q "entropy source drained" "$TMP_OUT"; then
log_fail "rngtest: entropy source drained, input too small"
echo "$TESTNAME FAIL" > "$res_file"
rm -f "$TMP_BIN" "$TMP_OUT"
exit 1
fi

# Parse FIPS 140-2 successes (robust to output variations)
# Try to extract success count regardless of return code
successes=$(awk '/FIPS 140-2 successes:/ {print $NF}' "$TMP_OUT" | head -n1)

if [ -z "$successes" ] || ! echo "$successes" | grep -Eq '^[0-9]+$'; then
log_fail "rngtest did not return a valid integer for successes; got: '$successes'"
log_fail "rngtest: Could not parse valid success count from output"
echo "$TESTNAME FAIL" > "$res_file"
cat "$TMP_OUT"
rm -f "$TMP_BIN" "$TMP_OUT"
exit 1
fi

log_info "rngtest: FIPS 140-2 successes = $successes"
# You can tune this threshold as needed (10 means <1% fail allowed)
if [ "$successes" -ge 10 ]; then
log_pass "$TESTNAME : Test Passed ($successes FIPS 140-2 successes)"
log_info "FIPS 140-2 successes: $successes / $COUNT"
percent=$(awk "BEGIN {printf \"%.2f\", ($successes/$COUNT)*100}")
log_info "Success ratio: $percent%"

if [ "$successes" -ge "$PASS_THRESHOLD" ]; then
log_pass "$TESTNAME : Test Passed ($successes ≥ $PASS_THRESHOLD successes)"
echo "$TESTNAME PASS" > "$res_file"
rm -f "$TMP_BIN" "$TMP_OUT"
exit 0
else
log_fail "$TESTNAME : Test Failed ($successes FIPS 140-2 successes)"
log_fail "$TESTNAME : Test Failed ($successes < $PASS_THRESHOLD successes)"
echo "$TESTNAME FAIL" > "$res_file"
rm -f "$TMP_BIN" "$TMP_OUT"
exit 1
fi

log_info "-------------------Completed $TESTNAME Testcase----------------------------"
Loading