Skip to content

Commit f8528a5

Browse files
committed
check: report number of failed test cases and skipped test cases
As a part of the Blktests CI preparation, improve the test result reporting. Record failed test cases in the file named "failures" in the results output directory. Print the count and the list of failed test cases at the end of test runs. Do the same for skipped test cases using the file named "skips". The output will be like this: $ sudo ./check meta meta/001 (do nothing) [passed] runtime 0.000s ... 0.000s [...] meta/019 (x=1 y=1 z=1) (combine three set_conditions() hooks) [passed] runtime 0.000s ... 0.000s 4 failures: meta/003 meta/005 meta/006 meta/009 2 skips: meta/007 meta/014 Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
1 parent 69591d9 commit f8528a5

File tree

1 file changed

+17
-2
lines changed

1 file changed

+17
-2
lines changed

check

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -452,10 +452,14 @@ _call_test() {
452452
}" "${seqres}.dmesg"
453453
;;
454454
esac
455+
456+
echo "$TEST_NAME" >> "$FAILURES"
455457
return 1
456-
else
457-
return 0
458+
elif [[ ${TEST_RUN["status"]} = 'not run' ]]; then
459+
echo "$TEST_NAME" >> "$SKIPS"
458460
fi
461+
462+
return 0
459463
}
460464

461465
_test_dev_is_zoned() {
@@ -1040,5 +1044,16 @@ fi
10401044
10411045
mkdir -p "$OUTPUT"
10421046
OUTPUT="$(realpath "$OUTPUT")"
1047+
FAILURES="$OUTPUT/failures"
1048+
SKIPS="$OUTPUT/skips"
1049+
rm -f "$FAILURES" "$SKIPS"
10431050
10441051
_check "$@"
1052+
1053+
declare -a failures skips
1054+
declare l
1055+
[[ -f $FAILURES ]] && while read -r l; do failures+=("$l"); done < "$FAILURES"
1056+
((${#failures[@]})) && echo "${#failures[@]} failures: ${failures[*]}"
1057+
1058+
[[ -f $SKIPS ]] && while read -r l; do skips+=("$l"); done < "$SKIPS"
1059+
((${#skips[@]})) && echo "${#skips[@]} skips: ${skips[*]}"

0 commit comments

Comments
 (0)