Skip to content

Commit e8b86f0

Browse files
notValordnamhyung
authored andcommitted
perf test stat_bpf_counter.sh: Stabilize the test results
The test has been failing for some time when two separate runs of perf benchmarks are recorded for cycles events and their counts are compared, while once the recording was done with option --bpf-counters and once without it. It is expected that the count of the samples should be within a certain range, firstly the difference was set to be within 10%, which was then later raised to 20%. However, the test case keeps failing on certain architectures as recording the provided benchmark can produce completely different counts based on the current load of the system. Sampling two separate runs on intel-eaglestream-spr-13 of "perf stat --no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t": Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t': 396782898 cycles 0.010051983 seconds time elapsed 0.008664000 seconds user 0.097058000 seconds sys Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t': 1431133032 cycles 0.021803714 seconds time elapsed 0.023377000 seconds user 0.349918000 seconds sys , which is ranging from 400mil to 1400mil samples. Instead of recording the cycles use instructions event, which provides more stable values. At the same time change the tested workload to one of the provided testing workloads by perf that is not based on a scheduler, which can provide another dependency on the current load. Sampling instructions event with the new workload provide much more stable results on intel-eaglestream-spr-13 of "perf stat --no-big-num -e instructions -- perf test -w brstack": Performance counter stats for 'perf test -w brstack': 6458449 instructions 0.009173945 seconds time elapsed 0.007262000 seconds user 0.002071000 seconds sys Performance counter stats for 'perf test -w brstack': 64672669 instructions 0.008888135 seconds time elapsed 0.005018000 seconds user 0.004018000 seconds sys Signed-off-by: Veronika Molnarova <[email protected]> Acked-by: Namhyung Kim <[email protected]> Cc: [email protected] Signed-off-by: Namhyung Kim <[email protected]> Link: https://lore.kernel.org/r/[email protected]
1 parent e4b19e2 commit e8b86f0

File tree

1 file changed

+18
-18
lines changed

1 file changed

+18
-18
lines changed

tools/perf/tests/shell/stat_bpf_counters.sh

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
set -e
66

7-
workload="perf bench sched messaging -g 1 -l 100 -t"
7+
workload="perf test -w brstack"
88

99
# check whether $2 is within +/- 20% of $1
1010
compare_number()
@@ -25,45 +25,45 @@ compare_number()
2525

2626
check_counts()
2727
{
28-
base_cycles=$1
29-
bpf_cycles=$2
28+
base_instructions=$1
29+
bpf_instructions=$2
3030

31-
if [ "$base_cycles" = "<not" ]; then
32-
echo "Skipping: cycles event not counted"
31+
if [ "$base_instructions" = "<not" ]; then
32+
echo "Skipping: instructions event not counted"
3333
exit 2
3434
fi
35-
if [ "$bpf_cycles" = "<not" ]; then
36-
echo "Failed: cycles not counted with --bpf-counters"
35+
if [ "$bpf_instructions" = "<not" ]; then
36+
echo "Failed: instructions not counted with --bpf-counters"
3737
exit 1
3838
fi
3939
}
4040

4141
test_bpf_counters()
4242
{
4343
printf "Testing --bpf-counters "
44-
base_cycles=$(perf stat --no-big-num -e cycles -- $workload 2>&1 | awk '/cycles/ {print $1}')
45-
bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- $workload 2>&1 | awk '/cycles/ {print $1}')
46-
check_counts $base_cycles $bpf_cycles
47-
compare_number $base_cycles $bpf_cycles
44+
base_instructions=$(perf stat --no-big-num -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
45+
bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
46+
check_counts $base_instructions $bpf_instructions
47+
compare_number $base_instructions $bpf_instructions
4848
echo "[Success]"
4949
}
5050

5151
test_bpf_modifier()
5252
{
5353
printf "Testing bpf event modifier "
54-
stat_output=$(perf stat --no-big-num -e cycles/name=base_cycles/,cycles/name=bpf_cycles/b -- $workload 2>&1)
55-
base_cycles=$(echo "$stat_output"| awk '/base_cycles/ {print $1}')
56-
bpf_cycles=$(echo "$stat_output"| awk '/bpf_cycles/ {print $1}')
57-
check_counts $base_cycles $bpf_cycles
58-
compare_number $base_cycles $bpf_cycles
54+
stat_output=$(perf stat --no-big-num -e instructions/name=base_instructions/,instructions/name=bpf_instructions/b -- $workload 2>&1)
55+
base_instructions=$(echo "$stat_output"| awk '/base_instructions/ {print $1}')
56+
bpf_instructions=$(echo "$stat_output"| awk '/bpf_instructions/ {print $1}')
57+
check_counts $base_instructions $bpf_instructions
58+
compare_number $base_instructions $bpf_instructions
5959
echo "[Success]"
6060
}
6161

6262
# skip if --bpf-counters is not supported
63-
if ! perf stat -e cycles --bpf-counters true > /dev/null 2>&1; then
63+
if ! perf stat -e instructions --bpf-counters true > /dev/null 2>&1; then
6464
if [ "$1" = "-v" ]; then
6565
echo "Skipping: --bpf-counters not supported"
66-
perf --no-pager stat -e cycles --bpf-counters true || true
66+
perf --no-pager stat -e instructions --bpf-counters true || true
6767
fi
6868
exit 2
6969
fi

0 commit comments

Comments
 (0)