Hi,

while working with ib_write_bw, I noticed that when using the software rate limiter for high bandwidths, the throughput of the generated traffic is, on average, slightly lower than it should be. It also has noticeable variance across runs.

Tests used the following parameters: --rate_limit X --rate_limit_type SW -D 2 --burst_size 1, where the rate limit X is varied between 1 and 80 Gbit/s. Tests used the code from commit 799923f (Wed Oct 22 09:42:45 2025 +0300), but the relevant code of the run_iter_bw function has not changed since, so this issue is almost certainly still present.

Below is a plot showing measured throughputs over the desired throughput given to the software rate limiter, averaged over 10 runs. Throughput was measured using InfiniBand Port Counters, and the shown values here are adjusted for RoCE packet overhead, such that the ideal result exactly matches the diagonal.

From the plot, it is visible that the unmodified perftest binary (labeled perftest_cr) generates traffic with a throughput slightly below the ideal diagonal. It also shows noticeable variance across runs. On the other hand, after applying the patch below to the application (labeled perftestmod_cr), the line is a perfect diagonal with no visible variance.

The patch involves simply replacing gap_deadline = get_cycles() + gap_cycles; with gap_deadline = gap_deadline + gap_cycles; in perftest_resources.c:4076 (latest revision).

Have a nice day!