Skip to content

Commit a8ff1f0

Browse files
Update nvbench and document how to plot results/diffs (#7873)
Fixes: #7828
1 parent 2d40fb8 commit a8ff1f0

File tree

2 files changed

+26
-1
lines changed

2 files changed

+26
-1
lines changed

cmake/CCCLGetDependencies.cmake

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ endmacro()
8282

8383
set(
8484
CCCL_NVBENCH_SHA
85-
"34f1e2a7eeeeec7bb2965a3ec9667d297b479cfe"
85+
"728212f9f162d1e598c21cc0fff456eb999b971a"
8686
CACHE STRING
8787
"SHA/tag to use for CCCL's NVBench."
8888
)

docs/cub/benchmarking.rst

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,22 @@ See the `NVBench documentation <https://github.com/NVIDIA/nvbench/blob/main/docs
108108
for more information on how to specify the axis values.
109109
If the specified axis does not exist, the benchmark will terminate with an error.
110110

111+
If you want to plot the benchmark results, you can use the following script:
112+
113+
.. code-block:: bash
114+
115+
PYTHONPATH=./_deps/nvbench-src/scripts ./_deps/nvbench-src/scripts/nvbench_plot_bwutil.py base.json
116+
117+
The `-a` option is supported to restrict the values for some axes as well,
118+
which is useful if you want to plot only a subset of workloads.
119+
Use the `-b` option to select a specific benchmark by name
120+
in case your JSON file contains results for multiple benchmarks.
121+
Multiple benchmarks are selected by repeating the `-b` option.
122+
123+
.. code-block:: bash
124+
125+
PYTHONPATH=./_deps/nvbench-src/scripts ./_deps/nvbench-src/scripts/nvbench_plot_bwutil.py \
126+
-b base -a Elements{io}[pow2]=28 base.json
111127
112128
.. _cub-benchmarking-comparing:
113129

@@ -144,6 +160,15 @@ which corresponds to the relative standard deviation.
144160
It then reports with statistical significance in the `Status` column
145161
how the runtime changed from the base to the new version.
146162

163+
You can also plot the comparison by adding the `--plot` argument.
164+
It's reasonable to combine this with the `-a` option again
165+
to restrict the values for some axes.
166+
167+
.. code-block:: bash
168+
169+
PYTHONPATH=./_deps/nvbench-src/scripts ./_deps/nvbench-src/scripts/nvbench_compare.py \
170+
-a Elements{io}[pow2]=28 --plot base.json new.json
171+
147172
148173
Running all benchmarks directly from the command line
149174
--------------------------------------------------------------------------------

0 commit comments

Comments
 (0)