Using Chairmarks for CUDA benchmarks

[prabhuramachandran]

Thanks for a very handy and quick benchmarking tool!

This is probably more an issue with my lack of understanding than a bug but when benchmarking CUDA kernels, I often use @benchmark CUDA.@sync some_function($a1, $a2) but this will not quite work the same way with Chairmarks, it would be good to clarify the best approach to do this using Chairmarks as the naive translation will not do the right thing. Here is a small example.

import KernelAbstractions as KA
using CUDA

KA.@kernel function f(y, x, a, b)
    i = KA.@index(Global, Linear)
    y[i] = a[i]*x[i] + b[i]
end
n = 100000
x = CUDA.rand(n)
a = CUDA.rand(n);
b = CUDA.rand(n);
y = CUDA.fill(0.0, n)

func = f(KA.get_backend(x), 64)
@benchmark CUDA.@sync $func($y, $x, $a, $b, ndrange=size(y))

So how do I get chairmarks to call CUDA.@sync after every invocation of func? From the documentation, this is not what it would do if I just changed @benchmark to @be. I could create another function to do this but it would very nice if I could use chairmarks as a close-to-drop-in replacement for BenchmarkTools.

Thank you.

[LilithHafner]

I'd be happy to help, but can't get your "working" example to work. I get ERROR: CUDA driver not found. Do you have portable example of this (perhaps using constructs from Base or portable packages only)?

[prabhuramachandran]

Thank you for offering to help! I am not sure about a portable package version, the non-GPU cases I have do not require the sync call. CUDA is really orthogonal to the issue, the main problem being that in principle CUDA.@sync should call the function (func above) and then synchronize the GPU. @sync calls are not usually added to GPU kernels to avoid unnecessary synchronization, here is some documentation https://cuda.juliagpu.org/stable/development/profiling/ where they use BenckmarkTools.jl. I am not sure this helps. I will try to provide a simpler example tomorrow.

[prabhuramachandran]

I think I am mistaken, CUDA.@sync evaluates the function and performs the synchronization correctly. I tried a simpler example with a macro of my own and it does the "right thing". I was trying to simplify an example that did not work correctly in a much more complex case but my simple examples work. Sorry for the noise. I will submit a fresh issue if I find one with a reproducible example.