Performance of integrate2d()

[JamesWrigley]

Hey there 👋 @twguest and I are trying to use pyfai to do 2D integration with integrate2d() per-frame on our detector, and we noticed that it seems to be significantly slower than integrate1d(). Unfortunately I don't have a MWE handy but I remember it was something like ~300ms vs ~5ms or less compared to integrate1d(), no matter which method we used.

I'll try to come up with an MWE in a bit, but do you know off the top of your head if this sort of difference is to be expected?

[kif]

Hi,
I am not really surprised by your figures, integrate2d is making a lot more calculation in comparison to integrate1d. It is also dependent on the number of azimuthal bins ... and on the integration method which depend on the pixel splitting, the implementation and on the algorithm. I can try to come up with a small notebook with compares the figures for integrate1d vs integrate2d and the 60x difference you observe does not look unreasonable.
Cheers,
Jerome

[kif]

Actually, it is only 25x at worse on my laptop...

[kif]

Here are the results on my (old) laptop:

split | algo      | impl  | 1d (ms)  | 2d (ms)  | ratio  | Device
--------------------------------------------------------------------------------
no    | histogram | python|   63.725 |  135.573 |    2.1 | 
no    | histogram | cython|   17.465 |   37.905 |    2.2 | 
bbox  | histogram | cython|   45.611 |   64.172 |    1.4 | 
full  | histogram | cython|  228.185 |  312.256 |    1.4 | 
no    | csr       | cython|   11.075 |   12.545 |    1.1 | 
bbox  | csr       | cython|   13.693 |   15.966 |    1.2 | 
no    | csr       | python|   19.258 |   25.799 |    1.3 | 
bbox  | csr       | python|   21.846 |   29.012 |    1.3 | 
no    | csc       | cython|   13.534 |   20.543 |    1.5 | 
bbox  | csc       | cython|   17.816 |   25.164 |    1.4 | 
no    | csc       | python|   16.682 |   23.221 |    1.4 | 
bbox  | csc       | python|   21.233 |   31.279 |    1.5 | 
bbox  | lut       | cython|   14.393 |   26.754 |    1.9 | 
no    | lut       | cython|   11.197 |   18.743 |    1.7 | 
full  | lut       | cython|   14.425 |   27.130 |    1.9 | 
full  | csr       | cython|   16.482 |   19.360 |    1.2 | 
full  | csr       | python|   21.188 |   28.768 |    1.4 | 
full  | csc       | cython|   23.758 |   34.939 |    1.5 | 
full  | csc       | python|   21.349 |   32.002 |    1.5 | 
no    | histogram | opencl|   32.043 |   37.674 |    1.2 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
no    | histogram | opencl|   80.712 |   56.640 |    0.7 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
bbox  | csr       | opencl|   26.490 |  331.641 |   12.5 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
no    | csr       | opencl|   18.251 |  316.649 |   17.3 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
bbox  | csr       | opencl|   29.409 |   62.706 |    2.1 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
no    | csr       | opencl|   20.499 |   56.883 |    2.8 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
full  | csr       | opencl|   27.079 |  329.307 |   12.2 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
full  | csr       | opencl|   29.757 |   63.372 |    2.1 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
bbox  | lut       | opencl|   24.377 |  429.258 |   17.6 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
no    | lut       | opencl|   15.522 |  296.737 |   19.1 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
bbox  | lut       | opencl|   32.221 |  772.728 |   24.0 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
no    | lut       | opencl|   21.114 |  452.177 |   21.4 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved
full  | lut       | opencl|   23.025 |  431.779 |   18.8 | Intel(R) OpenCL / Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
full  | lut       | opencl|   29.039 |  771.394 |   26.6 | Intel Gen OCL Driver / Intel(R) HD Graphics Haswell Ultrabook GT3 reserved

[kif]

Here is the notebooks to reproduce the results on your local computer:
profile_2d.pdf

[kif]

Here is the notebook integrated in the documentation.
The performances are rather similar for cython implementation of any algorithm. On GPU, 2D integration is up to 5x slower than 1D in CSR mode (LUT is deprecated, there is no added value in comparison to CSR, it might be removed in the future).

I will now close the discussion, feel free to reopen it if you have a more specific example.

[JamesWrigley]

Sorry for the late reply, I was on holiday last week. Thanks for adding those benchmarks to the docs, that's really useful! But I realize now that I was asking an X/Y question, my real goal is to get azimuthal profiles with integrate_radial(), which uses integrate2d() internally 😅 Do you think there's a way to improve the performance of integrate_radial() or is using integrate2d() the best we can do for now?