Probably make sense to re-implement this class through pimpl-approach

[SergeyKopienko]

Probably make sense to re-implement this class __result_and_scratch_storage_impl through pimpl-approach(https://en.cppreference.com/w/cpp/language/pimpl)
In this case we will copy/move only one smart-pointer when returns __future instance with __result_and_scratch_storage_impl instance inside from functions and so far and so on.

[akukanov]

This is the state of __result_and_scratch_storage as of now:

    mutable sycl::queue __q;
    std::shared_ptr<_T> __scratch_buf;
    std::shared_ptr<_T> __result_buf;
    std::shared_ptr<__sycl_buffer_t> __sycl_buf;

    std::size_t __scratch_n;
    bool __use_USM_host;
    bool __supports_USM_device;

Given that sycl::queue is also a "common reference" type that allows fast shallow move, I am not sure wrapping all that into yet another indirection layer (with associated overheads) is really justified. Is there any evidence that it would be a net positive change?

[SergeyKopienko]

I have justification now only on level of idea: better to copy only one std::shared_ptr instead of these four pointers.

[akukanov]

Why are shared pointers used here in the first place? Is it just for RAII memory management, or also for lifetime control (i.e., there are other "owners" of these objects besides __result_and_scratch_storage)? If only for RAII, unique_ptr should be used as it has lower overhead and proper semantics.

For similar reasons, I do not like the idea of shared_ptr-based "pimpl" for this class. I think more often than not it is used as a unique instance. Moreover, currently there is a degree of control for embedding it into a __future instance, while with pimpl the overhead of shared_ptr will always be there.

[SergeyKopienko]

I think std::shared_ptr used here because out implementation sometimes copied some instances of this structure.
Yes, they are used for RAII resource management.
I agree that better to avoid these copy operations.

[SergeyKopienko]

@akukanov, additionally I want to highlight that here we have one additional copy of sycl::queue on each custom deleter here.
And the usage of std::uniqie_ptr cant' fix this problem.
So probably some plain data structure with usm raw-pointers, one instance of syc::queue and smart-pointer for sycl::buffer may be more compact and fast. And then we will have only smart-pointer (std::unique_ptr) wich will manage this plain struct instance.