In memory checking for cuda driver version

[apozharski]

JuliaGPU/CUDA.jl#2998 <- from this PR.

Currently this package calling out to a (possibly nonexistant in the case of ahead-of-time compilation) julia executable.

This PR gives an option to do this via dlopen/dlclose in memory which makes AoT compiled CUDA feasible, though making this "nondefault" is still painful as forcing users to set environment variables and/or getting LocalPreferences.toml to be read correctly is nontrivial.

@maleadt

[maleadt]

Do you need the forwards-compatible driver? Otherwise there's JULIA_CUDA_USE_COMPAT=false already. This because I'm not a fan of duplicating that much of code in a terribly hard to test package, with a potentially very large fallout when something breaks.

[apozharski]

Hmmm, fair enough. The forwards-compatible driver is quite useful as it makes the bundled distribution standalone but in principle I can use the compile time preferences in CUDA_Runtime_jll to always look for the local runtime and use a local libcuda installation. I am not sure how well this plays with the CUDA_Compiler_jll artifact as it doesn't have a way to use a local install but I guess I just have to test this. Closing.

[maleadt]

in principle I can use the compile time preferences in CUDA_Runtime_jll to always look for the local runtime and use a local libcuda installation

Note that those are two distinct things. We'll always use the local libcuda installation, and CUDA_Driver_jll serves to upgrade it. Whether you use the CUDA toolkit from artifacts or locally does not affect how the driver (aka. libcuda.so) is loaded. So I'd recommend sticking with artifacts, and setting JULIA_CUDA_USE_COMPAT=false. The core advantage of the forwards compatible driver is that it may allow use of more up to date driver features.

[apozharski]

Hm, ok so then there are some other strange bugs happening. The symptoms are that the CUDA_Runtime_jll selected seems to be too new: my local driver is 550 but the selected artifact seems to be 13.0 which is incompatible. Checking in memory for the local version at runtime (after the library is compiled) seems to fix it but this may be a red herring and not solving the underlying problem.

Perhaps I do some more digging and get a clearer understanding of what is happening then.

[maleadt]

my local driver is 550 but the selected artifact seems to be 13.0 which is incompatible.

Can you look at Libdl.dllist to see which libcuda was loaded?