Why drop CUDA versions support so quickly?

[GeorgySk]

Jaxlib version 0.4.24 released on Feb 6 2024 dropped support of CUDA 12.2 which had its latest release in August 2023. Cuda 12.3 was released on October 2023. This means that CUDA 12.2 was supported for approximately four months after a newer minor version was released. While I understand that there must be a good reason to drop the support for older versions so quickly, unfortunately, I fail to see it. I'm curious about what these reasons might be.

I would expect older versions to be supported for as long as Nvidia will support them which, if I understand correctly, will be until June 2026.

References:

• https://jax.readthedocs.io/en/latest/changelog.html#jaxlib-0-4-24-feb-6-2024
• https://developer.nvidia.com/cuda-toolkit-archive
• https://docs.nvidia.com/datacenter/tesla/drivers/index.html (See Table 2. CUDA and Drivers)

[hawkinsp]

Simple enough: we are making use of newer CUDA features. In fact we're eager to upgrade to CUDA 12.4 since it fixes some known deadlocks in CUDA 12.3. NVIDIA don't retroactively make bug fixes or improvements to older CUDA releases as a rule, so if you're using an older release you will probably have a worse experience.

(We're looking into allowing some amount of backward compatibility in what we release, the fundamental blocker being testing at the moment. However the reason we're doing this is for PyTorch interoperability, since PyTorch is much slower to upgrade.)

However, given that CUDA is easy to install via pip these days, why is it difficult for you to upgrade?

[GeorgySk]

The reason is not particularly interesting. I'm currently stuck with an older Nvidia driver version because I lack the necessary permissions on my office PC. I wanted to avoid requesting an update as it takes time and sometimes things break, so I was exploring the possibility of installing JAX on an older CUDA version. However, it appears I may need to submit that request after all... :)
Thanks for your quick response!

[hawkinsp]

Unless I'm mistaken, you do not need to update your NVIDIA driver to update CUDA within the 12.x series.

[GeorgySk]

Hmm... If so, then I misunderstood the meaning of this table: https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id6.
In any case, it looks like you are right. I've just ran some tests after installing JAX following these guidelines: https://jax.readthedocs.io/en/latest/installation.html#pip-installation-gpu-cuda-installed-via-pip-easier. And while I'm getting a warning:

2024-03-14 14:54:49.829698: W external/xla/xla/service/gpu/nvptx_compiler.cc:742] The NVIDIA driver's CUDA version 
is 12.2 which is older than the ptxas CUDA version (12.4.99). Because the driver is older than the ptxas version, 
XLA is disabling parallel compilation, which may slow down compilation. You should update your NVIDIA driver or use 
the NVIDIA-provided CUDA forward compatibility packages.

I see that it's not critical according to your comment here: #18027 (comment)

[hawkinsp]

Yes, that's right. The warning is just that, a warning. If you can't upgrade your driver, don't worry about it. It's a minor thing, on the whole.

The table you should look at is this one:
https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id5
So long as your driver is at least the minimum, everything should work. And the minimum is the same for all CUDA 12 minor versions.