[BUG] pylibwholegraph: nightly wheel tests failing: "driver on your system is too old"

[jameslamb]

Description

See https://github.com/rapidsai/cugraph-gnn/actions/workflows/test.yaml?query=branch%3Amain

Nightly tests haven't passed on main here since February 5th (about 2 weeks ago). That wasn't caught because of a bug in the check-nightly-ci CI check (being fixed in #409).

Only pylibwholegraph wheel tests are failing.

CUDA 12.2 and 12.9 jobs using earliest-driver (v535) are failing like this:

E RuntimeError: The NVIDIA driver on your system is too old (found version 12020). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org/ to install a PyTorch version that has been compiled with your version of the CUDA driver.

(build link)

Approach

Maybe torch needs to be pinned back to older versions in those CI jobs? They're currently getting torch==2.10.

[bdice]

It seems like this is pulling CUDA 13 builds in a CUDA 12.2 test job.

Collecting torch>=2.3
  Downloading https://download.pytorch.org/whl/cu130/torch-2.10.0%2Bcu130-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (30 kB)

@alexbarghi-nv is working on a fix.