CuPy GPU Linux CUDA help

[hotzjacobb]

Hi,

I'm currently trying to get spaCy to utilize the GPU (8 x NVIDIA RTX A5000 PCIe 4.0 GPU) on a X86 Linux machine running Ubuntu 20.04.4.

I get an error when running spacy.require_gpu(). I've tried to follow instructions from #9268 since those seem to detail the installation process for GPU utilization. I tried it with cupy 11.1 (mentioned as tested/working in that issue) and 11.3 (the current latest PyTorch compatible CUDA version) with their respective PyTorch versions and get the same error.

I can install the packages, but when the code runs and calls spacy.require_gpu() it tries to import CuPy I get a failure. I know this because running

import cupy
arr = cupy.zeros((5,))

Gives the error

File ".../env/lib/python3.9/site-packages/cupy/__init__.py", line 18, in <module>
    from cupy import _core  # NOQA
  File ".../env/lib/python3.9/site-packages/cupy/_core/__init__.py", line 1, in <module>
    from cupy._core import core  # NOQA
ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../test_cupy.py", line 1, in <module>
    import cupy
  File ".../env/lib/python3.9/site-packages/cupy/__init__.py", line 20, in <module>
    raise ImportError(f'''
ImportError: 
================================================================
Failed to import CuPy.

If you installed CuPy via wheels (cupy-cudaXXX or cupy-rocm-X-X), make sure that the package matches with the version of CUDA or ROCm installed.

On Linux, you may need to set LD_LIBRARY_PATH environment variable depending on how you installed CUDA/ROCm.

I then tried set LD_LIBRARY_PATH (after pip installations) to :/usr/local/cuda-11.6/lib64:/usr/local/cuda/lib64, where the CUDA binary is on my system but still no dice. I'm stumped as to what the issue is. I'm unsure if it's related to the cupy._core import error ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory, or if it's something wrong with LD_LIBRARY_PATH, or there's something bigger here that I'm missing/doing wrong). I greatly appreciate any help; thanks. 🙂

The command nvidia-smi returns

Thu Jun  2 12:14:04 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.54       Driver Version: 510.54       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A5000    Off  | 00000000:01:00.0 Off |                  Off |
| 30%   35C    P0    56W / 230W |      0MiB / 24564MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[adrianeboyd]

It probably isn't necessary to set LD_LIBRARY_PATH if you only have one version of CUDA installed and you've installed in a standard way (through your package manager like apt) and you're sure it's in your PATH. If you can call nvcc --version your path is probably okay.

If you're sure that CUDA is installed correctly, I'd suggest going through the cupy install instructions to see if there are any useful tips for similar environments or similar error messages:

https://docs.cupy.dev/en/stable/install.html

In particular, be sure that you don't have more than one cupy package installed, since it doesn't work if you have both cupy and cupy-cuda113 installed.

[hotzjacobb]

Hi Adriane,

Thanks for your reply. I've since been able to use spaCy without needing to do this for my use case, but I suspect my error is exactly what you sent me: I don't think I have CUDA toolkit installed on the machine.

I don't know if I'll actually be able to try adding a CUDA toolkit version and testing it out since I don't have super user privileges on the machine, but in the odd chance I can I'll provide an update here for future readers of this discussion. Thanks for your help.