ValueError: No GPU devices detected

[skaistt]

Hi,

I have an issue with GPU - it seems like SpaCy does not detect any GPUs.

This gives me True:

import torch
torch.cuda.is_available()

True

but this gives me False:

import spacy
spacy.prefer_gpu(0)

False

Why torch detects GPU but SpaCy does not?
nvidia-smi works fine and lists all available GPUs. SpaCy transformers are also installed.

None of the solutions found on the Internet helped me: already tried to re-install cupy many times, downgraded the version of spacy - did not solve my issue.

Maybe there is something I am missing?

OS: Linux
Python: 3.10.6
spaCy: 3.5.1
cuda: cupy-cuda117

{'spacy_version': '3.5.1',
 'location': '/opt/pypoetry/virtualenvs/code-MATOk_fk-py3.10/lib/python3.10/site-packages/spacy',
 'platform': 'Linux-4.14.314-1684020202-x86_64-with-glibc2.35',
 'python_version': '3.10.6',
 'pipelines': {'de_dep_news_trf': '3.5.0'}}

[danieldk]

spaCy and Thinc (our machine learning library) use CuPy for GPU operations. So to debug this, it is best to check what CuPy's view of the world is. What it the output of the following in your environment? I have included the output from one of our GPU machines to show what to expect:

>>> import cupy
>>> cupy.cuda.runtime.getDeviceCount()
1
>>> cupy.cuda.device.Device(0)
<CUDA Device 0>

[skaistt]

I tested it with two different cupy versions:

cupy-cuda117 gives this:

Original error:
  ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory
================================================================

Then, I uninstalled cupy-cuda117 and installed cupy-cuda11x instead, and got:

AttributeError: module 'cupy' has no attribute 'cuda'

The versions of cupy I used are these:

cupy-cuda117 (10.6.0)
cupy-cuda11x (12.0.0)

I tried to downgrade the version of cupy-cuda11x to 11.0.0, 11.1.0, 11.0.0 but still get:

AttributeError: module 'cupy' has no attribute 'cuda'

This an overview of my GPUs:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:04:00.0 Off |                  Off |
| N/A   57C    P0    27W /  70W |    930MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla T4            Off  | 00000000:0B:00.0 Off |                  Off |
| N/A   34C    P8     9W /  70W |      3MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  Tesla T4            Off  | 00000000:13:00.0 Off |                  Off |
| N/A   36C    P8     9W /  70W |      3MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  Tesla T4            Off  | 00000000:1B:00.0 Off |                  Off |
| N/A   35C    P8     9W /  70W |      3MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

[danieldk]

Original error:
  ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory
================================================================

This usually means that the CUDA libraries are not correctly installed on the machine, libcudart is a library from the CUDA toolkit and CuPy uses the system-provided CUDA.

[skaistt]

yes, that was CUDA issue, thanks :-)