spacy nlp webtrf GPU error

[badri-thinker]

On google colab, I am getting an error since yesterday. I have GPU and is detected. spacy.require_gpu() returns true.
I have been using the standard install -U spacy, install spacy-transformers,!python -m spacy download en_core_web_trf
nlp = spacy.load("en_core_web_trf"). it used to work fine until now

/usr/local/lib/python3.7/dist-packages/spacy_transformers/align.py in
apply_alignment(ops, align, X)
43 shape = X.shape
44 indices = cast(Ints1d, align.dataXd)
---> 45 Y = Ragged(X[indices], cast(Ints1d,
ops.asarray(align.lengths)))
46
47 def backprop_apply_alignment(dY: Ragged) -> Floats2d:
cupy/_core/core.pyx in cupy._core.core.ndarray.getitem()
cupy/_core/_routines_indexing.pyx in
cupy._core._routines_indexing._ndarray_getitem()
cupy/_core/core.pyx in cupy._core.core.ndarray.take()
cupy/_core/_routines_indexing.pyx in
cupy._core._routines_indexing._ndarray_take()
cupy/_core/_routines_indexing.pyx in cupy._core._routines_indexing._take()
cupy/_core/_kernel.pyx in cupy._core._kernel.ElementwiseKernel.call()
cupy/_core/_kernel.pyx in cupy._core._kernel._preprocess_args()
cupy/_core/_kernel.pyx in cupy._core._kernel._check_array_device_id()
ValueError: Array device must be same as the current device: array device
= -2 while current = 0

[adrianeboyd]

Thanks for the report, I tried to replicate this in colab both with and without a GPU and didn't run into this error. Can you first check if restarting the runtime fixes the problem?

If it's still not working, can you provide the full code that leads to this error or an example colab notebook that shows the errors? (When copying and pasting code or error messages, please format using markdown code blocks with three backticks before and after the code.)

[badri-thinker]

/usr/local/lib/python3.7/dist-packages/spacy_transformers/align.py in apply_alignment(ops, align, X)
43 shape = X.shape
44 indices = cast(Ints1d, align.dataXd)
---> 45 Y = Ragged(X[indices], cast(Ints1d, ops.asarray(align.lengths)))
46
47 def backprop_apply_alignment(dY: Ragged) -> Floats2d:

cupy/_core/core.pyx in cupy._core.core.ndarray.getitem()

cupy/_core/_routines_indexing.pyx in cupy._core._routines_indexing._ndarray_getitem()

cupy/_core/core.pyx in cupy._core.core.ndarray.take()

cupy/_core/_routines_indexing.pyx in cupy._core._routines_indexing._ndarray_take()

cupy/_core/_routines_indexing.pyx in cupy._core._routines_indexing._take()

cupy/_core/_kernel.pyx in cupy._core._kernel.ElementwiseKernel.call()

cupy/_core/_kernel.pyx in cupy._core._kernel._preprocess_args()

cupy/_core/_kernel.pyx in cupy._core._kernel._check_array_device_id()

ValueError: Array device must be same as the current device: array device = -2 while current = 0

[adrianeboyd]

Thanks for the info. I still can't reproduce this, but I've seen in the past that replacing the default cupy installation with spacy[cuda111] can cause problems.

spacy and cupy should already be installed, so try just:

!spacy download en_core_web_trf
import spacy
spacy.require_gpu()
nlp = spacy.load("en_core_web_trf")
nlp("test")

[badri-thinker]

the installation commands I have used
'''
!pip install -U spacy
!pip install spacy-transformers
import spacy

'''
'''
!spacy download en_core_web_trf
import spacy
spacy.require_gpu()
nlp = spacy.load("en_core_web_trf")
nlp("test")
'''
I still get the same error on google colab. if we initiate spacy.require.gpu() and load the model in another cell there is no error but there is no speed up. only CPU times

[badri-thinker]

FYI: This error started appearing since Monday (Aug 15).

[adrianeboyd]

One thing you can try is downgrading to spacy-transformers==1.1.7 because it's possible one of the changes in v1.1.8 is causing this. In general, though, if we can't replicate it it's hard for us to debug.

Just to double-check, can you run this colab notebook? [removed]

[badri-thinker]

The colab code works. if nlp("test") is in the same cell , then it runs in GPU time (0s) . However, if nlp('test') is in another cell, I get the error

…

On Thu, Aug 18, 2022 at 1:30 PM Adriane Boyd ***@***.***> wrote: One thing you can try is downgrading to spacy-transformers==1.1.7 because it's possible one of the changes in v1.1.8 is causing this. In general, though, if we can't replicate it it's hard for us to debug. Just to double-check, can you run this colab notebook? https://colab.research.google.com/drive/12lG2NQRqvjxuOuH5nD1LMGlkKqgQkczt?usp=sharing — Reply to this email directly, view it on GitHub <#11330 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKBELOZWKJNQ7BC6AVVF3N3VZZXMTANCNFSM56ZLVMEQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[badri-thinker]

Thanks a million for all the help

[adrianeboyd]

I think this issue with jupyter is what you're noticing: https://spacy.io/usage/v3#jupyter-notebook-gpu

[badri-thinker]

if require_gpu, load and nlp("init") are all in the same cell, then it works. if nlp("init") is in a different cell , I get an error for spacy 3.4.1

[adrianeboyd]

Sorry, I still can't reproduce this. Can you provide a link to a minimal colab notebook that shows this problem?

[fatihbozdag]

I got the same error, see post

simply replace code lines as;

nlp = spacy.load('en_core_web_trf)
spacy.require_gpu()

[badri-thinker]

but processing time will not reflect GPU times. it will reflect CPU time

[badri-thinker]

can u check if time taken is any better than CPU even though you have GPU. the only think worked for me was that the spacy.load and nlp(doc) were in the same cell, Then i see the speedup

[fatihbozdag]

Well, it works. You can simply observe GPU Memory usage on Colab. Indeed, I work with big data mostly, roughly 5 million words corpus. Everything seems like working.

import time ###GPU RUN###
start = time.time()
doc = nlp("can u check if time taken is any better than CPU even though you have GPU. the only think worked for me was that the spacy.load and nlp(doc) were in the same cell, Then i see the speedup")
end = time.time()
print(end - start)

0.11311221122741699

import time ###CPU RUN###
start = time.time()
doc = nlp("can u check if time taken is any better than CPU even though you have GPU. the only think worked for me was that the spacy.load and nlp(doc) were in the same cell, Then i see the speedup")
end = time.time()
print(end - start)

/usr/local/lib/python3.7/dist-packages/torch/amp/autocast_mode.py:198: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
  warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')
0.35324931144714355

[simonschoe]

@adrianeboyd I just recently ran into the exact same issue. Having run my colab notebook ~3 months ago, it now wouldn't detect the GPU, i.e., require_gpu() would always throw a ValueError at me (irrespective of stuff being in the same cell):

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-2-3b785b943e48>](https://localhost:8080/#) in <module>
      3 from spacy import require_gpu
      4 
----> 5 require_gpu()

[/usr/local/lib/python3.7/dist-packages/thinc/util.py](https://localhost:8080/#) in require_gpu(gpu_id)
    193 
    194     if not has_gpu:
--> 195         raise ValueError("No GPU devices detected")
    196 
    197     if has_cupy_gpu:

ValueError: No GPU devices detected

For me the trick was to change my installations from !python -m pip install --upgrade cupy-cuda111 to cupy-cuda112.

[adrianeboyd]

Our experience is that it's better on colab NOT to install cupy yourself and to use the pre-installed version that's already available in the GPU runtime.