CUDA Out Of Memory

[akul-goyal]

Hi, I am trying to run some text preprocessing on my data, and I want to use the GPU. My code is as follows:

spacy.require_gpu()
nlp = spacy.load('en_core_web_trf')
nlp = en_core_web_trf.load()
nlp.add_pipe("doc_cleaner")
X = df['text'].values.tolist()
df['text'] = [nlp(x).text for x in X]

Even with the "doc_cleaner" the memory within the GPU is still increasing. On certain indexes within X, I will get the memory out of error. However, if I run preprocessing on those indexes in isolation (without running all the preprocessing before those indexes) I do not get a memory out of error. Is there a way to move the output of the preprocessor to CPU and free up the memory on the GPU?

[kadarakos]

One way I can think of optimizing your code is to change the df['text'] one row at a time? Here is an example overwriting cells in pandas DataFrames one by one:

import pandas as pd

d = {"text": ["first text", "second text"], "foo": [1, 2]}
df = pd.DataFrame.from_dict(d)
for row in range(len(df)):
    df.at[row, "text"] = f"here {row}"

The original data frame:

          text  foo
0   first text    1
1  second text    2

and after the modification:

     text  foo
0  here 0    1
1  here 1    2

In your case I think you can do:

for row in range(len(df)):
    text = df.iloc[row]["text"]
    doc = nlp(text)
    df.at[row, "text"] = doc.text

I think that this should work, because the docs should be garbage collected.