Spacy v3 freezes when running in a Celery task

[alitsaberi]

Hey guys,

I'm trying to use Spacy in a Celery task. but it seems that the task freezes at the line nlp("some text"). I've looked into discussions with similar issues with Spacy v2.x but none of their solutions seem to work for me. Anyone has an idea why this is happening? Let me know if any information is required to answer this. I'm using Spacy v3.1.3.

Thanks

[ljvmiranda921]

Hi @Alit8 ! Curious what model are you using? This may be related to libgomp, especially if you're using a *_trf (torch) model. If that's the case, please check #8678, #6662, and #8554 for possible solutions.

[alitsaberi]

Hey @ljvmiranda921 Thank you so mush. I wonder why I missed those discussions. I'm using a *_trf model and my problem was solved by adding torch.set_num_threads(1) to my code.