entity ruler hang when n_process >1

[zwd2080]

I am using costom entity ruler with a lot patterns.
When I use n_process >1 with nlp.pipe, the process hang.
The possible reason may be I use two many patterns created from a large dictionary.
The entity ruler pipeline is extremely slow.

my code would be:

from spacy.lang.en import English
nlp = English()  
ruler = nlp.add_pipe("entity_ruler"  ) 
ruler.from_disk( pattern_path) # contains more than 50000 patterns 
for  doc in tqdm(nlp.pipe(text_list,   n_process=4)) :
      proces_each_doc 
 

Linux
Spacy 3.3
model=en_core_web_sm

[polm]

Can tell us your Python version, the output of spacy info, and provide sample code? Hard to troubleshoot this with just the information you've provided.

[zwd2080]

Thank you. Sinice I use a lot of patterns from a large vocab, the entity ruler pipeline is extremely slow.
Do you have any advice to speed up?

[polm]

You should be able to speed the EntityRuler up by disabling components you aren't using, or putting it before components it doesn't rely on in the pipeline. There's a PR to specify components to disable directly (#10944) but it's still in progress.

Can you give more details about why you're using the EntityRuler with a lot of patterns? There may be a better way to approach your problem.