add_pipe not working with "en_core_web_sm"

[arditobryan]

Hi, I have tried to use my own lemmatizer on en_core_web_sm:

nlp = spacy.load('en_core_web_sm', disable=['lemmatizer']) 
nlp.remove_pipe("lemmatizer") #DISABLE seems not to be working, this is a good replacement

print(1, nlp.pipe_names)

#add pipeline component
config = {"mode": "lookup"}
lemmatizer = nlp.add_pipe("lemmatizer", config=config)
lemmatizer.initialize()

print(2, nlp.pipe_names)

sentence = "What a wonderful day today for playing"
doc = nlp(sentence)```

If I look at the nlp.pipe_names output:

1 ['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'ner']
2 ['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'ner']

The lemmatizer has not been added: I tried with nlp = English() and works.
Another issue, I had to use nlp.remove_pipe("lemmatizer") because disable was not working.

[polm]

It looks like there are a couple of things that are kind of mixed up here. First, I'm not sure what you mean when you say "disable doesn't work", disable and remove_pipe do different things.

• disable adds the name of the component to a list of disabled components internally. Disabled components are still in the pipeline, and will show in pipe_names, but they do not run. Because this is based on names, and not instances, if you add another component with the same name, it will still be disabled.
• remove_pipe actually removes the component from the pipeline.

What's happening in your sample code is that your new component is disabled because it has the same name as the old component. I think this is what you want to do (basically, just remove disable):

import spacy

nlp = spacy.load('en_core_web_sm') 
nlp.remove_pipe("lemmatizer")
print(1, nlp.pipe_names)

#add pipeline component
config = {"mode": "lookup"}
lemmatizer = nlp.add_pipe("lemmatizer", config=config)
lemmatizer.initialize()

print(2, nlp.pipe_names)

sentence = "What a wonderful day today for playing"
doc = nlp(sentence)
for tok in doc:
    print(tok, tok.lemma_, sep="\t")

If that isn't what you were trying to do, just let us know.