How to connect previously trained Spacy NER model to new empty Language pipeline? #13148
-
I've been able to train and load a model using a custom script involving a # post training
_model = spacy.load("./models/statutes/model-best")
_model.pipe_names
# ['tok2vec', 'ner']
doc = _model('Section 13 of PD No. 1869 provides xxx')
for e in doc.ents:
print(f"{e.text=} {e.label_=}")
# e.text='PD No. 1869' e.label_='STATUTE' I'd like to load this model in a separate language object and it seems to work but no entities are detected: x = spacy.blank("en")
x.add_pipe("ner", name="statute_ner", source=_model)
x.pipe_names
# ['statute_ner']
doc = x('Section 13 of PD No. 1869 provides xxx')
doc.ents
# () Note this follows the convention described in #10674. According to the docs:
Initially I thought that since I think the vocabulary matches. Wehn running There was some discussion about loading files via Hope for some guidance on what I'm doing wrong. Thank you! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
Ok, upon investigating #10674, @adrianeboyd links to https://github.com/explosion/projects/tree/v3/tutorials/ner_double and I see a curious: _model = spacy.load("./models/statutes/model-best")
_model.pipe_names
# ['tok2vec', 'ner']
# adapted from solution
_model.replace_listeners("tok2vec", "ner", ["model.tok2vec"])
# reuse same code above
x = spacy.blank("en")
x.add_pipe("ner", name="statute_ner", source=_model)
x.pipe_names
# ['statute_ner']
doc = x('Section 13 of PD No. 1869 provides xxx')
for e in doc.ents:
print(f"{e.text=} {e.label_=}")
# e.text='PD No. 1869' e.label_='STATUTE' |
Beta Was this translation helpful? Give feedback.
Ok, upon investigating #10674, @adrianeboyd links to https://github.com/explosion/projects/tree/v3/tutorials/ner_double and I see a curious:
drug_nlp.replace_listeners("tok2vec", "ner", ["model.tok2vec"])
. Didn't know this was a necessary step so I tried it and it solves my problem. Thank you!