Registering custom component for training NER on custom dataset

[shrinidhin]

Hello. I am trying to creating a blank spacy model with a custom ner pipeline trained on recognizing custom entities, in spacy v3. Now while adding the pipeline before training, I am getting the "Can't find factory" error, which means I have to register the component. I also went through the answer provided for a similar question on stackoverflow.com
https://stackoverflow.com/questions/66433496/value-error-nlp-add-pipelanguagedetector-name-language-detector-last-tru
However I am not able to understand what should be included within the decorators and custom functions. I tried going though the usage documentation as well, I would like to know what exactly needs to be included within the decorator or custom function while adding any new component. Thanks!

from spacy.language import Language

@Language.factory('new_ner_pipe')
def new_ner_pipe(nlp,name):
      #Code

nlp = spacy.blank('en')
nlp.add_pipe('new_ner_pipe')
nlp.to_disk('test_ner')

[polm]

You only need to create a factory if you're making a new kind of component with different functionality - if you want to use an existing component with NER, you don't need to (and shouldn't) create a factory.

while adding the pipeline before training

Note that unlike v2, in v3 the preferred method for training is to use the config file system, so you won't have code to do this at all.

It might be helpful to look at chapter 4 in the spaCy course, which goes through the process of training an NER model.

One thing I'm confused about is why you were getting an error though. Could you paste the code you wrote and the exact error you got?

[shrinidhin]

So I am trying to create a blank spacy model, have an ner pipe on it custom trained to recognize new entities. Then I will combine that along with spacy's NER similar to the double ner demo example.
For this purpose, I tried running following code:

from spacy.language import Language
nlp = spacy.blank('en')
nlp.add_pipe('test_ner_pipe')
nlp.to_disk('test_ner')

I get the following error:

ValueError: [E002] Can't find factory for 'test_ner_pipe' for language English (en). This usually happens when spaCy calls `nlp.create_pipe` with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator `@Language.component` (for function components) or `@Language.factory` (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, spancat, textcat_multilabel, new_ner_pipe, en.lemmatizer

[polm]

OK, that explains the issue. The first argument of the add_pipe call needs to be the name of the component in the registry, not a name for your instance of the component. So it needs to be "ner".

If you want to change the name of your instance of the component, you pass that as a separate argument, like this:

nlp.add_pipe("ner", name="my_ner")

But you can do that when you combine the components, and it's not necessary to do it at training time. There's an example of this in the double NER README.

[shrinidhin]

Thanks!But I would need an ner pipeline to train my model to recognize custom entities right? Since I am creating a blank model using spacy.blank

[polm]

Yes, you need an NER component. nlp.add_pipe("ner") adds an NER component with the name "ner" to your pipeline. It might help to think of the first argument to add_pipe as being like a class name, while the name= argument is like the name of a variable. If you give something like my_ner as the first argument, that's not in the registry, so spaCy has no way to figure out which component to use.

This is all covered in the course chapter I linked, I really encourage you to give it a look.

[shrinidhin]

Cool!Thanks a lot!