NER Model With additional Entities and Entity Linking

[hansm4305]

I already have a custom NER model built from scratch with 10 entities. But the number of documents is few and the accuracy is not good. So I am trying to make the accuracy a better one plus need to add additional 4 more entities. So the new model I am trying to build will have 14 entities.

From the present model, out of 10 entities, 4 entities (DATE, TIME, ORG, PERSON) are present in pre trianed models of spacy. Rest 6 are not present in pretrained models. So I went for custom model from scratch tagging all 10 entities.

My doubts are

1. Is it possible to train new model over the pretrained model so that I can use the 4 entities in pretrained model plus will get better accuracy for the new ones with fewer training documents
2. If existing entity in pretrained model plus new entity cannot be used as in 1, then is it possible to build all the entities (exisitng+new) on top of pretrained model for better accuracy with fewer training documents.
3. These entities are linked with each other, and if entity linking also needs to be applied for these, should I make separate tagging for entity linking OR can I prepare the document and training for entity linking + NER together

Or is there any better approach for achieving my requirements than the above.

[polm]

Is it possible to train new model over the pretrained model so that I can use the 4 entities in pretrained model plus will get better accuracy for the new ones with fewer training documents

It is possible, but hard to get good accuracy that way, especially when adding new entities.

If existing entity in pretrained model plus new entity cannot be used as in 1, then is it possible to build all the entities (exisitng+new) on top of pretrained model for better accuracy with fewer training documents.

I'm not entirely sure what you mean by this.

These entities are linked with each other, and if entity linking also needs to be applied for these, should I make separate tagging for entity linking OR can I prepare the document and training for entity linking + NER together

You can train NER and entity linking at the same time and it may or may not help - it depends on your data and the best way to check is to actually test it.

---

For the problem you have here, I would suggest you first make sure your NER component for new entities is working well on its own. If there are issues with that, combining it with other components will make it harder to evaluate and troubleshoot.

For using the already existing entities, for PERSON and ORG in particualr the pretrained pipeline has a lot of good training data, so I would suggest you use two separate NER components - the pretrained one and your own. You can see how to combine multiple NER components in the sample double NER project. If you want to remove entities you are not using, you can do that with a small custom component.

[hansm4305]

@polm Thanks. I will check the way as suggested

[EY4L]

@polm
For approaching the task of identifying custom entities.

Is it better to have 1 NER component for all custom entities
or
An NER component for each custom entity?

What is the best approach for this task?

If this is question is too broad then what factors need to be considered when deciding this?

[polm]

If the entities are like the entities in the spaCy default pipelines then it's usually better to have one model for all entities because then the model can learn interactions between labels. Managing a large number of NER models also gets complicated.

"What is the best approach" is indeed way too broad as a question. I would recommend you train a model with default settings as soon as possible, use that as a baseline, and then work on iterative improvements through experimentation rather than evaluating many approaches without trying anything. You may want to check the NER flowchart.

[EY4L]

@polm
Thank you very much for the reply.

After some initial training the accuracies are below 50, using 10k dataset.
I've long phrases as some of my entities, so going off the flowchart it seems it's not a great NER task.
Looking through other resources I think SpanCat could work better.
Thanks again for the guidance.