training NER: Influence on some forgotten entities from annotation

[tp-glt]

Hello,

Concerning datasets for training NER pipeline, I was wondering what is the impact on model accuracy with some forgotten entity from annotation.
For exemple, if we have 2 sentences along their entities in the training set:

Albert Einstein was born in Ulm

1. NAME : Albert Einstein
2. LOC : Ulm

Albert Einstein was born in Ulm

1. NAME : Albert Einstein

Will the accuracy impact of entity NAME be null, low, medium or severe ?
Will the accuracy impact of entity LOC be null, low, medium or severe ?

Thank you.

[polm]

It's hard to make a general statement about this, it really depends on how common issues like that are in your annotations.

That kind of inconsistency is definitely bad, and will hurt the model, but if it's one in a thousand annotations it shouldn't be a serious problem (though it might make that specific example work more poorly).