NER Training data with no labels to reduce false positive

[gunalanlakshmanan]

I am trying to create an NER model to identify one entity. I am facing with the problem of lot of false positives. Because the training data only contains data where the entity is present. During real time, I will get the data where we should not predict any entity. But I am getting lot of false positives on that.

How can I reduce the false positive? Can I add data where I should not predict anything in the training data without any labels? Will that work? Is there a way to add negative samples?

[kinghuang]

You can pass negatives via a SpanGroup. See the incorrect_spans_key config.

EntityRecognizer — Config and implementation

[polm]

One of the golden rules of models is that your training data should be as much like your real data as possible. For NER you should definitely have sentences with no entities (assuming any of your input data will be like that, which is typical).

It is also possible to add negative examples, like @kinghuang mentioned, though sometimes it's hard to get the balance right.

[gunalanlakshmanan]

@kinghuang @polm Thanks for the answer.

I have one doubt regarding incorrect_spans_key.
In the documentation for incorrect_spans_key (This key refers to a SpanGroup in doc.spans that specifies incorrect spans. The NER will learn not to predict (exactly) those spans), it is mentioned as "exactly". Will the model learn to generalise on those incorrect spans or is it just the rule based matching to filter out incorrect predictions?

[polm]

I'm not super familiar with the negative annotations code, but looking at the implementation, the model doesn't learn things not to tag (there is no model of "negative entities"), rather internally it makes sure to avoid reproducing those annotations. The NER model uses the same architecture as the transition parser and negative samples affect the cost calculations so that other predictions get made instead (whatever they may be).

[gunalanlakshmanan]

Thanks @polm.