NER assigning incorrect entity type

[rshahrabani]

Hi,

NER seems to be assigning an incorrect entity type to a token.

The code I'm using is:

text = 'On October 4, 2020, representatives of TC Energy delivered a non-binding offer (the “Offer Letter”) to the Issuer to acquire all of the outstanding Common Units of the Partnership not already owned by TC Energy and its affiliates, including TCPL, TransCan Northern and GP, in exchange for TC Energy Common Shares, at an exchange ratio of **0.650** TC Energy Common Shares for each Common Unit of the Partnership.'

nlp = spacy.load('en_core_web_lg')
doc = nlp(text)
print([t.text + '/' + t.ent_type_ for t in doc])

The output of this program is:

['On/', 'October/DATE', '4/DATE', ',/DATE', '2020/DATE', ',/', 'representatives/', 'of/', 'TC/ORG', 'Energy/ORG', 'delivered/', 'a/', 'non/', '-/', 'binding/', 'offer/', '(/', 'the/', '“/', 'Offer/', 'Letter/', '”/', ')/', 'to/', 'the/', 'Issuer/ORG', 'to/', 'acquire/', 'all/', 'of/', 'the/', 'outstanding/', 'Common/ORG', 'Units/ORG', 'of/', 'the/', 'Partnership/ORG', 'not/', 'already/', 'owned/', 'by/', 'TC/ORG', 'Energy/ORG', 'and/', 'its/', 'affiliates/', ',/', 'including/', 'TCPL/ORG', ',/', 'TransCan/ORG', 'Northern/ORG', 'and/', 'GP/PRODUCT', ',/', 'in/', 'exchange/', 'for/', 'TC/ORG', 'Energy/ORG', 'Common/ORG', 'Shares/ORG', ',/', 'at/', 'an/', 'exchange/', 'ratio/', 'of/', **'0.650/ORG'**, 'TC/ORG', 'Energy/ORG', 'Common/ORG', 'Shares/', 'for/', 'each/', 'Common/', 'Unit/', 'of/', 'the/', 'Partnership/', './']

The token in question is 0.650 which has an entity type of ORG. In a previous version of spaCy, this token had the correct entity type of CARDINAL.

Can you please assist us with this?

My spaCy info is:

spaCy version 3.1.3
Location C:\Python\lib\site-packages\spacy
Platform Windows-10-10.0.18363-SP0
Python version 3.9.7
Pipelines en_core_web_lg (3.1.0), en_core_web_md (3.1.0), en_core_web_sm (3.1.0)

Thanks.

[thomashacker]

Hey rshahrabani 🎉,
ML models are not always 100% correct with their predictions and do misclassification sometimes. For your case, I'd recommend using the EntityRuler to find CARDINALS rule-based or to further improve the model with your own data.

You can read more about the EntityRuler here: https://spacy.io/usage/rule-based-matching#entityruler
and about training models here: https://spacy.io/usage/training#basics

In general, these kinds of issues with predictions are related to the statistical models and issue #3052 is a good place to report them, thanks!

I've converted the issue into a discussion since this isn't a bug