Spcay recoginize similar words into different entities

[LeiGong0125Carrot]

Hello everyone,

I used the following code to do entity recognition in the MIMIC discharge_summary dataset.

nlp = spacy.load("en_core_sci_sm") 

nlp.add_pipe("scispacy_linker", config={"resolve_abbreviations": True, "linker_name": "umls"})
linker = nlp.get_pipe("scispacy_linker")

similar_list = ["spinal", "spinals", "Some SPINALS", "one SPINAL", "bulbar", "bulbars", "BULBAR", "BULBARS"]
for sent in similar_list:  
    doc = nlp(sent)
    entity = doc.ents[0]

    print("Name: ", entity)
    entity = doc.ents[0]
    print("Name: ", entity)
    for umls_ent in entity._.kb_ents:
        print(linker.kb.cui_to_entity[umls_ent[0]])
    print("-----"*15)

But when I tried to use some similar words to test. The spacy returned quite different entities. Like 'spinals' and 'spinal' gives me different entities. Any way to solve this?

[svlandeg]

Hi!

Please note that scispacy and its pretrained models are not created/maintained by us but by AllenAI: https://github.com/allenai/scispacy, so it's probably best to ask your question there. I don't know exactly how their NER and EL components work, but I would imagine that you're supposed to feed full sentences to the nlp document instead of just a single word, like what you're doing in the sample code. Additionally, the way the words are being resolved will depend on the vocabulary that is being used - perhaps "spinals" and "spinal" really does refer to a different concept. Anyway - best to ask again on their issue tracker.