SciSpaCy vs. SpaCy Entity Linker

[amirj]

I’m using SciSpaCy to link genes to UMLS entities. On the other hand, SpaCy released EntityLinker and I’ve been thinking to improve the baseline SciSpaCy by training a better EL based on MedMentions dataset. It seems that KnowledgeBase defined in SciSpaCy is a standalone class, so all SpaCy functionalities are not available, does it correct?

In this situation, what’s the best way to develop a supervised EL leveraging both SpaCy and SciSpacy? --If I create a KB from UMLS and train an Entity Linker based on SpaCy, what I will miss in comparison to SciSpacy EL?

[svlandeg]

You're right that the KnowledgeBase from SciSpacy is not linked to KnowledgeBase class from spaCy, because the two were developed in parallel.

ScispaCy's entity linker mainly looks at string overlap, while spaCy's entity linker relies mostly on the context of a sentence to determine similarity. The two will be quite different in performance, and I can imagine that it depends on the use-case which is better. If the entities you're linking have high variability & ambiguity in their synonyms, spaCy's Machine Learning approach might make more sense. But if the variety isn't as high, ScispaCy's approach could potentially yield higher precision.

Some kind of ensemble would probably be even more ideal, but to my knowledge this hasn't been done yet.

[amirj]

Thanks @svlandeg for clarification. I'm going to take a shot and compare SciSpaCy vs. SpaCy EL in a biomedical domain; hopefully I will be able to build an ensemble to leverage the best worlds! --Please let me know your thoughts.

In addition, may I ask you to provide more information about how kb. get_alias_candidates does work? I'm interested to know if candidate generation process supports any type of approximate matching?

[svlandeg]

On the topic of candidate generation, this thread would be relevant to you: #7801

In a nutshell - the default candidate generation step only looks at the exact mention text, but this should be easily overwritable to adjust for your use-case.

[amirj]

Thanks @svlandeg Do you have any plan to add more functionality to the candidate generation phase? --e.g. n-gram matching,...

[svlandeg]

Welcome!

I don't think we'll be providing more such built-in candidate generators, as I think it should be straightforward to implement your own, and each use-case might require a slightly different one. In the spirit of Ines' "Let them write code" philosophy ;-)

[amirj]

Sure. Just wanted to be aware of the status 😎 Thanks