spacy.TransitionBasedParser.v2 for NER: What is the Formal Description?

[cicciolado]

Hi all, I am trying to understand how the algorithm of spacy.TransitionBasedParser.v2 applied to NER works. After doing some research on the web, I found that it is based on the Transition-Based Chunking Model explained in this article:Neural Architectures for Named Entity Recognition Is this the real implementation, then using Stack LSTMs or not? Can a small formal description of how spacy.TransitionBasedParser.v2 works be provided?

[adrianeboyd]

The general transition-based parser algorithm is based on this and earlier related work: https://aclanthology.org/D15-1162/

For transition-based NER, this video gives an overview: https://spacy.io/universe/project/video-spacys-ner-model

I think that linked article is mostly unrelated to the specific implementation in spacy.

[cicciolado]

I got it thank you very much. But can no one provide a formal description of its actual implementation to actually understand how the NER component works?

[adrianeboyd]

This article on an early version of the parser may also be helpful:

https://explosion.ai/blog/parsing-english-in-python

For more detail our best advice is to take a look at the code. The models are currently defined in spacy.ml:

• https://github.com/explosion/spaCy/blob/master/spacy/ml/models/parser.py
• https://github.com/explosion/spaCy/blob/master/spacy/ml/parser_model.pyx
• https://github.com/explosion/spaCy/blob/master/spacy/ml/tb_framework.py

And the NER transition system is defined here:

• https://github.com/explosion/spaCy/blob/master/spacy/pipeline/_parser_internals/ner.pyx