Projectivity of the parse tree

[danmysak]

A side note here says that "for the default English pipelines, the parse tree is projective." It doesn’t seem to be the case though for a sentence like "What is it needed for?", at least with this definition of projectivity. Is it a mistake or is a different definition being used?

[adrianeboyd]

Hi, can you explain your question a bit more? The parse tree for the example "What is it needed for?" is projective not projective for en_core_web_sm v3.1.0:

The parser can produce both projective and non-projective trees, but unless it's trained on non-projective data, it shouldn't produce them.

(Edited: example is not projective.)

[adrianeboyd]

Ah, wait, sorry, I see what you mean about the ?, which I merged here by default.

[adrianeboyd]

I checked the English training data and ~2% of sentences are non-projective, so this info in the docs is incorrect or out-of-date. Sorry for the confusion!

[danmysak]

Thanks for the reply, Adriane. Just in case, let me clarify that it’s not just the question mark that makes the tree non-projective; the fact that the arc for → what is above its "parent" arc needed → for renders the tree non-projective too (at least, as defined by (1) and (2)).

[adrianeboyd]

Wow, I was not looking carefully enough here, sorry. You may get non-projective parses from the pretrained English pipelines because some of the training sentences are not projective.

Again, if your training data is projective, the parser should only produce projective parses. The internal parser algorithm can only produce projective parses and the parser component uses the "Head" encoding scheme from Nivre & Nilsson 2005 to handle non-projective parses.