merge_noun_chunks does not discard whitespace

[eyaler]

code to reproduce:

import spacy
from spacy.matcher import Matcher

spacy_model = 'en_core_web_trf'
nlp = spacy.load(spacy_model, exclude=['lemmatizer'])
matcher = Matcher(nlp.vocab, validate=True)
matcher.add('', [[{"POS":"NOUN"}]])
for par in nlp.pipe(['                boys and girls']):
    for span in matcher(par, as_spans=True):
        print(span.text)

# output:
#boys
#girls


nlp = spacy.load(spacy_model, exclude=['lemmatizer'])
nlp.add_pipe('merge_noun_chunks')
matcher = Matcher(nlp.vocab, validate=True)
matcher.add('', [[{"POS":"NOUN"}]])
for par in nlp.pipe(['                boys and girls']):
    for span in matcher(par, as_spans=True):
        print(span.text)

#output:
#                boys
#girls

• spaCy version: 3.4.0
• Platform: Windows-10-10.0.19044-SP0
• Python version: 3.9.12
• Pipelines: en_core_web_lg (3.4.0), en_core_web_trf (3.4.0)

[polm]

noun_chunks is not implemented using a Matcher; you can see the implementation here, but it uses dependency relations, so it gets words that are not nouns. For example, if you had he saw the red book, the red book would be a noun chunk. So this is not a bug or anything, though dependency relations for spaces are always kind of weird and arbitrary.

[eyaler]

i used to assume that matcher results give you the patterns specified. but it was silently failing giving me patterns with unwanted spaces. btw, i would also expect that "boy and girl" would be a single noun chunk. but for the main issue, this breaks user expectation, and can be easily fixed (at least partially) by applying strip. I would be interested in understanding what other unexpected failure modalities may rise from noun chunks.

[polm]

It's really hard to make a definition of noun chunks that everyone's happy with. Because of that, it's possible to change the noun chunks implementation if you have one you'd rather use, see #11072 for an example of how to do that.