Omitting determiners from merged noun chunks

[lolfert]

Is there a way to omit determiners when merging noun phrases when using the merge_noun_chunks pipeline function? Right now I'm using a custom pipeline component and Textacy's noun chunking function to accomplish this, but if there's a way to do this using only spaCy I'd rather do that.

To clarify, I'd for instance like The quick brown fox jumps over the lazy dog to be tokenized as The [quick brown fox] jumps over the [lazy dog] instead of [The quick brown fox] jumps over [the lazy dog].

[rmitsch]

Hi @lolfert! Customized noun chunk merging is out of the box not possible with merge_noun_chunks().

A spaCy-only alternative to using Textacy is a custom pipeline component post-processing noun chunks with a modified implementation of merge_noun_chunks(). If you look up the source for Textacy's noun_chunks(), you'll see it's rather straightforward (same with merge_noun_chunks()). It should be easy enough to adapt those for your purposes and write a solution achieving something similar using only spaCy.

[lolfert]

That's what I figured, thanks for the help!