Help Needed for Multi-Word Expressions

[mikeallen7]

Hi, I am looking for a Spacy expert to consult for my development team. We have implemented Spacy and the basics are working fine. We are struggling with multi-word expressions like "put on", "put off", etc.

We need to join these words into a single term (in our system, each row is a term):
https://ibb.co/PG2dRxy

We could write some custom code to conceptually do "if adposition, then join to the prior term if verb", but it seems like this is too simplistic and will cause hard to detect problems if there are many other unknown use cases involving adpositions (I am not a grammarian). This also seems like a common problem that the Spacy community has probably already solved.

If you know how to solve this issue (especially if you know how to do so via Spacy instead of writing custom code) and are looking for a consulting project, then please email me at [email protected].

Thanks!

[polm]

If you are focused on phrasal verbs, there's a prt dependency relation that specifically covers that. You should be able to use the dependency parse or the DependencyMatcher to act on that if you need to. You could also merge the tokens if they're adjacent.

If you want a consultant, there's a thread for that at the Prodigy forums that might be a better place to post.

[mikeallen7]

Thanks, Paul. This is very helpful!

Yes, for now, we are mostly focused on phrasal verbs, but I am sure this is just the start of our MWE (perhaps NLP in general) challenges. I see you're listed on the other thread. I will reach out to you by email. I would love to discuss this further.

Out of curiosity, can you comment on using the "Viterbi Algorithm" for solving MWE problems?

Thanks!

[polm]

Out of curiosity, can you comment on using the "Viterbi Algorithm" for solving MWE problems?

For MWEs the Viterbi Algorithm could be helpful for resolving potential overlaps, like "fire man eating dragon" or something (where "man eating" is a unit and "fire man" is not, I guess). It's used for this reason in Japanese tokenizers that I work with a lot. Whether it's useful for any particular application depends on what you're trying to do - MWE issues affect a wide variety of applications that can need very different approaches.