Not sure how Matcher with alignments is supposed to work

[polkaYK]

Hello!

Today, I discovered a with_alignments argument for Matcher, but something is unclear to me. Since Matcher.add accepts patterns as a list, I suppose that I can provide multiple patterns for one match_id. After matching, it seems that I don't know which pattern from the list was used to create the alignments list; I only get indices of token patterns inside one chosen pattern list.

For example, I want to match hashtags, but I want to cover more cases that are listed in the documentation example.

from spacy.matcher import Matcher

nlp = spacy.blank('en')
matcher = Matcher(nlp.vocab)
matcher.add(
    'HASHTAG', 
    patterns = [
        # I search for hashtag that has separate word before it, I then filter out the first word of match
        [{'SPACY': True}, {'ORTH': '#', 'SPACY': False}, {'IS_PUNCT': False}], 
        [{'ORTH': '#', 'SPACY': False, 'IS_SENT_START': True}, {'IS_PUNCT': False}],
    ]
)
text = "#sometag or another #hashtag or one_that_will_not_#match"
doc = nlp(text)
matches = matcher(doc, with_alignments=True)
print(matches)

[(16536914698459818706, 0, 2, [0, 1]), (16536914698459818706, 3, 6, [0, 1, 2])]

As you can see, I can distinguish two patterns only because of the match length, but if the length was the same it wouldn't be possible. Maybe I am not supposed to put several different patterns under one match_id? But in that case, why are we able to specify multiple patterns at the same time?

[polm]

You're not missing anything here. It's fine to provide multiple patterns for a single match_id, but it's true it doesn't always work well with alignments.

The issue here is that while the Matcher has been in spaCy for a long time, alignments were not initially planned as a feature and were only added relatively recently. Unfortunately they don't cover this kind of case where it's not clear how they map back because you have multiple patterns.

That said, in this case you can check whether the first token of your match is a # or not to tell between your two patterns.

If you really need to differentiate two patterns, then it would be better to add them with different labels.

[polkaYK]

Thanks for the answer!

Do you think this is worth mentioning somewhere in the documentation? If so, I can try adding this info

[polm]

Sure, here's a PR that adds an explanation. #10826