Matcher with LOWER patter fails if token is not lowercase already

[renaud]

In this example below, the matcher does not match ByeWorldfailing, since in {"LOWER": "Bye"}, Bye is not lowercase.

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")
matcher = Matcher(nlp.vocab)
matcher.add("HelloWorldworking", None, [{"LOWER": "hello"}, {"LOWER": "world"}])
matcher.add("ByeWorldfailing"  , None, [{"LOWER": "Bye"}  , {"LOWER": "world"}])

doc = nlp("Hello world! Bye world!")
matches = matcher(doc)
print(matches) # <-- does not match ByeWorldfailing, since in {"LOWER": "Bye"}, Bye is not lowercase

Took me some time to figure it out...

[renaud]

I searched a bit in the source code, but could not find where this happens.
If anyone could point me, I'm happy to provide a PR.

[svlandeg]

I feel your pain, tracking this down ;-)

I'm not sure this can really be considered a bug in spaCy though. The patterns {ATTR: value} will match whether a token.ATTR equals the value, and the pattern you created simply can never match. But to replace it automatically with the lowercase variant may also not be the best solution, because then perhaps that will be expected in other cases as well? For instance, if we're matching on LEMMA, we wouldn't take the lemma of the value and match on that, but instead always match on the actual literal provided value.

But maybe in this specific LOWER case, a warning could be thrown ?

[renaud]

you're right it's not a bug per se.
and replacing it with the lowercase is indeed not the best solution.
a warning would be nice :-)
thanks @svlandeg