Matcher in spacy3.0 is slower than that in spacy2.0.18

[Pandalei97]

Hello,
I'm doing migrations from Spacy 2.0.18 to Spacy 3.0.6. I am aware that there are significant changes for the implementation of Matcher.

During my tests with large number of patterns on a document, the Matcher in Spacy2 takes about 0.16 second to process the document while it takes 0.9 second in Spacy3. The patterns and the test document are the same. I only have simple patterns with 'LOWER' or 'LEMMA' attribute.
(I tested execution time on matcher(doc))

Do you have any idea about the reason why it takes so much longer in spacy3 ? Maybe the problem has already occurred in the previous versions.

[polm]

Thanks for the report.

We don't do automated speed testing on the Matcher, so while we have designed it to be efficient and we do basic checks that it's not really slow, I don't think we profile it regularly. So we wouldn't know exactly when a performance change happened without retrospective profiling or reviewing code changes (unless someone remembers a change that might have caused it, which I don't).

Can you clarify how many patterns you have and how big your document is? This time is for only one input document, right?

One thing I would note is that while the change in times is a large fraction, your absolute time is quite small, so it could be effected by a lot of unrelated things, including even other processes on your machine. For a match against one doc it's hard to make any general conclusions about overall performance. If you see similar trends when doing a large batch on many different documents that would be a more reliable benchmark.

[Pandalei97]

We have around 25000 patterns generated to recognize entities. The test is applied on a document of 735 tokens. When I tripled the size of the document, the match time got tripled, too.

The absolute time seems quite small but we will also have other treatments in our webservice. In the past the whole process time in the webservice in around 0.9 second. After the upgrade, our webservice is generally 2.5 times slower than before.

[polm]

Thanks for explaining your use case, that makes it clear why the difference is important.

Relatively recently we fixed an issue with the EntityRuler (#8246) where a large number of duplicate patterns could be constructed, which could have had significant performance impact in some cases. But that issue was introduced after 2.0 and fixed by 3.0.6, and was only in the EntityRuler anyway.

I will take a look at this, though I can't promise a fix timeline or anything.

[Pandalei97]

Thanks a lot ! I will try to investigate too and let you know if I have possible solutions.

One possible cause may be the recognition of the attributes takes long time. We also have several phraseMatchers, and those are really fast. (Maybe just because we don't have many matched patterns in phraseMatcher on my test document) .

As the answer of Ines says:

Some background on how the PhraseMatcher works and why its mechanism is typically faster: When you add Doc objects to the PhraseMatcher, it sets flags on the tokens included in the patterns, indicating that they're matching a given pattern. It then calls into the regular Matcher and adds token-based patterns using the previously set flags. When you're matching, spaCy will only have to check the flags and not retrieve any token attributes – that's what should make the matching itself significantly faster at scale.

Since many more keys/attributes are allowed in the new matcher, maybe the verification step takes long time ?

[adrianeboyd]

As a note, PhraseMatcher was completely rewritten in v2.3, so Ines's comment is only relevant for v2.2 and earlier. In either case, the design doesn't affect the speed of Matcher on its own at all.

[polm]

Using this sample code it looks like the Matcher became slower between 2.0.18 and 2.1.0 (there are no releases in between those).

import spacy
from spacy.matcher import Matcher

nlp = spacy.blank("en")
matcher = Matcher(nlp.vocab)
patterns = [[{"LOWER": "hello"}]] * 25000
matcher.add("Hello", None, *patterns)

text = ' '.join(['bye'] * 1000)

doc = nlp(text)

import timeit

def domatch():
    matches = matcher(doc)
    len(matches)

iters = 100
print(timeit.timeit(domatch, number=iters) / iters)

For this particular test - which is mainly about cases for long single documents with many patterns - it looks like time may have doubled in between these versions. It also got slower later but that was more gradual.

I have looked at the intervening commits and it's not clear that any of them did anything that would cause this. It's a little hard to follow though, because in 2.0.18 all the matchers were in one file, and it was split up after that, so the git history is a little unclear.

[polm]

One thing about the intervening commits is that there are several bugfixes, so it's possible the Matcher was faster because it was was cheating by skipping certain cases.

[shadeMe]

Hi, we're still investigating this specific regression, but I wanted to mention that we recently fixed a related performance issue in the Matcher code. This fix should be available in the next release, most likely v3.3.0.