NER patterns activating the pipeline

[SimonPre]

Hi all,

I am currently working on a nlp pipeline part of which as rule-based NER. When adding the patterns to the ruler, the pipeline seems to be partially executed on the patterns. I figured out that every module preceding the ruler in the pipeline is executed. I have three questions about this:

• Why is this happening?

• Can I deactivate it?

• Are these runs flagged in a way to distinguish them from executing the pipeline on other strings?

The last question relates to the fact that I am currently writing the results of each step directly to a DB, but the runs on the patterns also trigger the write operations.

Thanks in advance for your help and explanations.

[rmitsch]

Hi @SimonPre! Could you post a code snippet so we can reproduce your setup?

[SimonPre]

Hi @rmitsch thanks for your response. I am sorry but currently I am failing to reproduce a nice minimal example showing the behaviour I have observed previously. But my question relates to entitiruler.py line 302 in spacy version 3.1.4. in particular this loop here:

for label, pattern, ent_id in zip(
                phrase_pattern_labels,
                self.nlp.pipe(phrase_pattern_texts),
                phrase_pattern_ids,
            ):
                phrase_pattern = {"label": label, "pattern": pattern}
                if ent_id:
                    phrase_pattern["id"] = ent_id
                phrase_patterns.append(phrase_pattern)

[rmitsch]

The reasoning behind this is that you might use a phrase_matcher_attr requiring annotations by preceding components in the pipeline, or components retokenizing the text (such as merge_entities). Hence you'll likely want to generate phrase pattern documents matching what a document would look like when it's processed at this point in the pipeline.

If you want to disable this, you have to do that outside of the entity ruler itself - see here for an example.

[SimonPre]

@rmitsch Thank you very much. This is very helpful!