Add two different entity rulers to the pipeline - spaCy v3.1

[zmwangBE]

Hi all,

I used to have two different entity rulers in v2.3.1, i.e. one base ruler, the other one is called composite ruler. For example,

In the base ruler, I will have
{"label": "L1-FLOOR-NUMBER", "pattern": [{"LOWER": "level"}, {"LOWER": "1"}], "id": "floor-number"}

The composite ruler is built on top of the base ruler
{"label": "PAGE-TITLE", "pattern": [{"ENT_TYPE": "L1-FLOOR-NUMBER", "OP": "+"}, {"ENT_TYPE": "FLOOR-LEVEL"}], "id": "first-floor-reference"}

We wanted to upgrade to v3.1. However, I could not use the old method any more to add two different rulers, one after the other.

I can only add 'entity_ruler' for the base rules, but when I try to add another 'entity_ruler', it complains that the 'entity_ruler' already exists.

I understand that I may have to use custom components, but I still need the 'entity_ruler' class to load all the rules from the disk.

Any comments on this?

Thanks in advance.

[polm]

You can add two copies of the same component by specifying a different name for at least one of them. See the double NER project for an example of this.

[zmwangBE]

Thanks for the quick reply.

I use the follow code and it does not behave in the way I want

nlp = English()
ruler = nlp.add_pipe("entity_ruler")

patterns = [{"label": "L1-FLOOR-NUMBER", "pattern": [{"LOWER": "level"}, {"LOWER": "1"}], "id": "floor-number"}, 
{"label": "FLOOR-LEVEL", "pattern": [{"LOWER": {"REGEX": "^(floor|level)$"}}], "id": "floor-level"}
]
ruler.add_patterns(patterns)

comp_nlp = English()
comp_nlp.add_pipe("entity_ruler")
comp_ruler = nlp.add_pipe("entity_ruler",
                          config={"overwrite_ents": True},
                          name="composite_ruler",
                          source=comp_nlp,
                          after="entity_ruler")

comp_ruler.add_patterns([{"label": "PAGE-TITLE", "pattern": [{"ENT_TYPE": "L1-FLOOR-NUMBER", "OP": "+"}, {"ENT_TYPE": "FLOOR-LEVEL"}], "id": "first-floor-reference"}])

doc = nlp("Apple is at level 1 floor opening its first big office in San Francisco.")
print([(ent.text, ent.label_) for ent in doc.ents])

The result is only
[('level 1', 'L1-FLOOR-NUMBER'), ('floor', 'FLOOR-LEVEL')]

I would expect it can find entity called PAGE-TITLE

[polm]

You're almost there - the source is only used when you are pulling an existing component out of another pipeline. That was necessary in the NER example, because it's a trained component, but in this case you're building the Ruler from scratch, so you don't need the comp_nlp or source argument. This works:

import spacy

nlp = spacy.blank("en")
ruler = nlp.add_pipe("entity_ruler")

patterns = [{"label": "L1-FLOOR-NUMBER", "pattern": [{"LOWER": "level"}, {"LOWER": "1"}], "id": "floor-number"}, 
{"label": "FLOOR-LEVEL", "pattern": [{"LOWER": {"REGEX": "^(floor|level)$"}}], "id": "floor-level"}
]
ruler.add_patterns(patterns)

comp_ruler = nlp.add_pipe("entity_ruler",
                          config={"overwrite_ents": True},
                          name="composite_ruler",
                          after="entity_ruler")

comp_ruler.add_patterns([{"label": "PAGE-TITLE", "pattern": [{"ENT_TYPE": "L1-FLOOR-NUMBER", "OP": "+"}, {"ENT_TYPE": "FLOOR-LEVEL"}], "id": "first-floor-reference"}])

doc = nlp("Apple is at level 1 floor opening its first big office in San Francisco.")
print([(ent.text, ent.label_) for ent in doc.ents])