"SpanRuler" for generating SpanGroup Spans

[kinghuang]

I'm interested in generating span groups using patterns. Something like the EntityRuler, but with matching token sequences stored as Spans in a specified SpanGroup instead of being set as entities.

I've considered writing a SpanCategorizer suggester function that uses Matcher/PhraseMatcher to generate ranges. But, there'd be no way to directly specify the label.

[adrianeboyd]

It's probably not too difficult to convert the entity ruler into a custom component that does something like this:

doc.spans[spans_key] = matcher(doc, as_spans=True)

[nrodnova]

@adrianeboyd @kinghuang I wrote a class that does exactly that a while ago. My main interest was having overlapping entities. Since there is one more level of grouping/categorizing that comes with the doc.spans (label, kb_id and a span_group), I added it to the rule hierarchy, so it looks like this:

{
    'GROUP_NAME': {
        'LABEL' : {
            'KB_ID' :
                [rule_1, rule_2, rule_N],
        }
    }
}

There are also different levels of allowing/prohibiting overlapping of matching spans : on group level, on label level and on kb_id. It's currently done in python, but I could quickly change match manipulation to cython to make it faster, now when I unleashed the power of cython for myself :)

If there is an interest in exploring this, I can see if it makes sense to extend EntityRuler or have a new class. Intuitively, I'd say keep it separate, as now Spans and doc.ents are different beasts. But I'm happy with any direction here.

Let me know what you think.