Implementing SpanRuler to train data on - weak supervision

[goonhoon]

Hi,

I have been exploring Spacy with manual annotation for most part and would like to switch to weak supervision with SpanRuler as this seems to fit my use-case the best, while keeping only a certain part of my datased manually annotated.

I have about 1000 contracts containing 19 clause types of interest. Most contracts should contain at least 10 of such clauses. For each clause, I have between 4-8 patterns that will locate and annotate the clause within the document. I will use this, and only this data to then train my blank model.

For practice purposes, I tried the following six patterns to extract and annotate contract clauses mentioning governing law:

import spacy

nlp = spacy.load("models/model-best")
config = {"spans_key": None, "annotate_ents": True, "overwrite": False}
ruler = nlp.add_pipe("span_ruler")

patterns = [{"label": "Governing Law", "pattern": "governed by"},
            {"label": "Governing Law", "pattern": "governed in accordance with"},
            {"label": "Governing Law", "pattern": "governed under"},
            {"label": "Governing Law", "pattern": "in accordance with the laws of"},
            {"label": "Governing Law", "pattern": "governed by the laws of"},
            {"label": "Governing Law", "pattern": "shall be the substantive law of"}]
ruler.add_patterns(patterns)

test_text = open("test.txt", "r")
read2 = test_text.read

doc = nlp("read2")
print([(ent.text, ent.label_) for ent in doc.ents])

nlp.to_disk("models/model-best")

My questions are:

1. How to get my SpanRuler to save the annotated data to then be used/converted to .spacy (DocBin) file for training? What I cannot understand from the documentation is how the patterns actually annotate the data, and where it is then saved. I know the ruler can be loaded before the ner component, but where can I see the output annotated .json or docbin of the ruler with the corresponding 'start' and 'end' values of the entities?

2. to expand on 1), I always start my training in the CLI with my config.cfg specifying where the training and testing data is located. Since I am annotating data through rulers to then train on, how do I create and train a blank model on the ruler data if the ruler doesn't output a .spacy docbin file? I would like to test whether the ruler works with no manually-annotated data existing in the pipeline.

I understand that this is a beginner question but I keep reading the documentation back and forth and googling for similar projects and/or issues but still can't comprehend the above clearly, so any explanation will help greatly :) Thanks!

[thomashacker]

Hey, thanks for your question!

All information generated by the individual components is saved in the doc object. The SpanRuler adds all extracted spans to the doc.spans attribute. You can read more about this in our docs about processing pipelines. See here how to create a DocBin object with a list of docs, you can then save the DocBin as a .spacy file to use it for training/testing.

Looking at your code, there are some minor issues. I would recommend watching videos like these first to get a better understanding of how matching works in spaCy + having some nice examples to refer back to:

https://www.youtube.com/watch?v=BXzFAjtenHM&ab_channel=Explosion
https://www.youtube.com/watch?v=1UnAECwDLIY&ab_channel=PythonTutorialsforDigitalHumanities

About the training, could you provide us with more information about your config.cfg and what pipeline you want to train?