Can't run the spacy spancat (spancategorizer) model?

[steve-solun]

I am trying to train the spancat model without luck.
I am getting:

ValueError: [E143] Labels for component 'spancat' not initialized. This can be fixed by calling add_label, or by providing a representative batch of examples to the component's 'initialize' method.

I did convert my NER ents to spans:

def main(loc: Path, lang: str, span_key: str):
    """
    Set the NER data into the doc.spans, under a given key.
    The SpanCategorizer component uses the doc.spans, so that it can work with
    overlapping or nested annotations, which can't be represented on the
    per-token level.
    """
    nlp = spacy.blank(lang)
    docbin = DocBin().from_disk(loc)
    docs = list(docbin.get_docs(nlp.vocab))
    for doc in docs:
        doc.spans[span_key] = list(doc.ents)
    DocBin(docs=docs).to_disk(loc)

Here is my config file:

[paths]
train = null
dev = null
vectors = null
init_tok2vec = null

[system]
gpu_allocator = null
seed = 444

[nlp]
lang = "id"
pipeline = ["spancat"]
batch_size = 1000
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.spancat]
factory = "spancat"
max_positive = null
scorer = {"@scorers":"spacy.spancat_scorer.v1"}
spans_key = "grit"
threshold = 0.5

[components.spancat.model]
@architectures = "spacy.SpanCategorizer.v1"

[components.spancat.model.reducer]
@layers = "spacy.mean_max_reducer.v1"
hidden_size = 128

[components.spancat.model.scorer]
@layers = "spacy.LinearLogistic.v1"
nO = null
nI = null

[components.spancat.model.tok2vec]
@architectures = "spacy.Tok2Vec.v1"

[components.spancat.model.tok2vec.embed]
@architectures = "spacy.MultiHashEmbed.v1"
width = 96
rows = [5000,2000,1000,1000]
attrs = ["ORTH","PREFIX","SUFFIX","SHAPE"]
include_static_vectors = false

[components.spancat.model.tok2vec.encode]
@architectures = "spacy.MaxoutWindowEncoder.v1"
width = 96
window_size = 1
maxout_pieces = 3
depth = 4

[components.spancat.suggester]
@misc = "spacy.ngram_suggester.v1"
sizes = [1,2,3]

[corpora]

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0
gold_preproc = false
limit = 0
augmenter = null

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 2000
gold_preproc = false
limit = 0
augmenter = null

[training]
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"
seed = ${system.seed}
gpu_allocator = ${system.gpu_allocator}
dropout = 0.1
accumulate_gradient = 1
patience = 1600
max_epochs = 70
max_steps = 20000
eval_frequency = 200
frozen_components = []
before_to_disk = null
annotating_components = []

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2
get_length = null
size = 100

[training.logger]
@loggers = "spacy.ConsoleLogger.v1"
progress_bar = false

[training.optimizer]
@optimizers = "Adam.v1"
beta1 = 0.9
beta2 = 0.999
L2_is_weight_decay = true
L2 = 0.0
grad_clip = 1.0
use_averages = false
eps = 0.00000001
learn_rate = 0.001

[training.score_weights]
spans_sc_f = null
spans_sc_p = null
spans_sc_r = null
spans_grit_f = 1.0
spans_grit_p = 0.0
spans_grit_r = 0.0

[pretraining]

[initialize]
vectors = null
init_tok2vec = ${paths.init_tok2vec}
vocab_data = null
lookups = null
before_init = null
after_init = null

[initialize.components]

[initialize.tokenizer]

I am using the "sc" key. Please advise how to solve it.

[ljvmiranda921]

Hi @steve-solun , please don't tag individual maintainers in the forums. As for your question, can you try using a SpanGroup instead of a list when assigning to doc.spans[spans_key]? Something like this:

from spacy.tokens import SpanGroup

group = SpanGroup(doc, name="sc", spans=spans)
doc.spans["sc"] = group

Where the spans are your entities in the Doc. After converting them, you might want to sanity-check if doc.spans do indeed contain the spans you assigned (plus their annotations).

[steve-solun]

@ljvmiranda921 thanks a lot for your answer dear. I am sorry but what are spans here? The IDE shows it in red.
I think there should be a tutorial A-Z to show folks how to build it with spacy blank model. I am not the first and not the last that will face this issue.
Can you please have a look? What should I change?

def convert_to_spancat(nlp_model, DocBinObjPath='./train.spacy', spans_output_path='./spansdata.spacy'):
	""" After getting NER data converting into doc.spans["sc"] """
	docbin = DocBin().from_disk(DocBinObjPath)
	docs = list(docbin.get_docs(nlp_model.vocab))
	group = SpanGroup(docs, name="sc", spans=spans) # here the IDE doesn't know what spans are.
	for doc in docs:
		doc.spans["sc"] = group
	DocBin(docs=docs).to_disk(spans_output_path)

[steve-solun]

I have solved it using the following function, but one should address the spans Span(doc, start, end, label) according to the project/text for their task. It worked for me because all the text (a few words in my case) are labeled with a label and this is my need.

def convert_to_docbin(input, output_path="./train.spacy", lang='en'):
	""" Convert a pair of text annotations into DocBin then save """
	# Load a new spacy model:
	nlp = spacy.blank(lang)
	# Create a DocBin object:
	db = DocBin()
	for text, annotations in input: # Data in previous format
		doc = nlp(text)
		ents = []
		spans = []
		for start, end, label in annotations: # Add character indexes
			spans.append(Span(doc, 0, len(doc), label=label))
			span = doc.char_span(start, end, label=label)
			ents.append(span)
		doc.ents = ents # Label the text with the ents
		group = SpanGroup(doc, name="sc", spans=spans)
		doc.spans["sc"] = group
		db.add(doc)
	db.to_disk(output_path)

[ljvmiranda921]

Hi @steve-solun , you're on the right track. If you're just training a SpanCategorizer model, you don't need to assign values to the doc.ents attribute. Also, for readability, you can respond to the same thread if you want to continue the conversation (instead of making a new one) as each new reply is treated as a potential answer.

[steve-solun]

@ljvmiranda921 sure I know, just left it there cuz I am using the same code to train NER also.