NER and Overlapped entities

[GenVr]

Hi all,
I would like to use Spacy (with the NER network) to manage an entity search in a sentence, but with the possibility of overlap between close words.

As example:
"therefore specialized players and"
have to give both "players" and "specialized players".

Is it possible to get this overlap only with the Spacy NER?
I have read about spancat, but I have not quite understood how it can help me with this job, or how I can train the network to do so.
Thanks

[koaning]

It indeed sounds like you're more interested in span categorisation than named entity recognition. Both tasks involve finding substrings in text, but span categorisation allows for overlapping spans. It's supported as of spaCy 3.1 and you can read more about it here. There's an example project on GitHub that you might find useful as a starting point.

Training a model for span categorisation is very similar to training one for named entity recognition in spaCy, you'd still use the spacy train command from the terminal. You'll need the have training data prepared and a configuration file, which can be configured easily from the online interface on the docs.

If you're unfamiliar with the spaCy command-line interface or the project structure, then you may appreciate this YouTube video.

[GenVr]

@koaning Thank you for you answer. I created a train.spacy file and the base_config.cfg . Now i'm trying a spancat training.

First I did this:
python -m spacy init fill-config base_config.cfg config.cfg

And then:
python -m spacy train config.cfg --gpu-id 0 --paths.train ./corpus/train.spacy --paths.dev ./corpus/dev.spacy

But I had this error:
ValueError: [E143] Labels for component 'spancat' not initialized. This can be fixed by calling add_label, or by providing a representative batch of examples to the component's 'initialize' method.

So I think i need this command, and I did it:
python -m spacy init labels config.cfg ./corpus --paths.train ./corpus/train.spacy

but that gave me this error:
ValueError: [E913] Corpus path can't be None. Maybe you forgot to define it in your .cfg file or override it on the CLI?

Also with this:
!python -m spacy init labels config.cfg ./corpus --paths.train ./corpus/train.spacy --path.dev ./corpus/dev.spacy

I had this error:

✘ Error parsing config overrides
path -> dev   not a section value that can be overwritten

This is my config.cfg:

# This is an auto-generated partial config. To use it with 'spacy train'
# you can run spacy init fill-config to auto-fill all default settings:
# python -m spacy init fill-config ./base_config.cfg ./config.cfg
[paths]
train = null
dev = null
vectors = null
[system]
gpu_allocator = "pytorch"

[nlp]
lang = "en"
pipeline = ["transformer","spancat"]
batch_size = 128

[components]

[components.transformer]
factory = "transformer"

[components.transformer.model]
@architectures = "spacy-transformers.TransformerModel.v3"
name = "roberta-base"
tokenizer_config = {"use_fast": true}

[components.transformer.model.get_spans]
@span_getters = "spacy-transformers.strided_spans.v1"
window = 128
stride = 96

[components.spancat]
factory = "spancat"
max_positive = null
scorer = {"@scorers":"spacy.spancat_scorer.v1"}
spans_key = "sc"
threshold = 0.5

[components.spancat.model]
@architectures = "spacy.SpanCategorizer.v1"

[components.spancat.model.reducer]
@layers = "spacy.mean_max_reducer.v1"
hidden_size = 128

[components.spancat.model.scorer]
@layers = "spacy.LinearLogistic.v1"
nO = null
nI = null

[components.spancat.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0

[components.spancat.model.tok2vec.pooling]
@layers = "reduce_mean.v1"

[components.spancat.suggester]
@misc = "spacy.ngram_suggester.v1"
sizes = [1,2,3]

[corpora]

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 0

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0

[training]
accumulate_gradient = 3
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"

[training.optimizer]
@optimizers = "Adam.v1"

[training.optimizer.learn_rate]
@schedules = "warmup_linear.v1"
warmup_steps = 250
total_steps = 20000
initial_rate = 5e-5

[training.batcher]
@batchers = "spacy.batch_by_padded.v1"
discard_oversize = true
size = 2000
buffer = 256

[initialize]
vectors = ${paths.vectors}

I have the "dev" param in my [paths] so I don't know why I have this errors. How can I train?
Thanks

[pmbaumgartner]

!python -m spacy init labels config.cfg ./corpus --paths.train ./corpus/train.spacy --path.dev ./corpus/dev.spacy

If this was the command you're using, I think you might be missing an 's' on the end of paths.dev (you currently have path.dev, hence the error)

[GenVr]

@pmbaumgartner Oh sorry, my mistake. Now all it's working.
I'm using the pipeline [transfomers, spancat]. Maybe with NER also ( transfomer, ner, spancat) is it better?

[pmbaumgartner]

If you have entities formed from overlapping spans, you won't be able to do NER with spaCy's architecture, so you'll probably just need to stick with spancat. If you have entities from non-overlapping spans, it's fairly easy to convert back and forth between spans and entities in your data to try that out. If you want to do both, you'd have to have some final custom component that resolves disagreements between the two components.

[GenVr]

@pmbaumgartner Okay thank you. I trained both a pipeline with ner and spancat, and one with spancat only. In the one with spancat, I noticed that by running this:

# my trained model only spancat
nlp = spacy.load ("./ model / model-best /")

# my dev.spacy file
examples = list (DocBin (). from_disk ("./ corpus / dev.spacy"). get_docs (nlp.vocab))

# The doc, the esamples is like "players and goals"
doc = nlp (examples [12])

Nothing is found in doc.ents and doc.spans!

In the pipeline with ner & spancat I have some answers (however doc.spans does not give me the spans classes, only the words), like:

for doc.ents:
players my_class
goals my_class

doc.spans:
{'sc': [players, goals]}

However doc.spans does not give me the classes (player-> my_class, like doc.ents).

[pmbaumgartner]

I trained both a pipeline with ner and spancat, and one with spancat only

I thought one of the assumptions of your problem was you had overlapping spans, in this case, NER wouldn't work. You may be overwriting your relevant spans if you were actually able to train a model.

However doc.spans does not give me the classes (player-> my_class, like doc.ents).

You can access the label on each span in the same way you do for entities, like this:

[span.label_ for span in doc.spans['sc']]

Nothing is found in doc.ents and doc.spans!

How much training data do you have? This might just be an artifact of this training instance - if you don't have many examples, I wouldn't expect the model to pick up on the same information every time.

[GenVr]

You can access the label on each span in the same way you do for entities, like this: [span.label_ for span in doc.spans['sc']]

Thank you for this.

How much training data do you have? This might just be an artifact of this training instance - if you don't have many examples, I wouldn't expect the model to pick up on the same information every time.

As a test, I only have about 5k samples, I am trying to increase to reach a much larger number. Maybe for this, yes.
Should I also include samples with overlapped sentences? Or no overlapping sentences? (and automatically spancat will also give overlaps as a result)

For example:
sentence: "Diego Armando did six frontal goals"
spans I want: [Diego Armando, Diego, Armando, frontal goals, goals, six]

Thanks.