KeyError when training the tagger

[JessitaMS]

I am presenting the following KeyError: '' when I am going to train the model with the data that I provide.

The data has the following format:
Word POS Tag

I then convert them to iob, then to spacy.
Then I configure the config so that within the pipeline I include tagger, ner.

And when it is going to execute it gives me this error, I don't know why it is not taking the POS values ​​that go in the input dataset.

I have found another issue similar to this one, for now no link of what is provided is active, for this reason I have reopened the discussion

[polm]

What is the text of your error? What command did you run that produced the error?

What does your data actually look like? Please give an example, it's not clear from the headers you posted.

How do you convert your data to IOB and then spaCy?

If there is a similar issue, please link to it.

[JessitaMS]

What is the text of your error? What command did you run that produced the error?

File "spacy\pipeline\tagger.pyx", line 205, in spacy.pipeline.tagger.Tagger.update
File "spacy\pipeline\tagger.pyx", line 266, in spacy.pipeline.tagger.Tagger.get_loss
File "C:\Users\anaconda3\envs\nlp\lib\site-packages\thinc\loss.py", line 193, in call
grads = self.get_grad(guesses, truths)
File "C:\Users\anaconda3\envs\nlp\lib\site-packages\thinc\loss.py", line 206, in get_grad
d_yh = self.cc.get_grad(yh, y)
File "C:\Users\anaconda3\envs\nlp\lib\site-packages\thinc\loss.py", line 120, in get_grad
target, mask = self.convert_truths(truths, guesses)
File "C:\Users\anaconda3\envs\nlp\lib\site-packages\thinc\loss.py", line 93, in convert_truths
neg_index = self._name_to_i[truths[i]]
KeyError: ''

What does your data actually look like? Please give an example, it's not clear from the headers you posted.

**Word	POS	Tag**

0 La DET O
1 jaula NOUN O
2 estaba AUX O
3 terminada ADJ O
4 . PUNCT O
5 Baltazar PROPN B-CHAR
6 la PRON O**

How do you convert your data to IOB and then spaCy?

#Convert into iob format
!python -m spacy convert dataAll.tsv ./ -t json -s -c iob

#Convert into spacy format
!python -m spacy convert dataAll.json ./ -t spacy

If there is a similar issue, please link to it.

#2711

[polm]

Please read the Github guide to Markdown, in particular the section on code blocks. Your code should be surrounded by backticks, not asterisks (which just make it bold text).

https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code

[svlandeg]

Are you running spaCy from the command line? If so - can you provide the command you ran?

If not - can you provide a minimal code snippet that reproduces the error?

[JessitaMS]

This is all the code I have, when the mentioned error appears:

import spacy
from spacy.tokens import DocBin
from tqdm import tqdm #to show the progress

nlp = spacy.blank("es")
db = DocBin()

###Add all components (ner|transformer|merge_entities...etc)
!python -m spacy init config config.cfg --lang es --pipeline tagger,ner --optimize efficiency

import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'

###Training dataset
!python -m spacy train config.cfg --output ./ --paths.train ./dataAll.spacy --paths.dev ./dataAll.spacy

[svlandeg]

Could you please heed Paul's advice and format your posts with markdown? That really helps us a lot to understand your question and help you better.

[JessitaMS]

Well, I have modified the previous code format, to make it more understandable, in any case, if something else is missing, tell me, to see if a solution to this problem is found. Because I don't understand what is happening with this error. Thanks

[polm]

I suspect that what is happening here is that a value in your CSV (probably Tag) is missing/blank, which is why the KeyError is for an empty string. I would check your original CSV to see if that's the case for any entries.

[JessitaMS]

Hi @polm ,

I have reviewed the entire dataset and there is no empty value in the Tag, do you suggest something else I can do?, because I really do not understand why this happens...

[polm]

Sorry that didn't help. If that's not the cause I'm not really sure what could explain it... Maybe you can try using a small part of your dataset to see if it runs without error?

I think you must have an invalid Tag somewhere based on the failing line.

 neg_index = self._name_to_i[truths[i]]

i is an index here so it can't be the empty string in the KeyError. So truths[i] must be an empty string, which must be caused by your input data. So I'm not sure what's going on.

If you can give us your data I could check it, though I understand that may not be possible.

[polm]

One other thing to check: could you share the output of debug data --verbose with your data?

[JessitaMS]

Hi @polm:

I send you a mini sample of the dataset
dataset1.csv

I have tried to do what you tell me to take only a mini piece of the dataset but it keeps doing the same thing. I hope now that with this data set, we can actually find where the problem lies.

[polm]

I have not been able to reproduce your error but I have some questions.

Is this TSV the TSV file at the start of your code, or after running the preprocessing above? I think you skipped a step where you save your TSV file before running spacy convert. Please just share the TSV file you pass to spacy convert.

You have a place in the code where you call spacy convert -c iob -t json. Your comments state that converts the TSV to IOB format, but that is not what that command does. That's converts data to the spaCy JSON training format, which is not required. You should skip that step and just call spacy convert something.tsv ./, maybe with -c something for your format. (The IOB format is also a format with one sentence per line and not relevant to your data.)

Your data looks like it's in what spaCy calls "ner" format with an extra field with token number on the front. If I do this I can train a model on your data without any error:

spacy convert -c ner dataAll3.tsv ./
spacy train config.cfg --paths.train dataAll3.spacy --paths.dev dataAll3.spacy

That works whether I leave the numbers column or not. The config is a config generated with the spacy command you gave upthread.

Also you should double check if the POS tags are being read in correctly - the NER format converter supports POS tags but normally would expect the conll 2003 format, which has fields of token pos chunk ner ("chunk" is not used in spaCy).

[JessitaMS]

ks before your code, and another line of three backticks after it.

Edit, done!

[JessitaMS]

Hi @polm,

I'm sorry I didn't put the step to save the tsv.

The dataset creation code is finally the following. I have skipped the conversion step to iob format (just like you told me)

#Save as tsv file
dataAll.to_csv('dataAll3.tsv', sep='\t')

#Convert directly tsv into spacy format
!python -m spacy convert -c ner dataAll3.tsv ./

I was hoping this would fix it, because I see you have tried this and it worked for you and I still get the same error. With which to avoid any conflict that I may be having in my code, I enclose everything I do for the training of the dataset.spacy

import spacy
from spacy.tokens import DocBin
from tqdm import tqdm #to show the progress

nlp = spacy.blank("es")
db = DocBin()

# Add all components (ner|transformer|merge_entities...etc)
!python -m spacy init config config.cfg --lang es --pipeline tagger,ner --optimize efficiency

import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'

#Training dataset
!python -m spacy train config.cfg --output ./ --paths.train ./dataAll3.spacy --paths.dev ./dataAll3.spacy

Here the error again :'(

[adrianeboyd]

I can't reproduce this error and it's really hard for us to know what's going on without seeing the exact tsv input you're using in the conversion.

You want your input tsv file to look like this (no header line; no line numbers; and because of how the converter works, you can leave out the chunk column and just have three columns):

La      DET     O
jaula   NOUN    O
estaba  AUX     O
terminada       ADJ     O
.       PUNCT   O
Baltazar        PROPN   B-CHAR
la      PRON    O
colgo   VERB    O
en      ADP     O
el      DET     O
alero   NOUN    O
,       PUNCT   O
por     ADP     O
la      DET     O
fuerza  NOUN    O
de      ADP     O
la      DET     O
costumbre       NOUN    O
,       PUNCT   O
y       CCONJ   O
cuando  SCONJ   O
acabo   VERB    O

Then for a quick test (obviously don't train your actual model on the same train and dev data):

spacy convert -c ner file.tsv .
spacy train config.cfg --paths.train file.spacy --paths.dev file.spacy

A better conversion would segment the data into shorter docs (but the exact details depend a lot on your data):

spacy convert -c ner -s -n 10 file.tsv .

If it helps, you can see samples of the supported formats here (this is using the same converter as the conll2003 examples): https://github.com/explosion/spaCy/tree/4b227f4861332ec40e45893dd49efa66ee786b67/extra/example_data/ner_example_data

[adrianeboyd]

Debugging a related issue I realized that I have run into the technical problem here before.

What's probably going on: because your TSV columns don't correspond to CoNLL 2003 format, you have accidentally converted words as token.tag. If you have ! as a tag it can cause this error due to the tagger's neg_prefix setting.

I think that fixing your conversion will solve this problem, in particular if you're using the UPOS tags shown in your examples as the tags.

[JessitaMS]

I don't know if I've mentioned it before @polm , but all this creation of the dataset with POS and NER Tag is because I would like to have a model like es_core_news_lg., hence in the pipeline use tagger

[JessitaMS]

Thanks a lot @adrianeboyd, I have found the error and it was precisely what he commented on in the first message he wrote me, the dataset had header and index and that is why it was not able to be processed correctly. Once this was fixed, the model was able to train properly. I didn't write to youbefore, because I was finishing testing it to work.

However, the only thing that strikes me is that the NER is executed perfectly, but the dependencies from the POS given in the dataset, the system is not able to paint it once I do
displacy.serve(doc, style="dep")

I don't know why spacy not paint the POS relations?, Do you have any suggestion?

[JessitaMS]

Dear @polm and @adrianeboyd:

An other issue that I got is, if I put more that 1 tag (CHAR, LOC, FAC, GPE), the trainig it's not capable to learn about this.

When I testing the model returns me this:
La T jaula UN estaba X terminada J . NCT Baltazar OPN la T colgo UN en P el T alero UN , por NCT

I don't know what they do, 'cause it's the first time that I saw these tags.

[adrianeboyd]

The CoNLL 2003 converter does not support dependency annotation. It only supports the columns:

• token
• tag
• [any number of optional ignored columns]
• IOB

For dependency annotation the only built-in converter is for CoNLL-U.

[JessitaMS]

Thanks a lot @adrianeboyd.

[JessitaMS]

Hi @adrianeboyd: The trained model only understands the tags with which it was trained, example: Maria(CHAR iob) appears 5 times in the training text) and it does recognize it if I introduce a new test text, but if for example I write a sentence that start with Pedro... it is not able to recognize Pedro as (CHAR iob), the model is not able to learn this??, what should I do to make it learn without having to put all the specific cases??

[polm]

As previously mentioned, new questions should go in new threads. Also please do not tag maintainers to get their attention or post the same question multiple times. See #10724 for an answer to your question here.

[JessitaMS]

Now I have another doubt @adrianeboyd I understand that to train the model, the more text it has, the better it will be able to learn the labels, but for example with the model es_core_news_lg, I have seen that PERSON labels many entities that actually corresponds to this label, but I understand that it may not have in its data all the names that I work with in my dataset.

So my question is, why with the data and labels that my model trained, is it not able to learn to associate another name with which it has not been trained, I do not know that it should be able to learn, without specifically putting all the cases? .

If you can help me understand this a bit, since I'm very new to the subject, thank you!

[thomashacker]

Hello,
it'd be great if you could start a new thread for your new question! This will also help other users with a similar problem to find the discussion

[JessitaMS]

Ok, perfect

[polm]

Please see #10724.