Is it possible to train a new label with 'en_core_web_trf' model

[rammohanrrr]

Am able to train a new label to 'sm' model.

How ever, while I training 'md' / 'lg' / 'trf' models,
am getting error """site-packages\spacy_transformers\layers\listener.py, line 47, in verify_inputs raise ValueError""" at nlp.update()

Can I get suggestions here

Thanks

[polm]

Training that works with the small model shouldn't give an error with the other models. Can you post the code you're using to train the model?

[rammohanrrr]

This is the code which am using to train 'sm' model

model = "en_core_web_trf"

nlp = spacy.load(model)  # load existing spaCy model
print("Loaded model '%s'" % model)

ner = nlp.get_pipe("ner")

# add labels
for text, annotations in TRAIN_DATA:
    for ent in annotations.get("entities"):
        ner.add_label(ent[2])

# get names of other pipes to disable them during training
pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]
other_pipes = [pipe for pipe in nlp.pipe_names if pipe not in pipe_exceptions]

with nlp.disable_pipes(*other_pipes):  # only train NER
    examples = []
    optimizer = nlp.resume_training()
    for text, annots in TRAIN_DATA:
        examples.append(Example.from_dict(nlp.make_doc(text), annots))
    for itn in range(n_iter):
        random.shuffle(examples)
        losses = {}
        for batch in minibatch(examples, size=compounding(4.0, 16.0, 1.001)):
            nlp.update(batch, sgd=optimizer, drop=0.2, losses=losses)
        print('losses : ', losses)

[rammohanrrr]

Is it a correct way approach or not?

Thanks

[rammohanrrr]

@polm Please see the above details.
Still am getting same error, please open this as bug

[adrianeboyd]

Hi, the problem above is trf_wordpiecer and trf_tok2vec are the names of the components from spacy-transformers for v2. In v3, this is now one component called transformer. You don't want to disable the transformer component in the training loop because the ner component "listens" to it: https://spacy.io/usage/embeddings-transformers#embedding-layers

Note that training just ner will degrade the performance of tagger and parser in the same pipeline since they all listen to the same transformer component. You probably want to create a new pipeline with only transformer + ner for further training.

Once you move beyond toy data, we'd also recommend using spacy train with a config instead of a script like this. You can use the config.cfg from en_core_web_trf as a starting point with the source option to load components from en_core_web_trf instead of training them from scratch: https://spacy.io/usage/processing-pipelines#sourced-components

[sureshPalla230393]

@adrianeboyd which loss function is used to calculate transformer component loss. To my understanding from the Spacy v3 documentation when training for NER task the NER component loss function is used to backpropagate the gradients. But when i train the en_core_web_trf for NER task, i see transformer loss and NER loss separately. Can you please explain what is this transformer loss.

[polm]

You have asked this question in three places, please don't do that. We will get back to you eventually. This is where you first asked: #8211