Error on Trianing Custom NER on Spacy 3.0

[vrunm]

import spacy
import random
from spacy.util import minibatch, compounding
from spacy.training import Example
import spacy_transformers
nlp = spacy.load("en_core_web_trf")
After training the custom NER script:
# Add new entity labels to the entity recognizer
ner = nlp.get_pipe("ner")
ner.add_label("quarter")
ner.add_label("year")
ner.add_label("ticker")

# Disable other pipeline components
disable_pipes = [pipe for pipe in nlp.pipe_names if pipe != "ner"]

# Define optimizer and loss function
optimizer = nlp.begin_training()
loss_function = nlp.entity.create_loss_function()

# Train the model for 10 epochs
for epoch in range(10):
    random.shuffle(TRAIN_DATA)
    losses = {}
    batches = minibatch(TRAIN_DATA, size=compounding(4.0, 32.0, 1.001))
    for batch in batches:
        examples = []
        for text, annotations in batch:
            doc = nlp.make_doc(text)
            example = Example.from_dict(doc, annotations)
            examples.append(example)
        nlp.update(examples, sgd=optimizer, losses=losses, drop=0.2)
    print("Epoch:", epoch, " Losses:", losses)

# Test the trained model
for text, _ in TRAIN_DATA:
    doc = nlp(text)
    print("Text:", text)
    print("Entities:", [(ent.text, ent.label_) for ent in doc.ents])

I ran into this error:

[2023-03-29 06:55:25,299] [INFO] Added vocab lookups: lexeme_norm
[2023-03-29 06:55:25,301] [INFO] Created vocabulary
[2023-03-29 06:55:25,305] [INFO] Finished initializing nlp object

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_28/2958423719.py in <module>
     17 # Define optimizer and loss function
     18 optimizer = nlp.begin_training()
---> 19 loss_function = nlp.entity.create_loss_function()
     20 
     21 # Train the model for 10 epochs

AttributeError: 'English' object has no attribute 'entity'

[adrianeboyd]

This example is an incompatible mix spacy v2 and v3 code and there are a lot of additional details that will be incorrect for working with v3 transformer pipelines like en_core_web_trf.

We strongly recommend using the pipeline demo projects as a starting point, see an intro here: https://github.com/explosion/spaCy/tree/v3.5.1/examples. For updating en_core_web_trf, refer to the project pipelines/ner_demo_update (there are a number of training details to get correct for effective transformer training).

[vrunm]

I went through the documentation. Also I have got the training examples in the correct format but I am not sure about the training loop:

import spacy
import random
from spacy.util import minibatch, compounding
from spacy.training import Example

nlp = spacy.load("en_core_web_trf")

# Add new entity labels to the entity recognizer
ner = nlp.get_pipe("ner")
ner.add_label("quarter")
ner.add_label("year")
ner.add_label("ticker")

# Disable other pipeline components
disable_pipes = [pipe for pipe in nlp.pipe_names if pipe != "ner"]

# Define optimizer and dropout rate
optimizer = nlp.begin_training()
dropout = 0.2

# Train the model for 10 epochs
for epoch in range(10):
    random.shuffle(TRAIN_DATA)
    losses = {}
    batches = minibatch(TRAIN_DATA, size=compounding(4.0, 32.0, 1.001))
    for batch in batches:
        examples = []
        for text, annotations in batch:
            doc = nlp.make_doc(text)
            example = Example.from_dict(doc, annotations)
            examples.append(example)
        nlp.update(examples, sgd=optimizer, drop=dropout, losses=losses)
    print("Epoch:", epoch, " Losses:", losses)

After training it on a large no of samples it is not able to generate the correct NER tags.
Does this training loop work well for Spacy 3.0?

[kadarakos]

Hey vrunn,

We recommend using the training loop from spacy train to make development easier for the most common usecases. Please check out this material for more information: https://spacy.io/usage/training#api