Update existing part-of-speech tagger without updating tok2vec weights

[andreacimino]

Hello everyone,

I am trying to build a custom train loop to update existing tagger weights with new examples.
I am trying to do something like this:

`other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'tagger']
with nlp.disable_pipes(*other_pipes): # only train NER
optimizer = nlp.create_optimizer()
for itn in range(n_iter):
random.shuffle(TRAIN_DATA)
losses = {}
for text, annotations in tqdm(TRAIN_DATA):
#doc = nlp.make_doc(text)
#example = Example.from_dict(doc, annotations)

        # # Training data for a part-of-speech tagger
        doc = Doc(nlp.vocab, words=["I", "like", "stuff"])
        # Fake example
        gold_dict = {"tags": ["DT", "DT", "DT"]}
        example = Example.from_dict(doc, gold_dict)
        nlp.update(
          [example], 
          drop=0.5,  
          sgd=optimizer,
          losses=losses)
    print(losses)`

Spacy complains with this error:

ValueError: [E954] The Tok2Vec listener did not receive any valid input from an upstream component.
How to update the tagger weights without updating tok2vec weights?

[adrianeboyd]

Let me go from general recommendations to the specific questions:

• In general, we strongly recommend using spacy train instead of a custom training loop. If you prefer to do it programmatically, you can also access the training methods directly in spacy.training.loop.

• If you want to update tagger without modifying any other components in the pipeline, then use replace_listeners to create an internal copy of the tok2vec model within the tagger so that the tagger can be updated independently of all other components.

• If you want to update tagger without replacing the listener:

  • You need to have tok2vec enabled in the pipeline.
  • If you want to update tagger without modifying tok2vec, then you need to add tok2vec to annotating_components and frozen_components in the config, which correspond to annotates and exclude in Language.update. Be aware that training without updating tok2vec is unlikely to work as well as replacing the listener.

[andreacimino]

Thank you for the prompt reply, and the useful suggestions!