Unable to load vectors when training textcat (textcat.model has no attribute toke2vec')

[mrxiaohe]

Your Environment

• Operating System: Windows 10
• Python Version Used: 3.8
• spaCy Version Used: 2.3.1
• Environment Information:

I am trying to train a text classifier. I am following the example from https://spacy.io/usage/examples#training. However, it looks like I am unable to load custom vectors as evidenced in the error message below:

Is the method of loading vectors different for version 2.3.1? Thanks!

[svlandeg]

It looks like textcat.model is not yet properly initialized. Did you call nlp.begin_training()?

If that's not the problem - could you please post a minimal, working code snippet that reproduces the error, and the full stack trace (pasted, not as screenshot)?

[mrxiaohe]

@svlandeg Ah! I know what the problem was -- For some reason, I put nlp.begin_training() before the lines that load vectors, which is why the model was not properly initialized. Once I moved nlp.begin_training() up, the model became properly initiated.

I have a follow up question. Is it possible to load vectors from another model? I tried the following:

    optimizer = nlp.begin_training()  
    nlp_lg = spacy.load("en_core_web_lg")           #load another model
    textcat.model.tok2vec.from_bytes(nlp_lg.vocab.vectors.to_bytes())

I got the following error:

Thanks for the help!

[svlandeg]

The textcat.model.tok2vec variable refers to a sublayer of a neural network, which is not compatible with the data in vocab.vectors - so this approach won't work.

The best way to accomplish this, is to create your new nlp object with the vocab of the existing one:

nlp = English(nlp_lg.vocab)

You could have also just loaded that model without any of the pretrained components, effectively keeping the vocab/vectors only:

nlp_lg = spacy.load("en_core_web_lg", disable=["ner", "parser", "tagger"])

or just download the vectors model:

nlp = spacy.load("en_vectors_web_lg")

You can read more about vectors and serialization in the docs: https://spacy.io/usage/vectors-similarity & https://spacy.io/usage/saving-loading. Have fun ;-)