Cannot deserialize model error

[sonynavdeep81]

I am trying to load the fasttext vectors "cc.pa.300.bin" and for the same I am giving the path for the vectors in config.cfg but when I am trying to train the model, I get the following error;

Please help me, is it because spaCy is expecting vectors in some other format? Kindly help me resolve this problem. Thank You.

[adrianeboyd]

The vectors should be loaded into a spacy pipeline using spacy init vectors and then the model name (if installed) or path should be used for this setting: https://spacy.io/api/cli#init-vectors

[sonynavdeep81]

In this command python -m spacy init vectors lang vectors_loc output_dir , it asks for lang but my language is punjabi which spaCy currently does not support though "cc.pa.300.bin" contains punjabi vectors. Shall I use en then? If I use en then it gives the following error;

The exact command I am using is following and getting the above output;

Please help!

[adrianeboyd]

init vectors only loads vectors in word2vec text format (with the size header line), so it can't load from fasttext .bin format.

[sonynavdeep81]

Yeah got you but is it ok to run this command with en although the vectors belong to another language?

[adrianeboyd]

You should use the same language here that you intend to use in your config later. For unsupported languages I would typically recommend xx, but check that the tokenizer defaults work well enough.

You can use any language where the tokenizer defaults work well (and adjust the tokenizer settings as necessary), but be aware that some of the other languages defaults will be incorrect (stop words, lemmatizer, etc.).