Clarification on providing vectors (glove, elmo, fasttext, etc) when training NER model

[CatarinaPC]

I have used spacy pretrain command to obtain pretrained weights and use them when training NER model.

1 - I have read somewhere, but I'm not sure that when training the model you can also provide other already trained vectors (I think I read something about elmo, glove and/or fasttext). Where can I find more information about this? Or is it possible for someone to give more information regarding this?

2 - Also, what is the difference between the parameters --vectors and --init-tok2vec ? What vectors does the first parameter refer to?

[svlandeg]

Hi and sorry for the late follow-up!

The pretrain command was still experimental in spaCy v2, and not very well documented. The good news is, that the recent pre-release of spaCy v3 comes with a revamped API for pretraining and swapping out Tok2Vec layers. The relevant documentation is here: https://nightly.spacy.io/usage/embeddings-transformers

The pretrain command is now run with the same config file you'd also use during training, making the whole process much more robust and transparent. You can get started with these config files using init config and setting the pretraining flag, e.g.:

python -m spacy init config config.cfg --lang en --pipeline ner --optimize accuracy --pretraining

which will produce a config file with (hopefully) sensible defaults.

Hope that's useful to you! I will preliminary close this issue, but feel free to comment here or open a new one if you run into any issues.