Where are static vectors used during training?

[nick-antisoc]

Hello spaCy community!
Thanks for all the great content here.

I would love some more info on some config options regarding static vectors.
I understand that "vectors" field under [initialize] section is for static vectors.
Is this setting related with "include_static_vectors" option under "tok2vec" model and how?
Are these options independent?

If for example I use the following under [initialize] section:

[initialize]
vectors = "en_core_web_lg"

and under "tok2vec" settings I use:

[components.tok2vec.model.embed]
...
include_static_vectors = false

where are these vectors applied/used?
Which components of a pipeline take into consideration these vectors during training, in order to improve the accuracy of the final model?

Thanks in advance,
Nick

[polm]

Good question. The interaction between these is a little subtle.

The initialize block is used when creating the pipeline for training. If you specify a model there, those vectors will be loaded into the pipeline, just like the vectors in the medium and large models. They will also be available in the final pipeline if you want to do something with them directly.

The tok2vec option determines whether tok2vec should use those same vectors when creating its embeddings. The tok2vec layer, and not the original embeddings, will be used for downstream pipeline components. If you've loaded the vectors you'd typically want to do this.

If you set include_static_vectors to be True but don't have any vectors loaded I think that would result in an error in training (if you load a model with no vectors using the vectors parameter you will get a warning).

Also be sure to check the docs on static vectors.

[nick-antisoc]

Thank you for the reply.
This clarifies things a lot.

I do not know if this is indeed the expected behaviour but I do not get a warning/error in training if I set:
include_static_vectors = true with vectors = null