pubmedbert's vocabulary to spacy vocabulary

[ud2195]

Background:-
We have a spacy pipeline in production where in some of our components expect spacy vocab to do certain things. Recently we tried out pubmedbert on our data and it is performing really nicely. We would like to integrate the same in production pipeline but without spacy's vocabulary certain components are failing.

Is there any way out where i can convert pubmedbert's vocabulary to spacy's vocab and use the same ?

[svlandeg]

With the pre-release of spaCy v3, it should be much easier to integrate transformers in your pipeline. Cf documentation here: https://nightly.spacy.io/usage/embeddings-transformers. This integration works via https://github.com/explosion/spacy-transformers, which can basically be used as an interface to the HuggingFace Transformer model hub. You'll then be able to use that Transformer as the first layer of your neural network models in spaCy, by "swapping out" the "standard" tok2vec layer. This should hopefully give you a nice performance boost, like we obtained ourselves with our pretrained models.

I hope that answers your question? If not, let me know!