How to make custom trained spaCy pipeline more portable? #11695
-
I created a pipeline for my needs following the documentation, however the folder it creates is over 600mb and when I distribute the code to run it it requires the inclusion of spacy-transformers as a runtime dependency. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Some of the pretrained pipelines are small because they don't use transformers. The pretrained transformers pipelines, like |
Beta Was this translation helpful? Give feedback.
Some of the pretrained pipelines are small because they don't use transformers. The pretrained transformers pipelines, like
en_core_web_trf
, are over 400MB in size - transformers are just big. If you train a model without transformers it should be much smaller, though it depends on your use of word vectors, for example.