issue loading custom packaged spacy model

[hkodela1]

I have created a custom factory component, added it the pipeline, created extension to access the component, saved it using to_disk() and packed the component using python -m spacy package ./en_example_pipeline ./packages --code functions.py.

When i load the custom packaged component using nlp = spacy.load("/path/to/pipeline")
I referenced the directory where config.cfg file is in the newly packed model
I get this error

ValueError: [E002] Can't find factory for 'themes' for language English (en). This usually happens when spaCy calls `nlp.create_pipe` with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator `@Language.component` (for function components) or `@Language.factory` (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, spancat, textcat_multilabel, en.lemmatizer

[adrianeboyd]

After packaging, you want to install the model and then load it from the package name rather than the path. The included code is only loaded automatically if you load from an installed package.

The actual names will be different but it looks something like this, look for the .tar.gz or .whl in the dist/ subdirectory:

spacy package /path/to/model_name /output_dir
pip install /output_dir/en_model_name-0.0.0/dist/en_model_name-0.0.0.tar.gz

Then:

nlp = spacy.load("en_model_name")

If you load from a path, you'll still need to import the additional code in an extra step before loading the model.

[hkodela1]

i did cd /output/en_pipeline and ran pip install dist/en_core_web_sm-3.1.0.tar.gz it ran successfully . But when I do spacy.load("en-core-web-sm-3.1.0") . I get error " [E050] Can't find model 'en-core-web-sm-3.1.0'. It doesn't seem to be a Python package or a valid path to a data directory.

Building wheels for collected packages: en-core-web-sm
  Building wheel for en-core-web-sm (setup.py) ... done
  Created wheel for en-core-web-sm: filename=en_core_web_sm-3.1.0-py3-none-any.whl size=13621009 sha256=de9e7f5b71016c3c0ad3d5c059a21463655b54f6239f829457d3652a5ecac5c5
  Stored in directory: c:\users\hkodela\appdata\local\pip\cache\wheels\4a\24\ef\939577415a04b8f8face14928695b7ccb5adefc5704e9be4d0
Successfully built en-core-web-sm

Installing collected packages: en-core-web-sm
  Attempting uninstall: en-core-web-sm

    Found existing installation: en-core-web-sm 3.1.0
    Uninstalling en-core-web-sm-3.1.0:
      Successfully uninstalled en-core-web-sm-3.1.0
Successfully installed en-core-web-sm-3.1.0

[adrianeboyd]

Just like you use import spacy without a version number, you want to use the package name without the version number:

nlp = spacy.load("en_pipeline")

You should be able to see the package name you installed with pip freeze.

(I would recommend against using en_core_web_sm as a name for your own custom package because you'll be too likely to get it confused with our pretrained package.)

[hkodela1]

That worked thanks!

[wjbmattingly]

Funnily enough I posted a video on this issue this morning, for others who are in the same situation and need a bit more explanation and a Jupyter Notebook to follow:

https://youtu.be/2xi9SvgFLks