Could not find function 'spacy.MultiHashEmbed.v1' in function registry 'architectures'

[erotavlas]

I'm upgrading to spacy to 3.1.1 from 3.0.5 and creating a new executable using pyinstaller. This time when I run the exe I get this error when loading the model file.

I was able to successfully create a pyinstaller exe from 3.0.5 and load models from it just fine.

Error 3:<class 'catalogue.RegistryError'>[E893] Could not find function 'spacy.MultiHashEmbed.v1' in function registry 'architectures'. If you're using a custom function, make sure the code is available. If the function is provided by a third-party package, e.g. spacy-transformers, make sure the package is installed in your environment.

Available names: spacy.CharacterEmbed.v2, spacy.EntityLinker.v1, spacy.HashEmbedCNN.v2, spacy.MaxoutWindowEncoder.v2, spacy.MishWindowEncoder.v2, spacy.MultiHashEmbed.v2, spacy.PretrainCharacters.v1, spacy.PretrainVectors.v1, spacy.SpanCategorizer.v1, spacy.Tagger.v1, spacy.TextCatBOW.v2, spacy.TextCatCNN.v2, spacy.TextCatEnsemble.v2, spacy.TextCatLowData.v1, spacy.Tok2Vec.v2, spacy.Tok2VecListener.v1, spacy.TorchBiLSTMEncoder.v1, spacy.TransitionBasedParser.v2

[adrianeboyd]

Hi, check that you've included spacy-legacy>=3.0.7? Actually because of bug fixes spacy-legacy>=3.0.8 would be better, but v3.0.7 is when this particular architecture was moved from the core library to spacy-legacy.

[erotavlas]

yes, spacy-legacy 3.0.8 is installed and also included this submodule in the pyinstaller spec file.

My imports look like

copy_metadata('spacy_legacy')

block_cipher = None
datas = []

datas.extend(PyInstaller.utils.hooks.collect_data_files('thinc'))
datas.extend(PyInstaller.utils.hooks.collect_data_files('spacy'))

all_hidden_imports = [
			 'spacy.lexeme',
			 'spacy.lang.en',
			 'spacy.parts_of_speech',
			 'spacy.tokens._dict_proxies',
			 'spacy.lang.norm_exceptions',
			 'spacy.lang.lex_attrs',
			 'spacy.pipeline.transition_parser',
			 'spacy.pipeline._parser_internals.stateclass',
			 'spacy.pipeline._parser_internals._beam_utils',
			 'spacy.pipeline._parser_internals.transition_system',
			 'spacy.pipeline._parser_internals.arc_eager',
			 'spacy.pipeline._parser_internals.ner',
			 'srsly.msgpack.util', 
			 'cymem', 
			 'cymem.cymem', 
			 'preshed.maps',
			 'thinc.backends.linalg',
			 'thinc.extra.search',
			 'blis',
			 'blis.py',
			 ] + collect_submodules('spacy_legacy')

When it analyzes the hidden imports I specified, this is what it says.

12919 INFO: Analyzing hidden import 'spacy.lexeme'
12919 INFO: Analyzing hidden import 'spacy.lang.en'
12957 INFO: Analyzing hidden import 'spacy.parts_of_speech'
12957 INFO: Analyzing hidden import 'spacy.tokens._dict_proxies'
12960 INFO: Analyzing hidden import 'spacy.lang.norm_exceptions'
12961 INFO: Analyzing hidden import 'spacy.lang.lex_attrs'
12965 INFO: Analyzing hidden import 'spacy.pipeline.transition_parser'
12966 INFO: Analyzing hidden import 'spacy.pipeline._parser_internals.stateclass'
12966 INFO: Analyzing hidden import 'spacy.pipeline._parser_internals._beam_utils'
12966 INFO: Analyzing hidden import 'spacy.pipeline._parser_internals.transition_system'
12967 INFO: Analyzing hidden import 'spacy.pipeline._parser_internals.arc_eager'
12967 INFO: Analyzing hidden import 'spacy.pipeline._parser_internals.ner'
12967 INFO: Analyzing hidden import 'srsly.msgpack.util'
12968 INFO: Analyzing hidden import 'cymem'
12986 INFO: Analyzing hidden import 'cymem.cymem'
12986 INFO: Analyzing hidden import 'preshed.maps'
12986 INFO: Analyzing hidden import 'thinc.backends.linalg'
12987 INFO: Analyzing hidden import 'thinc.extra.search'
13002 INFO: Analyzing hidden import 'blis'
13019 INFO: Analyzing hidden import 'blis.py'
13020 INFO: Analyzing hidden import 'spacy_legacy.tests.test_legacy'
13056 INFO: Analyzing hidden import 'spacy_legacy.architectures'
13072 INFO: Analyzing hidden import 'spacy_legacy.loggers'
13074 INFO: Analyzing hidden import 'spacy_legacy.architectures.textcat'
13079 INFO: Analyzing hidden import 'spacy_legacy.layers'
13096 INFO: Analyzing hidden import 'spacy_legacy.layers.staticvectors_v1'
13100 INFO: Analyzing hidden import 'spacy_legacy.architectures.tok2vec'
13106 INFO: Analyzing hidden import 'spacy_legacy.architectures.parser'

[erotavlas]

I just got this error again today. This time it was with brand new conda environments running latest spacy 3.1.1. I'm not sure what is causing it. It only fails when I try loading the model using the compiled exe from pyinstaller.

I'm able to load the models fine in the environment.

I have spacy legacy 3.0.8 installed on all my environments. Maybe something else is missing, what else can I try and include

[erotavlas]

I think I used the config file from 3.0.5 when training model in 3.1.1 Do you think that could be the problem? I noticed that corpus tokenizer etc all have v1 at the end. Should they be v2?

[erotavlas]

I think I have it working now. There was nothing wrong with the pyinstaller exe. I think the problem was that I used the config from version 3.0.5 to train the 3.1.1 models, so to fix I updated the config file using the command

python -m spacy init fill-config config-v3.0.cfg config-v3.1.cfg

However it still didn't change the MultiHashEmbed to v2 so I manually changed it to v2 in the config.
Then retrained the model.

I was then able to load the model using the pyinstaller exe I created.

[mbrunecky]

Changing config to specify v2 solved the problem for that particular model, but that means your PyInstaller is still not including the
spacy_legacy module.
My PyInstaller comes with contributed hook-spacy.py, but that hook does not import spacy_legacy either.
I got around the problem by adding my own hook_spacy.py where I import spacy_legacy.

A similar problem is with transformers. Importing spacy_transformers does not do the trick. So I imported transformers directly - which
did not help either. Again, I had to write my own (hacky) hook-tranformer.py, and import about 8? transformer imports.