Custom components changes between spacy 3.1.2 and 3.1.3+

[databill86]

Hello,

I have a lot of custom components that were perfectly working on spacy==3.1.2, but after upgrading to any 3.1.3+ version, I've got a lot a thrown exceptions:

[E893] Could not find function 'my_custom_func' in function registry 'factories'. If you're using a custom function, make sure the code is available. If the function is provided by a third-party package, e.g. spacy-transformers, make sure the package is installed in your environment.
Available names: 'fr.my_custom_func'...

Each time I try to rerun the build, I get a different 'custom_func' component name.

Does this have any relation to those two changes made here ?

Fix issue #9009: Include component factories in third-party dependencies resolver.
Fix issue #9012: Correct type of config in create_pipe.

Here is how I registered my components:

""" my_custom_pipelines.py """

@French.factory("my_custom_func", default_config={"language": None,
                                                          ...
                                                          "verbose": False})
def create_custom_func(nlp: Language,
                                  name: str,
                                  language: str,
                                  ....
                                  verbose: bool = False):
    return CustomFunc(nlp, language=language,
                                ....
                                verbose=verbose)

Then I save the model to disk:

""" my_build_script.py """
import my_custom_pipelines

nlp_extr = spacy.load("fr_core_news_lg") 
nlp_extr.add_pipe("my_custom_func", config={"verbose": False, "language": self.language,
                                                       ....},
                          after="lemmatizer")

nlp_extr.to_disk()

And finally I run:

python -m spacy package models/fr models/mymodel --name=my_model --version=3.9.10 --code=my_custom_pipelines.py

Am I missing something ?

Again, there is nothing that I changed between spacy==3.1.2 and 3.1.3+

I used fr_core_news_lg-3.1.0 for the packaging.

python -m spacy download -d fr_core_news_lg-3.1.0

[databill86]

FYI: I restarted everything from a clean environment (spacy 3.1.2), and now, even with spacy==3.1.2, I'm having problems to build the package, I'm getting:

atalogue.RegistryError: Cant't find 'spacy.WandbLogger.v1' in registry spacy -> loggers. Available names: spacy-legacy.WandbLogger.v1, spacy.ConsoleLogger.v1, spacy.WandbLogger.v2

Is there a way to disable the WandbLogger ?

[training.logger]
@loggers = "spacy.WandbLogger.v1"
project_name = "spacy-v3.0.0a2"
remove_config_values = []

Thanks

[databill86]

In the new version 3.2.0, @Language.factory works.

[databill86]

Thank you @polm for your response.

Actually the original problem was the packaging of model with: python -m spacy package ...

I've created a simple project that reproduce the issue:

https://github.com/databill86/spacy-tests

As you may see in the file src/my_pipelines.py

@French.factory("custom_sent")  # doesn't work for packaging
# @Language.factory("custom_sent")  # work
def my_senter(nlp, name):
    return CustomSenter(nlp)

The packaging python -m spacy package models/my_model models/packaged --name=custom_packaged_spacy --version=0.0.1 --code=src/my_pipelines.py only work when we use @Language.factory instead of @French.factory.

This is the thrown error:

catalogue.RegistryError: [E893] Could not find function 'custom_sent' in function registry 'factories'. If you're using a custom function, make sure the code is available. If the function is provided by a third-party package, e.g. spacy-transformers, make sure the package is installed in your environment.
Available names: attribute_ruler, beam_ner, beam_parser, entity_linker, entity_ruler, fr.custom_sent, fr.lemmatizer, lemmatizer, merge_entities, merge_noun_chunks, merge_subtokens, morphologizer, ner, parser, sentencizer, senter, sp
ancat, tagger, textcat, textcat_multilabel, tok2vec, token_splitter

[polm]

Thank you, I didn't realize the issue was spacy package specific. Your sample repo illustrates the issue clearly and it looks like a bug. I'll see if I can figure out the details.

[databill86]

Thank you!

[polm]

Not sure if this is a final fix, but there's a PR in #9674 and you can follow development there.

[databill86]

This fixes the problem. I hope that's the right fix and that it will be merged soon.

Thank you again.