use_gold_ents error when using EntityLinker.v1

[kudryk-enverus]

On my project we are using spaCy 3.3.0 but still using the EntityLinker.v1. The configuration for it in our NER model is as follows:

[components.entity_linker]
factory = "entity_linker"
entity_vector_length = 64
get_candidates = {"@misc":"oi.LowercaseCandidateGenerator.v1"}
incl_context = true
incl_prior = false
labels_discard = []
n_sents = 0
overwrite = false
scorer = {"@scorers":"spacy.entity_linker_scorer.v1"}

[components.entity_linker.model]
@architectures = "spacy.EntityLinker.v1"
nO = null

After the already built NER model is installed and I'm assembling it into NLP model, I get the following error:

================================== assemble ==================================
Running command: /usr/local/bin/python -m spacy assemble configs/config.cfg training/model-best --paths.train ../ner/corpus/train/ --paths.dev ../ner/corpus/dev/ --paths.data data --paths.kb_data ../ner/assets/kb/ --paths.packs ../packs --code ../src/initialize.py

=========================== Initializing pipeline ===========================
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/usr/local/lib/python3.9/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/typer/main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "/usr/local/lib/python3.9/site-packages/spacy/cli/assemble.py", line 45, in assemble_cli
    nlp = load_model_from_config(config, auto_fill=True)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 528, in load_model_from_config
    nlp = lang_cls.from_config(
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 1804, in from_config
    source_nlps[model] = util.load_model(
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 420, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 453, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "/usr/local/lib/python3.9/site-packages/en_enverus_oi_ner/__init__.py", line 10, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 619, in load_model_from_init_py
    return load_model_from_path(
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 488, in load_model_from_path
    nlp = load_model_from_config(
  File "/usr/local/lib/python3.9/site-packages/spacy/util.py", line 528, in load_model_from_config
    nlp = lang_cls.from_config(
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 1783, in from_config
    nlp.add_pipe(
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 792, in add_pipe
    pipe_component = self.create_pipe(
  File "/usr/local/lib/python3.9/site-packages/spacy/language.py", line 674, in create_pipe
    resolved = registry.resolve(cfg, validate=validate)
  File "/usr/local/lib/python3.9/site-packages/thinc/config.py", line 746, in resolve
    resolved, _ = cls._make(
  File "/usr/local/lib/python3.9/site-packages/thinc/config.py", line 795, in _make
    filled, _, resolved = cls._fill(
  File "/usr/local/lib/python3.9/site-packages/thinc/config.py", line 850, in _fill
    filled[key], validation[v_key], final[key] = cls._fill(
  File "/usr/local/lib/python3.9/site-packages/thinc/config.py", line 916, in _fill
    raise ConfigValidationError(
thinc.config.ConfigValidationError: 

Config validation error

entity_linker -> use_gold_ents   extra fields not permitted

{'nlp': <spacy.lang.en.English object at 0x7f9f27b5f970>, 'name': 'entity_linker', 'entity_vector_length': 64, 'get_candidates': {'@misc': 'oi.LowercaseCandidateGenerator.v1'}, 'incl_context': True, 'incl_prior': False, 'labels_discard': [], 'model': {'@architectures': 'spacy.EntityLinker.v1', 'nO': None, 'tok2vec': {'@architectures': 'spacy.Tok2Vec.v2', 'embed': {'@architectures': 'spacy.MultiHashEmbed.v2', 'width': 96, 'attrs': ['NORM', 'PREFIX', 'SUFFIX', 'SHAPE', 'SPACY'], 'rows': [5000, 2500, 2500, 2500, 50], 'include_static_vectors': True}, 'encode': {'@architectures': 'spacy.MaxoutWindowEncoder.v2', 'width': 96, 'depth': 4, 'window_size': 1, 'maxout_pieces': 3}}}, 'n_sents': 0, 'overwrite': False, 'scorer': {'@scorers': 'spacy.entity_linker_scorer.v1'}, 'use_gold_ents': True, '@factories': 'entity_linker'}

If I examine the installed NER package, the use_gold_ents attribute is in config.cfg whereas above it was absent.

[components.entity_linker]
factory = "entity_linker"
entity_vector_length = 64
get_candidates = {"@misc":"oi.LowercaseCandidateGenerator.v1"}
incl_context = true
incl_prior = false
labels_discard = []
n_sents = 0
overwrite = false
scorer = {"@scorers":"spacy.entity_linker_scorer.v1"}
use_gold_ents = true

[components.entity_linker.model]
@architectures = "spacy.EntityLinker.v1"
nO = null

If I then manually comment out that line, only then will our model successfully assemble.

As use_gold_ents is only for v2 of the entity linker, how is this being inserted?

[polm]

Sorry you've run into trouble with this. I need to investigate further, but I think what is happening is that spacy assemble is filling in default config values. The intent of this is to avoid having hidden defaults in your config at the time it's created, the downside is that in cases like this where values were added it can break things.

Separate from that issue though, have you tried upgrading to v2 of the EntityLinker? The only difference in behavior should be that v2 supports a tok2vec with multiple listeners - there shouldn't be any actual change in predictions or anything. You can upgrade by just changing the line in your config.

[kudryk-enverus]

An attempt to upgrade to v2 was made by my team lead but had to set that work aside temporarily to address other priorities. Our intent is to upgrade to v2 eventually.

[polm]

If everything goes smoothly, I think you should be able to upgrade to v2 just by changing the lines in the config file - it shouldn't even be necessary to retrain or anything.

[polm]

So after looking in to exactly how this happens, it's a limitation of the way configs are generated when assembling models.

When doing assemble, any sourced components are added using add_pipe. One thing add_pipe does is merge in the default config. (This is technically covered in the dev docs on the Language class.)

While the model is versioned, code for pipeline components is not versioned, and so the default config is always the most recent one, and that will cause issues like this if fields have been added.

I'll see if we can come up with a better fix, but for now I'm afraid the best workaround is to delete the value from the config.

[polm]

Also, I checked using the example Emerson NER project, and unfortunately you can't just change the config and reload the existing model, you'll have to retrain. However the new architecture shouldn't require any modifications to your data or anything.