Is it possible to update NER and tok2vec components simultaneously? Error debug model

[lzmodel]

Hello,
I'm trying to use spaCy Project: Demo updating an NER component in a pretrained pipeline to train with my English law text data. Some of my entities are the same as in model en_core_web_lg, and I have 1 new ner. I've read discussions like Updating NER model #8425, Is It Possible to Resume Training Via CLI in Spacy v3 (transformers)? #8176, and created config.

`[paths]
train = "./train/data/v2104/train.spacy"
dev = "./train/data/v2104/dev.spacy"
labels = "./train/labels_spacy/ner.json"
vectors = "en_core_web_lg"
init_tok2vec = "en_core_web_lg"

[variables]
wandb_project_name = "Agr_NER_difmod"
wandb_run_name = "cpu2111-lgvec-lgtok-upd"

[system]
gpu_allocator = "pytorch"
seed = 0

[nlp]
lang = "en"
pipeline = ["tok2vec","ner"]
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
batch_size = 32
#batch_size = 256
tokenizer = {"@Tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.ner]
source = "en_core_web_lg"

[components.tok2vec]
source = "en_core_web_lg"

[corpora]

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
gold_preproc = false
max_length = 0
limit = 0
augmenter = null

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
gold_preproc = false
max_length = 0
limit = 0
augmenter = null

[training]
train_corpus = "corpora.train"
dev_corpus = "corpora.dev"
seed = ${system:seed}
gpu_allocator = ${system:gpu_allocator}
dropout = 0.1
accumulate_gradient = 1
patience = 1600
max_epochs = -1
max_steps = 20000
eval_frequency = 200
frozen_components = []
before_to_disk = null
annotating_components = []

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2
get_length = null

[training.batcher.size]
@schedules = "compounding.v1"
start = 100
stop = 1000
compound = 1.001
t = 0.0

[training.logger]
@Loggers = "spacy.WandbLogger.v3"
project_name = ${variables.wandb_project_name}
run_name = ${variables.wandb_run_name}
remove_config_values = []
model_log_interval = null
log_dataset_dir = null
entity = null

[training.optimizer]
@optimizers = "Adam.v1"
beta1 = 0.9
beta2 = 0.999
L2_is_weight_decay = true
L2 = 0.01
grad_clip = 1.0
use_averages = true
eps = 0.00000001
learn_rate = 0.001

[training.score_weights]
ents_f = 1.0
ents_p = 0.0
ents_r = 0.0
ents_per_type = null
speed = 0.0

[pretraining]

[initialize]
vocab_data = null
vectors = null
init_tok2vec = ${paths.init_tok2vec}
after_init = null
lookups = null

[initialize.before_init]
@callbacks = "spacy.copy_from_base_model.v1"
tokenizer = "en_core_web_lg"
vocab = "en_core_web_lg"

[initialize.components]

[initialize.components.ner]

[initialize.components.ner.labels]
@readers = "spacy.read_labels.v1"
path = ${paths.labels}
require = false

[initialize.tokenizer]`

But I get the same error when I run debug data or debug model commands.

python -m spacy debug model ./train/cfg_spc/cpu-lgvec-lgtok-upd.cfg ner --gpu-id 0
ℹ Using GPU: 0
ℹ Fixing random seed: 0
ℹ Analysing model with ID 199
Traceback (most recent call last):
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/spacy/main.py", line 4, in
setup_cli()
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/spacy/cli/_util.py", line 71, in setup_cli
command(prog_name=COMMAND)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/typer/main.py", line 532, in wrapper
return callback(**use_params) # type: ignore
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/spacy/cli/debug_model.py", line 78, in debug_model_cli
debug_model(config, T, nlp, pipe, print_settings=print_settings)
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/spacy/cli/debug_model.py", line 115, in debug_model
nlp.initialize(lambda: train_corpus(nlp))
File "/home/mel/anaconda3-p3.9/envs/spacy_gpu/lib/python3.9/site-packages/spacy/language.py", line 1317, in initialize
proc.initialize(get_examples, nlp=self, **p_settings)
File "spacy/pipeline/transition_parser.pyx", line 557, in spacy.pipeline.transition_parser.Parser.initialize
File "spacy/pipeline/_parser_internals/transition_system.pyx", line 169, in spacy.pipeline._parser_internals.transition_system.TransitionSystem.initialize_actions
TypeError: '<' not supported between instances of 'int' and 'str'

What I did wrong? Thanks a lot for helping me.

spaCy version 3.4.1
Platform Linux-5.15.0-48-generic-x86_64-with-glibc2.35
Python version 3.9.13

[adrianeboyd]

I think this is a misunderstanding of the pipeline design (https://spacy.io/models#design).

The ner component in en_core_web_lg does not depend on the tok2vec component, so you can source ner and update it by itself. Remove the tok2vec component and init_tok2vec from your config and try training again? You will still need the en_core_web_lg vectors settings.

[lzmodel]

@adrianeboyd Thanks for the quick response. I changed the config as follows, but the error remains the same.

[paths]

train = "./train/data/v2111/train.spacy"
dev = "./train/data/v2111/dev.spacy"
labels = "./train/labels_spacy/ner.json"
vectors = "en_core_web_lg"
init_tok2vec = null

[variables]
wandb_project_name = "Agr_NER_difmod"
wandb_run_name = "cpu2111-lgvec-upd"

[system]
gpu_allocator = "pytorch"
seed = 0

[nlp]
lang = "en"
pipeline = ["ner"]
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
batch_size = 32
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components]

[components.ner]
source = "en_core_web_lg"
component = "ner"

[corpora]
... ...
[pretraining]

[initialize]
vocab_data = null
vectors = ${paths.vectors}
init_tok2vec = null
after_init = null
lookups = null
before_init = null

[initialize.components]

[initialize.components.ner]

[initialize.components.ner.labels]
@readers = "spacy.read_labels.v1"
path = ${paths.labels}
require = false

[initialize.tokenizer]

I also tried adding a section

[initialize.before_init]
@callbacks = "spacy.copy_from_base_model.v1"
tokenizer = "en_core_web_lg"
vocab = "en_core_web_lg"

but it didn't change anything

[adrianeboyd]

Try it without the [initialize.components.ner] block (so it reads the labels from your training data instead of ner.json). You never rarely have to do the label initialization separately, it's mainly helpful if you have a large training corpus and you don't want to initialize from the data repeatedly for multiple training runs.

If you do want to keep the label initialization step, double-check the contents of ner.json and if needed, regenerate the labels with spacy init labels.