ValueError: Mismatched IDs! 57406478778838888474 vs 4311360199468911749078

[mashagua]

the error occured when i run these code:

with nlp.disable_pipes(*other_pipes) :

  # sizes = compounding(1.0, 4.0, 1.001)
  # Training for 30 iterations
  for itn in range(35):
    # shuffle examples before training
    random.shuffle(TRAIN_DATA)
    # batch up the examples using spaCy's minibatch
    batches = minibatch(TRAIN_DATA, size=2)
    # ictionary to store losses
    losses = {}
    for batch in batches:
        for text, annotations in batch:
            doc = nlp.make_doc(text)
            example = Example.from_dict(doc, annotations)
            # Update the model
            nlp.update([example], losses=losses, drop=0.4)
            print("Losses", losses)

the terminial

Traceback (most recent call last):
  File "D:/all_rasa_2.5.X/spacy_related/data_utils_v2.py", line 115, in <module>
    nlp.update([example], losses=losses, drop=0.4)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\spacy\language.py", line 1107, in update
    proc.update(examples, sgd=None, losses=losses, **component_cfg[name])
  File "spacy\pipeline\transition_parser.pyx", line 358, in spacy.pipeline.transition_parser.Parser.update
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\model.py", line 306, in begin_update
    return self._func(self, X, is_train=True)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\spacy\ml\tb_framework.py", line 31, in forward
    step_model = ParserStepModel(
  File "spacy\ml\parser_model.pyx", line 216, in spacy.ml.parser_model.ParserStepModel.__init__
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\model.py", line 288, in __call__
    return self._func(self, X, is_train=is_train)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\layers\chain.py", line 54, in forward
    Y, inc_layer_grad = layer(X, is_train=is_train)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\model.py", line 288, in __call__
    return self._func(self, X, is_train=is_train)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\layers\chain.py", line 54, in forward
    Y, inc_layer_grad = layer(X, is_train=is_train)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\thinc\model.py", line 288, in __call__
    return self._func(self, X, is_train=is_train)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\spacy_transformers\layers\listener.py", line 58, in forward
    model.verify_inputs(docs)
  File "D:\all_rasa_2.5.X\venv\lib\site-packages\spacy_transformers\layers\listener.py", line 51, in verify_inputs
    raise ValueError(f"Mismatched IDs! {batch_id} vs {self._batch_id}")
ValueError: Mismatched IDs! 57406478778838888474 vs 4311360199468911749078

Operating System::
spacy 3.0
python 3.8
transformers 4.5
model: zh_core_web_trf

• Python Version Used:
• spaCy Version Used:
• Environment Information:

[svlandeg]

Hi! As a maintenance note: this is a good question for the discussions forum, so I'll move it there. The original issue will be closed and will link through to the open discussion thread.

[svlandeg]

Could you provide the full code snippet? This will help us reproduce & debug the issue.

In particular, it would be good to see how the nlp object was constructed and which pipeline components were disabled. If the transformer was disabled, but another component is still listening to it, you might run into this type of error.

[svlandeg]

Thanks! I think your issue should be resolved if you change

pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]

into

pipe_exceptions = ["ner", "transformer"]

Hope that works!

[mashagua]

thank you,it works,but I have another question:why the loss is become bigger like this
Losses {'transformer': 0.8309147357940674, 'ner': 1.9628533341367143}
Losses {'transformer': 1.4531161189079285, 'ner': 3.5705724134391463}
Losses {'transformer': 2.876827299594879, 'ner': 5.072797097218728}
Losses {'transformer': 4.045475542545319, 'ner': 7.071713950772282}
Losses {'transformer': 18.164145052433014, 'ner': 13.063390056010467}
Losses {'transformer': 27.164434015750885, 'ner': 17.06334797555873}
Losses {'transformer': 29.452207624912262, 'ner': 21.005296076555254}
Losses {'transformer': 30.808163583278656, 'ner': 23.00526878180536}
Losses {'transformer': 36.469610154628754, 'ner': 27.834982085563674}
Losses {'transformer': 42.30055898427963, 'ner': 29.732546144481418}
Losses {'transformer': 43.436122596263885, 'ner': 31.52985998503349}
Losses {'transformer': 44.986505925655365, 'ner': 35.516283775778966}
Losses {'transformer': 46.441544353961945, 'ner': 39.50117567250458}
Losses {'transformer': 48.579690754413605, 'ner': 43.49610498644939}
Losses {'transformer': 57.57888299226761, 'ner': 47.49502006613342}
Losses {'transformer': 4.983771324157715, 'ner': 4.729535410298746}
Losses {'transformer': 6.152164101600647, 'ner': 6.7279290745581335}

[svlandeg]

This is likely due to the way you've annotated your data. There's a few things to check:

• You're starting from an already trained pipeline zh_core_web_trf which has been trained on a different dataset. It will take some time for this network to start tuning itself towards your new annotations. This means that some "burn in" period is to be expected - it would be good to check how the model behaves on the longer term (e.g. in your output it's already coming down again)
• An important topic to keep in mind is catastrophic forgetting. If you only feed new data into the model, it will start overfitting to this data after some time, and may "forget" how to produce predictions it had correct before. To avoid this, you typically want to mix in "old" annotations with your new ones, either by feeding in examples from the original dataset, or by making predictions with the original model and feeding those in as training data among your new examples
• If the loss value is not consistently going down (after some time), this may point to inconsistencies in the data annotation, but I can't really judge that as I don't read Chinese :(

[svlandeg]

Ah, and one more thing I noticed. You create batches of your training data, but then you update on each example one by one:

nlp.update([example], losses=losses, drop=0.4)

It would be better to collect a batch of example and train on them together:

nlp.update(examples, losses=losses, drop=0.4)

Also, the drop value seems quite high. While this may prevent overfitting, it also means that it will have a more difficult time to learn from the examples in the beginning.

[mashagua]

Ok,I'll try it again,thank you

[rdemorais]

I can see the same error, but I'm using the project.yml strategy. Still the same error with Bert-base-uncased.

The important areas from yml file:

[nlp]
lang = "en"
pipeline = ["transformer", "entity_linker"]
batch_size = 128
disabled = []
before_creation = null
after_creation = null
after_pipeline_creation = null
tokenizer = {"@tokenizers":"spacy.Tokenizer.v1"}

[components.transformer]
factory = "transformer"

[components.transformer.model]
@architectures = "spacy-transformers.TransformerModel.v3"
name = "bert-base-uncased"
tokenizer_config = {"use_fast": true}

[components.entity_linker]
factory = "entity_linker"
entity_vector_length = 768
get_candidates = {"@misc":"spacy.CandidateGenerator.v1"}
incl_context = true
incl_prior = true
labels_discard = []
n_sents = 1
overwrite = true
scorer = {"@scorers":"spacy.entity_linker_scorer.v1"}

[components.entity_linker.model]
@architectures = "spacy.EntityLinker.v1"
nO = null

[components.entity_linker.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0
upstream = "*"

[components.entity_linker.model.tok2vec.pooling]
@layers = "reduce_mean.v1"

[training]
train_corpus = "corpora.train"
dev_corpus = "corpora.dev"
seed = ${system.seed}
gpu_allocator = ${system.gpu_allocator}
dropout = 0.2
patience = 10000
eval_frequency = 200
accumulate_gradient = 2
max_epochs = 0
max_steps = 600
frozen_components = []
# annotating_components = []
before_to_disk = null

The KB generation is like that:

  for qid, desc in desc_dict.items():
    desc_doc = nlp(desc)
    trf_vector = []
    lengths = desc_doc._.trf_data.align.lengths
    for i, token in enumerate(desc_doc):
      for len_idx in range(lengths[token.i]):
        trf_vector.append(desc_doc._.trf_data.tensors[0][0][token.i + len_idx])
    trf_vector = np.array(trf_vector)
    vector = np.average(trf_vector, axis=0)
    kb.add_entity(entity=qid, entity_vector=vector, freq=342)
    count += 1
    if count % 100 == 0:
      msg.info(f'{qid} added')

any ideias?

[polm]

How did you generate your config? The v1 EntityLinker had a bug with listeners that could cause this. You should be able to avoid it by using @architectures = "spacy.EntityLinker.v2".

[rdemorais]

that worked like a charm! Thank you very much.

About the config generation, I took the NEL Nelson example and change the areas related to transformers.

No I'm getting CUDA out of memory. Tried to allocate 1.99 GiB (GPU 0; 15.90 GiB total capacity; 13.31 GiB already allocated; 1.09 GiB free; 13.91 GiB reserved in total by PyTorch) but I believe I should change something related to batch/minibach.

Thanks!

[polm]

Since that's a separate issue can you open a new thread for it and link back here for context?

[rdemorais]

yes, sure, thank you.