Using bert-base-uncased instead of default roberta-base inside an en_core_web_trf based pipeline

[nramrakhiyani]

Using bert-base-uncased instead of default roberta-base inside an en_core_web_trf based pipeline

On loading the en_core_web_trf model using spacy.load, the nlp.pipeline has the default transformer component (based on roberta-base) loaded before the tagger. I want to replace it with the bert-base-uncased model. I tried the following steps, which seem to give no error. But when parsing a sentence using nlp, it seems to raise a "NoneType object isn't callable" error.

1. Removing the transformer component
   nlp.remove_pipe("transformer")
nlp.pipeline (shows the transformer component removed)

2. Creating a config (based on the example on https://spacy.io/api/transformer)
   config = { "model": { "@architectures": "spacy-transformers.TransformerModel.v3", "name": "D:\\Work\\Tools\\bert-base-uncased", "tokenizer_config": {"use_fast": True}, "transformer_config": {"output_attentions": True}, "mixed_precision": True, "grad_scaler_config": {"init_scale": 32768} } }

3. Adding the desired transformer model (using the config) in the pipeline
   trf = nlp.add_pipe("transformer", config=config, before="tagger")

4. Checking nlp.pipeline (Shows a new transformer as part of the pipeline)

5. Checking on a sentence leads to a NoneType object isn't callable error
   s = 'John went to London.'
doc = nlp(s)

Traceback (most recent call last):
File "", line 1, in
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\language.py", line 1018, in call
error_handler(name, proc, [doc], e)
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\util.py", line 1618, in raise_error
raise e
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\language.py", line 1013, in call
doc = proc(doc, **component_cfg.get(name, {})) # type: ignore[call-arg]
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\pipeline_component.py", line 192, in call
outputs = self.predict([doc])
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\pipeline_component.py", line 228, in predict
activations = self.model.predict(docs)
File "D:\Work\Tools\envs\env_main2\lib\site-packages\thinc\model.py", line 315, in predict
return self._func(self, X, is_train=False)[0]
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\layers\transformer_model.py", line 177, in forward
batch_encoding = huggingface_tokenize(tokenizer, [span.text for span in flat_spans])
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\layers\transformer_model.py", line 278, in huggingface_tokenize
padding="longest",
TypeError: 'NoneType' object is not callable

Assuming the bert-base-uncased model is available and downloaded in a suitable local directory, is this a right way to use a transformer model different from the default roberta-base? If not, is there a better way to achieve the desired setting? And if this is a possible way, can some light be thrown on why the NoneType error?

Your Environment

• spaCy version: 3.2.0
• Platform: Windows-10-10.0.14393-SP0
• Python version: 3.7.11
• Pipelines: en_core_web_md (3.2.0), en_core_web_trf (3.2.0)

[brgsk]

facing same issue

[polm]

Sorry you're having trouble with this. We just added an FAQ post about changing the base Transformer model you use.

In this case the important thing is that you can't change the underlying Transformer model and keep using a trained pipeline - you'd have to retrain it. Changing the base without changing the downstream layers is like cutting a branch off a rosebush and trying to graft it onto a cow - they're just not compatible and you'll get errors or nonsense output.

If you want to use a different base model, you'll need to train a new pipeline.

[nramrakhiyani]

Thanks a lot for your reply. And my bad on not understanding the pipeline component usage and training completely. Also Thanks for the FAQ post, it clears multiple things.

[polm]

No worries - a lot of people have been asking about this, which is why we felt the need to make the docs clearer and add an FAQ post.