-
-
Notifications
You must be signed in to change notification settings - Fork 4.6k
Description
Using bert-base-uncased instead of default roberta-base inside an en_core_web_trf based pipeline
On loading the en_core_web_trf model using spacy.load, the nlp.pipeline has the default transformer component (based on roberta-base) loaded before the tagger. I want to replace it with the bert-base-uncased model. I tried the following steps, which seem to give no error. But when parsing a sentence using nlp, it seems to raise a "NoneType object isn't callable" error.
-
Removing the transformer component
nlp.remove_pipe("transformer")
nlp.pipeline (shows the transformer component removed) -
Creating a config (based on the example on https://spacy.io/api/transformer)
config = { "model": { "@architectures": "spacy-transformers.TransformerModel.v3", "name": "D:\\Work\\Tools\\bert-base-uncased", "tokenizer_config": {"use_fast": True}, "transformer_config": {"output_attentions": True}, "mixed_precision": True, "grad_scaler_config": {"init_scale": 32768} } } -
Adding the desired transformer model (using the config) in the pipeline
trf = nlp.add_pipe("transformer", config=config, before="tagger") -
Checking nlp.pipeline (Shows a new transformer as part of the pipeline)
-
Checking on a sentence leads to a NoneType object isn't callable error
s = 'John went to London.'
doc = nlp(s)
Traceback (most recent call last):
File "", line 1, in
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\language.py", line 1018, in call
error_handler(name, proc, [doc], e)
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\util.py", line 1618, in raise_error
raise e
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy\language.py", line 1013, in call
doc = proc(doc, **component_cfg.get(name, {})) # type: ignore[call-arg]
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\pipeline_component.py", line 192, in call
outputs = self.predict([doc])
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\pipeline_component.py", line 228, in predict
activations = self.model.predict(docs)
File "D:\Work\Tools\envs\env_main2\lib\site-packages\thinc\model.py", line 315, in predict
return self._func(self, X, is_train=False)[0]
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\layers\transformer_model.py", line 177, in forward
batch_encoding = huggingface_tokenize(tokenizer, [span.text for span in flat_spans])
File "D:\Work\Tools\envs\env_main2\lib\site-packages\spacy_transformers\layers\transformer_model.py", line 278, in huggingface_tokenize
padding="longest",
TypeError: 'NoneType' object is not callable
Assuming the bert-base-uncased model is available and downloaded in a suitable local directory, is this a right way to use a transformer model different from the default roberta-base? If not, is there a better way to achieve the desired setting? And if this is a possible way, can some light be thrown on why the NoneType error?
Your Environment
- spaCy version: 3.2.0
- Platform: Windows-10-10.0.14393-SP0
- Python version: 3.7.11
- Pipelines: en_core_web_md (3.2.0), en_core_web_trf (3.2.0)