BERTopic (Can't retrieve unregistered extension attribute 'trf_data'. Did you forget to call the set_extension method?)

[FranValero97]

Good morning, this is my code obtained from
the following page: https://spacy.io/universe/project/bertopic after running it I get the following error: Can't retrieve unregistered extension attribute 'trf_data'. Did you forget to call the set_extension method?

How can I solve this error?

Instalación de las bibliotecas necesarias

!pip install spacy
!pip install bertopic
!pip install scikit-learn

Descargar el modelo de spaCy en inglés (medium)

!python -m spacy download en_core_web_md

Cargar las bibliotecas y el modelo

import spacy
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

Cargar los documentos de la base de datos de 20 Newsgroups

docs = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))['data']

Cargar el modelo de spaCy en inglés (medium) excluyendo componentes innecesarios

nlp = spacy.load('en_core_web_md', exclude=['tagger', 'parser', 'ner', 'attribute_ruler', 'lemmatizer'])

Crear el modelo BERTopic con spaCy

topic_model = BERTopic(embedding_model=nlp)
topics, probs = topic_model.fit_transform(docs)

Visualizar los tópicos

fig = topic_model.visualize_topics()
fig.show()

[shadeMe]

You'll need to use one of spaCy's Transformer models in conjunction with BERTopic. For instance,

• Install the en_core_web_trf model: python -m spacy download en_core_web_trf.
• Load and use that as the pipeline: nlp = spacy.load('en_core_web_trf', exclude=['tagger', 'parser', 'ner', 'attribute_ruler', 'lemmatizer'])

[shadeMe]

Furthermore, spaCy v3.4.0 changed how the vectors from the transformer were stored in the Doc object. Since BERTopic hasn't been updated to accommodate those changes, you'd want to use spacy<3.4.0 with the above code.

[FranValero97]

I have tried to do what you told me and I keep getting the same error, I have tried changing the version of spacy to one that is between version 3.3.0 and version 3.4.0, I still get the same error trying all of them spacy models (sm, md, lg, trf)

[shadeMe]

Hmm, if the problem continues to persist, I'd suggest opening an issue in the BERTopic GitHub repository as this most likely needs to be rectified on their end.