sense2vec factory issue

[sdspieg]

Following this blog item in Google Colab (with high ram) to try out sense2vec, I run into this error. Any suggestions on how to fix this?

Here is the code cell:

nlp = spacy.load("en_core_web_lg")
s2v = Sense2VecComponent(nlp.vocab).from_disk("/content/drive/MyDrive/RuBase/Corpora/sense2vec/s2v_reddit_2019_lg")
nlp.add_pipe("s2v")`

And here's the error message:

ValueError                                Traceback (most recent call last)
[<ipython-input-7-7af9fb234ba5>](https://localhost:8080/#) in <module>()
      2 nlp = spacy.load("en_core_web_lg")
      3 s2v = Sense2VecComponent(nlp.vocab).from_disk("/content/drive/MyDrive/RuBase/Corpora/sense2vec/s2v_reddit_2019_lg")
----> 4 nlp.add_pipe("s2v")

1 frames
[/usr/local/lib/python3.7/dist-packages/spacy/language.py](https://localhost:8080/#) in create_pipe(self, factory_name, name, config, raw_config, validate)
    653                 lang_code=self.lang,
    654             )
--> 655             raise ValueError(err)
    656         pipe_meta = self.get_factory_meta(factory_name)
    657         # This is unideal, but the alternative would mean you always need to

ValueError: [E002] Can't find factory for 's2v' for language English (en). This usually happens when spaCy calls `nlp.create_pipe` with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator `@Language.component` (for function components) or `@Language.factory` (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, doc_cleaner, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, spancat, textcat_multilabel, sense2vec, en.lemmatizer

[koaning]

It may be that you're running into a version mismatch. What version of spaCy and sense2vec are you using?

The sense2vec docs mention the usage for spaCy v3, if you're interested in using an older version, you should check this branch of the project.

[sdspieg]

Thanks for that quick response. I am using that page and am trying to replicate it in Google colab. So I did indeed have spaCy 2.2.4 . I uninstalled it and python -m spacy info now tells me I am running 3.2.4
But I still get the same error

[<ipython-input-4-7af9fb234ba5>](https://localhost:8080/#) in <module>()
      2 nlp = spacy.load("en_core_web_lg")
      3 s2v = Sense2VecComponent(nlp.vocab).from_disk("/content/drive/MyDrive/RuBase/Corpora/sense2vec/s2v_reddit_2019_lg")
----> 4 nlp.add_pipe("s2v")

1 frames
[/usr/local/lib/python3.7/dist-packages/spacy/language.py](https://localhost:8080/#) in create_pipe(self, factory_name, name, config, raw_config, validate)
    653                 lang_code=self.lang,
    654             )
--> 655             raise ValueError(err)
    656         pipe_meta = self.get_factory_meta(factory_name)
    657         # This is unideal, but the alternative would mean you always need to

ValueError: [E002] Can't find factory for 's2v' for language English (en). This usually happens when spaCy calls `nlp.create_pipe` with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator `@Language.component` (for function components) or `@Language.factory` (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, doc_cleaner, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, spancat, textcat_multilabel, sense2vec, en.lemmatizer```

[sdspieg]

Thank you. But in the meanwhile, could you still please tell me (/us) how I (/we) can input text strings and obtain semantically similar text spans? As in the blog entry

assert doc[3:6].text == "natural language processing"
freq = doc[3:6]._.s2v_freq
vector = doc[3:6]._.s2v_vec
most_similar = doc[3:6]._.s2v_most_similar(3)
# [(('machine learning', 'NOUN'), 0.8986967),
#  (('computer vision', 'NOUN'), 0.8636297),
#  (('deep learning', 'NOUN'), 0.8573361)]```

[koaning]

One small thing, could you apply markdown formatting on codeblocks? That makes it much easier to read. You can apply markdown formatting by editing your comment cell and by wrapping your code with three ticks "```".

Here's an example;

This is normal text. 

```python
# This cell contains python code

doc = nlp("A sentence about natural language processing.")
```

[koaning]

But in the meanwhile, could you still please tell me (/us) how I (/we) can input text strings and obtain semantically similar text spans?

I believe your code already does that if you open up a new cell with just this variable:

most_similar

... then when you run the cell it should print the variable contents, which in this case I expect to look something like:

[(('machine learning', 'NOUN'), 0.8986967),
  (('computer vision', 'NOUN'), 0.8636297),
  (('deep learning', 'NOUN'), 0.8573361)]

[sdspieg]

One small thing, could you apply markdown formatting on codeblocks? That makes it much easier to read. You can apply markdown formatting by editing your comment cell and by wrapping your code with three ticks "```".

I now did. But I was just following what I saw in the editing ribbon, and that just suggests one tick. Maybe that could be fixed?

[sdspieg]

I believe your code already does that if you open up a new cell with just this variable:

most_similar

Thanks Vincent. That did the trick...

Maybe the team could consider adding that one line to the docu?
So now I'll have to start digging into the documentation on how we can start training our own vectors and how to use this to generate noun phrases and bootstrapping NER annotations in Prodigy.
I so hope that people will also start making and sharing FULL Google colabs or Jupyter notebooks for all of these things, where noobs like me can then FIRST successfully run the entire demo on their own systems to see what happens and THEN plug in their own data/corpora to obtain similar results in the fields they are working in. That would IMO REALLY mean 'democratization'...