setting pymorphy2_lemmatize vs pymorphy2_lookup_lemmatize in UkrainianLemmatizer

[DSLituiev]

I would like to run UkrainianLemmatizer().pymorphy2_lookup_lemmatize() instead of UkrainianLemmatizer().pymorphy2_lemmatize() , as the latter mis-handles lemmatization of certain pronouns.

I can't quite find where it is called. I cannot see pymorphy2_lemmatize mentioned in the config (in "uk_core_news_lg" model), or being called in either UkrainianLemmatizer or RussianLemmatizer, but assume it can be changed by re-setting the config "lemmatizer": {'mode': 'pymorphy2', ...} to "lemmatizer": {'mode': 'pymorphy2_lookup',}.

Here is my attempt that fails:

nlp = spacy.load("uk_core_news_lg")

nlp("якусь")[0].lemma_
# "якусь"

dict(nlp.pipeline)["lemmatizer"].pymorphy2_lemmatize(nlp("якусь")[0])
#  ['якусь']

dict(nlp.pipeline)["lemmatizer"].pymorphy2_lookup_lemmatize(nlp("якусь")[0])
# ['якийсь']

# remove un-used pipeline
nlp.remove_pipe("ner")

old_lemmatizer = nlp.remove_pipe("lemmatizer")

lemmatizer = nlp.add_pipe("lemmatizer", config=
{ #'factory': 'lemmatizer',
 'mode': 'pymorphy2_lookup',
 'model': None,
 'overwrite': True,
  # 'scorer': {'@scorers': 'spacy.lemmatizer_scorer.v1'}
 })


nlp("якусь")[0].lemma_

  File "/opt/anaconda3/envs/asr/lib/python3.10/site-packages/spacy/lang/ru/lemmatizer.py", line 102, in pymorphy2_lookup_lemmatize
    analyses = self._morph.parse(string)

AttributeError: 'UkrainianLemmatizer' object has no attribute '_morph'

I tried adding nlp.initialize() but that does not seem to help.

This also does not seem to help lemmatizer._morph = old_lemmatizer[1]._morph as it returns empty lemmas: nlp("якусь")[0].lemma_

This may relate to a complexity of the RussianLemmatizer initalization

Would you advise how to initialize it properly in this setting?

[adrianeboyd]

Thanks for the report! I think the problem is that the pymorphy2_lookup mode isn't accounted for in __init__ in either lemmatizer.