"storytelle" is not Lemma

[pejmannavi]

Hi

I tested this sentence : "What is storytelling game?"
token.lemma_ result : [what, be, storytelle, game, ?]
token.pos_ result : [PRON, AUX, VERB, NOUN, PUNCT]
I think this result is wrong. Storytelling is noun and storytelle is incorrect.

PS : My spaCy version is 3.0+

[adrianeboyd]

The English models use a rule-based lemmatizer based on the POS, but POS can be incorrect or the rules might not be 100% correct in all cases.

In this case, I think the POS is probably incorrect due to the missing article ("a storytelling game") and if I add the article with en_core_web_sm (v3.4.0) I do get storytelling/NOUN.

If this is a frequent case or a problem for you task, you can add exceptions to the lemmatizer like this, here an individual exception for "storytelling" as a verb:

nlp = spacy.load("en_core_web_sm")
nlp.get_pipe("lemmatizer").lookups.get_table("lemma_exc")["verb"]["storytelling"] = ["storytelling"]

There's a lemmatizer cache, so you need to do this before processing any texts with nlp() or you won't see the effects of your changes.

You can save this model with nlp.to_disk() and it will save your modifications with the rest of the model.

[pejmannavi]

Thanks Adrian.
I will use your solution.