How lemmatizer works in spaCy?

[teohsinyee]

I lemmatized some words using
example["new"] = example["ori"].apply(lambda row: " ".join([w.lemma_ for w in nlp(row)]))
However, I can't understand the logic behind it. Please refer to the table below.

No.	ori	new
0	Co-Curricular Activities	Co - Curricular Activities
1	WORK EXPERIENCES	work experience
2	Seminar and extra curricular activities:	seminar and extra curricular activity :
3	ACADEMIC QUALIFICATIONS	ACADEMIC qualification
4	Seminars/Symposiums attended	seminar / symposium attend

1. Why at row 0, Activities is not being lemmatized? While row 1, EXPERIENCES is lemmatized?
2. Why EXPERIENCES has converted to lowercase?
3. At row 3, I expect the lemmatized words to be 'Academic qualification'. Why is ACADEMIC still all in UPPER case? Because those all UPPER case word has converted to lower case (Refer to row 1).

I was trying to figure out the pattern of the lemmatizer but it seems inconsistent in my case.

[thomashacker]

Hello, the English models use a rule-based lemmatizer based on the POS, but POS can be incorrect, or the rules might not be 100% correct in all cases. The accuracy also depends on whether you run the lemmatizer on short paragraphs or whole sentences. Here, you can read more about how the lemmatizer works and how the token.pos influences the results.