Morphological features - tagset for English

[k-sap]

For a given language, I can get a set of possible tags by:

nlp = spacy.load('en_core_web_lg')
components = {name: component for name, component in nlp.components}
morphological_features = components['morphologizer'].labels

However, for some languages, e.g. for the English, there is no morphologizer components in the pipeline. Is there a different way to get these tags for English?

[polm]

Which tags do you want exactly? You can get the fine-grained part of speech tags, which are based on the Penn Treebank, by checking the labels of the tagger component using your snippet above. You can also find those listed on the model page, under "label scheme" (it's collapsed by default).

Some languages have features like whether a noun is plural or tense information, but English doesn't have features like that, though some of it is encoded in the fine-grained tags.

[polm]

Looking at the context in #9648, I see you're looking specifically for the morphological features. English and many other languages simply don't have those in their training data.

[polm]

Sorry, let me correct myself - we do have that information for English, but it's not set by a statistical component. Instead it's assigned based on the fine-grained tags using an AttributeRuler. You can check how the assignment works by checking the patterns of the AttributeRuler, for example:

import spacy
nlp = spacy.load('en_core_web_sm')
components = {name: component for name, component in nlp.components}

aruler = components["attribute_ruler"]

patterns = aruler.patterns
for pattern in patterns:
    if "MORPH" not in pattern["attrs"]:
        continue
    print(pattern["patterns"], "=>", pattern["attrs"]["MORPH"])

Example output:

[[{'TAG': 'VBD', 'LOWER': 'were'}]] => Mood=Ind|Tense=Past|VerbForm=Fin
[[{'TAG': 'VBD', 'LOWER': "'d"}]] => Mood=Ind|Tense=Past|VerbForm=Fin
[[{'LOWER': 'i', 'DEP': 'nsubj'}]] => Case=Nom|Number=Sing|Person=1|PronType=Prs
[[{'LOWER': 'you', 'DEP': 'nsubj'}]] => Case=Nom|Person=2|PronType=Prs

This will show you what patterns / rules result in which morphological tags.