How to get part of speech tags for Japanese?

[KTRosenberg]

I am aware based on some closed issues that there were some attempts to get part-of-speech tagging for Japanese, but it doesn't look like this was finalized.

If there IS Japanese POS Tagging in addition to just the dependencies, may I have an example for how to get this information per-token?

Thank you very much.

[polm]

The Japanese models have support for part of speech tags just like any other models, I'm not sure what you're talking about.

import spacy

nlp = spacy.load("ja_core_news_sm")

doc = nlp("わたくしという現象は仮定された有機交流電燈のひとつの青い照明です")

for word in doc:
    print(word, word.pos_, word.tag_, sep="\t")

Does that not work for you?

[polm]

The tagger component is not the only way to get part of speech tags. The Japanese models use the part of speech information from the tokenizer and a tag mapping (in the attribute ruler) to get UD tags. This has limitations, but it's not like the tags aren't there.

[KTRosenberg]

Okay, I see. Thanks for clarifying. On a last note, do the Japanese models have morphology? I am also curious whether the coreference module that is being developed currently will support languages other than English.

Thanks for your time.

[polm]

On a last note, do the Japanese models have morphology?

Yes, the models include all (or basically all) the information from the SudachiPy tokenizer, so they include lemmas.

I am also curious whether the coreference module that is being developed currently will support languages other than English.

The new coreference model is language-neutral, but the training data is only available in English, Arabic, and Chinese, and I think there are issues with the non-English annotations.

[KTRosenberg]

That's too bad about the non-English coreferences, but I'm glad Japanese has morphology. I gather most other languages e.g. Chinese (Mandarin) also have similar features(?)

In any case, thanks. I think that answers all of my questions on this subject.

Also, thanks again for this library.

[polm]

That's too bad about the non-English coreferences, but I'm glad Japanese has morphology. I gather most other languages e.g. Chinese (Mandarin) also have similar features(?)

As far as I'm aware every language has meaningful lemmas, though quality may vary.