Korean support without manual system-level operations

[alexcombessie]

Hi,

I am using spaCy on secure servers where system-level operation are not possible, only pip is possible. I am very happy to say that it's OK for almost all languages supported by spaCy 🙇

The odd one out is Korean, which today (and including in spacy 3.0 if I understand correctly) requires installation of mecab-ko on the server.

It would be really great if there was an option to handle Korean with only a pip install, like jieba in Chinese.

Thanks,

Alex

[adrianeboyd]

The mecab installation is a total pain, yes. We'd be happy to accept user contributions that add new alternative tokenizers for Korean.

If you're interested, please take a look at the notes here: #6304 (comment)

[alexcombessie]

Hi @adrianeboyd,

Thanks for your prompt reply.

I discovered konlpy while browsing. What do you think of it?
At first glance, it seems easier to setup than mecab-ko. But I am a bit unsure about the Java dependencies...

Cheers,

Alex

[adrianeboyd]

I've heard of it, but I haven't tried to intall it and the java dependencies do look potentially problematic. But on our end, accepting a PR that adds segmenter options + support in the Korean language class that assumes that the user handles the details of the konlpy installation separately would be fine.

[alexcombessie]

Interesting. Is it OK to leave this issue open? I'll see if someone from my team @dataiku can have a look at it next year.

[adrianeboyd]

Sure, the issues labeled "help wanted" or "enhancement" should stay open by default.