spaCy Project: Part-of-speech Tagging & Dependency Parsing questions

[kanayer]

I have 3 questions regarding the spaCy tagger_parser project located here Github repo

1. In the project.yml file GPU is set to -1. Is it possible to train the pipeline using GPU? Do I have to add CUDA_VISIBLE_DEVICES before the spacy project run command? Or do I have to change the config file?

2. If I want the spaCy model to be able to do word segmentation for Thai (e.g. separate "Iwantcoffee" into "I want coffee"), can tokenizer help with the problem? And if yes, can I add a tokenizer to train together with the pipeline?

3. If I want to use transformers instead of the tok2vec models, do I just replace the tok2vec models with the transformer ones in the config file?

[adrianeboyd]

1. The GPU setting just is a variable that is filled in in the commands below, so you can check out the steps/commands to see for sure what the options should be. In all our demo projects you'd set it the GPU ID project variable to 0 to enable GPU 0. The GPU settings aren't in the config file at all, since you might want to use the same config for training with/without GPU.
2. You can use pythainlp with the built-in ThaiTokenizer. spacy doesn't currently include a trainable tokenizer, but some languages have options to use external trainable tokenizers like pkuseg (Chinese) or sudachipy (Japanese). For any other options, you'd need to implement a custom tokenizer and train it outside of spacy.
3. It's better to start with a new config generated by spacy init config -G to get the settings right for transformer training. A number of the training settings (in particular the optimizer) change between tok2vec and transformer and the models won't train well with mismatched settings.