Can I train a custom NER in Google Colaboratory?

[vmatter]

I want to train a custom NER for a work that I am developing in Google Colaboratory and according to spaCy's documentation, to train a custom ner I need to generate a config file and after that run the following commands:

python -m spacy init fill-config base_config.cfg config.cfg

python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy

Is there a way to do this in Google Colaboratory? If not, is there another way to train a customer NER using spaCy?

Many thanks.

[ljvmiranda921]

Most of the training process in spaCy is done via the command line. If you can access this via Google Colab then it should be possible. I don't use Colab that much, but I recall that you can prefix "!" in your cells to make it a command (in case you can't access Colab's terminal directly).

[vmatter]

I managed to run it on Google Colab already. I just added the config file in one of my Google Drive directories:

!python -m spacy init fill-config "/content/drive/MyDrive/.../base_config.cfg" config.cfg

And regarding the other command line, the file was locally saved by Colab, allowing me to access it easily.

Now I am on a journey to understand how to configure the epochs and other parameters of the base_config file.

[ljvmiranda921]

That's great that you managed to run it. So the config.cfg file dictates how the training process will run. You can check the FAQ for common training questions that you might encounter, and refer to this discussion on how to configure when/how the training ends

[vmatter]

Thanks, I will have a look at the discussion. And if I may ask, do you have some documentation or discussion that you would suggest for calculating ner's F-score using SpaCy? From what I have seen in the training of my custom ner, the F-score oscillates in the starting epochs through 0 but after a considerable amount of epochs, the value reaches 100. Would you be able to suggest what is the best practice regarding calculating the "true" ner's F-score?

[ljvmiranda921]

Hi @vmatter , not sure what you meant by the "true" F-score, but another way you can compute for your model's metrics is to run spacy evaluate (this requires you to pass a test set). Under the hood, it uses the Scorer which you can call programmatically.

Also, the F-score you see while training is what's being computed during that process. What you might actually need is the last F-score that was reported.

[vmatter]

Thanks, I will have a look at the spaCy evaluate function and the Scorer.