Provide a script to cleanly download and normalize text

[ruohoruotsi]

Rather than the current system of each sub-corpora it is own folder with its own code. Create a top-level downloads.sh which can re-assemble the sub-corpora.

Separately, have the downloaded & pre-processed sub-corpora ready to be referenced from ADR, and NMT repos as submodules etc.

[ruohoruotsi]

This work is done here: https://github.com/Niger-Volta-LTI/yoruba-adr/blob/master/scripts/aggregate_corpora_make_parallel_text.sh

It may be useful to have something like this local to this text repo.

[realsama]

Hi @ruohoruotsi , so i have been going through this repo and i am impressed by the work being done here. so i am thinking of contributing if more help is needed (like scraping and others).

[ruohoruotsi]

@shepherd1530 Thank you very much! Please have a look around our repos, for example https://github.com/Niger-Volta-LTI/iranlowo and see what you would like to help with. We welcome any and all contributions. cheers!