scripts/preprocess.py should sort tokens lexicographically

[AlekzNet]

Currently the indexes are assigned to tokens on the first occurrence basis. If the text is changed (think of fixing a typo or training a pre-trained model on a different corpus) the indexes might be reassigned what will break subsequent training initialized from a checkpoint.

[dgcrouse]

Duplicate of #12. We are addressing this by creating a script that encodes new data using an existing JSON schema. This is faster and more space-friendly, esp. for non-text datasets.