cache/store chardet results per file #4
Description
The chardet detection works pretty well but also takes a long time—I didn't time it exactly, but it felt like it added at least an hour to the time it took the corpus-building process to run on my MacBook Air. Since these files aren't going to change, it makes sense to pre-build and cache the results so that subsequent runs of the corpus-building script don't need to re-run the detection process.