Pretraining example

[michaellin99999]

Is there a pretraining example in this repo? using soft masking etc..

[NanoCode012]

I'm not sure what soft masking is. To use pretraining_dataset: just point to a HF dataset with text column.