Is it possible to perform validation during SFT training to monitor overfitting in Slime? #1584
Description
Dear Slime Team,
I am using Slime for LLM SFT. I noticed that all the SFT examples you provided lack evaluation settings. I tried various configurations myself but was unable to calculate the validation dataset loss during the training process to observe if the model is overfitting.
I also reviewed your source code. It appears that your framework does not support running validation simultaneously with training, limiting observations to performance on the training dataset only.
Is my understanding correct? If I am mistaken, could you please advise on how to enable validation during SFT?