Values for warmup_steps and val_set_size

[Eichhof]

Hello

How should I choose the value for warmup_steps and for val_set_size for the config yaml file? In the example config files 10 warmup steps and a val set size of 0.05 is used but others also used 100 warm up steps and 0.01 or 0.02 for val set size. I have a dataset with around 3800 samples and I'm fine-tuning meta-llama/Llama-2-13b-chat-hf.

[NanoCode012]

I think it's a hyper parameter that'll need tuning based on dataset. For my own use, I just try to do 5-10% of total steps.