Training and testing RMSE energy drops significantly at training step=decay steps

[hadi-bourji]

I have run several trainings with the same decay step value and once the training gets past the point of the decay steps value the error drops significantly. Is there any reasoning behind this? I had assumed once it gets to decay steps the only significance is that the learning rate reaches its minimum value

[njzjz]

The total loss is a function of the learning rate.

[hadi-bourji]

Right, but the RMSE energy is only a function of the predicted and actual energies correct? I am confused why the learning rate would affect the RMSE

[njzjz]

If the learning rate is too high, the learning will jump back and forth over a minimum. You can read Goodfellow et al, Deep Learning (2016), MIT press, pp 424.