The oscillating loss function curve when train wannier centroid prediction model of DPLR

[wankiwi]

Hey there.

Recently, I was trying to train a DPLR model for my research system. When I performed the training on wannier centroid prediction model, I got the following loss function curve.

Before average:

After average per 100 points:

I don't think this is a good signal for getting a high-accuracy model. I also trained a DPLR model based on this wannier centroid prediction model. When using this DPLR model, my MD simulation always aborted with the error, ERROR on proc 0: Out of range atoms - cannot compute PPPM (src/KSPACE/pppm.cpp:1839). BTW, The same training data for energy prediction worked well on the normal deepmd model.

Did I make some mistakes in the preparation of wannier centroid training data or any other reasons for causing this? And what should I do to improve it? Any discussion will be helpful, thanks in advance.

PS: My deepmd-kit version is v2.2.6.

[njzjz]

Your initial learning rate seems too large.