Trained model is not ideal, what Deepmd paramaters should I tune.

[XRTan-github]

Dear Deepmd developer and the community. First I would like to thank you for this wonderful tool.

Currently, I am using LAMMPS to MD simulate a perovskite structure at 450k using a model trained from 450k VASP AIMD result. But the MD results are not ideal and the perovskite structure messed up after MD. My hunch is my trained model is wrong. I have tried to tune some parameters myself, such as numb_steps, neuron rcut_smooth, etc. No luck. They all show a similar temperature varying behavior: first steady rise to 450k after approx 1000steps then a rapid rise to a radicular number such as over 100000k. After that, the temperature either drops to less than 0k or drops to around 1000k, remaining at this temperature until the end of the MD simulation. I can also see the perovskite structure begin to mess up at around 1000 steps.

I wonder has anyone with more experience in LAMMPS and Deepmd seen a similar situation before? If so generally what Deepmd parameters would you choose to tune your model? Or do you think this is the problem of my training data (rubbish in, rubbish out)?

Attach link includes some of my Deepmd tries.
https://github.com/AshoreMrFish/MyDeepmdtry.git

Regards

[njzjz]

Did you use different systems for training and for MD simulations? The possible reason is that your simulation was not covered by training data.

[XRTan-github]

Thank you for the reply. I used the same system for training and for MD simulation. The training data is a 2 by 2 MAPbI3 structure running in 450k AIMD for 7.5 ns, NVT system. The MD simulation is an 8 by 8 MAPbI3 structure running in fixed temperature 450k, also NVT system.
The training data has been added to the link.
https://github.com/AshoreMrFish/MyDeepmdtry.git

Another reason why I suspect something is wrong with my training is that I observe several suspicious sudden drops of the error value when I was tracking the training process (as shown below). Is this sudden drop error normal? If not what does it mean?

Thank you

[njzjz]

Did you change the number of steps or other parameters during training?

[XRTan-github]

Thank you for your reply.

I am not sure if I understand the question correctly. If you mean did I stop during the middle of the training, change some parameters then continue the training. I did not do that. I always just change the parameters at the beginning, start the process and wait until it finishes.

The examples I talked about above are what after I changed rcut_smooth, number of step, and neuron number. But before I changed these parameters, I did try the example water input.json setting. Similar to other LAMMPS results, the temperature first steady at the set temperature of 450k until approx 3000steps then it rapid rise to a radicular number. At which points you can see the structure begin to mess up.

I mean I supposed maybe this model is better as the structure didn't break down until 3000 steps while other breaks down after 1000 steps. But it still breaks down. Also, the LAMMPS simulation didn't finish and it stopped with an error message. "ERROR on proc 0: Non-numeric atom coords - simulation unstable (src/domain.cpp:550)" This is why I decided to try to tune the parameters.

Thank you.

[huaitings]

Your decay rate is less than 0.95