Temperature error when run NPT ensemble MD use lammps with deep-kit MLP file

[chunweizhu]

Version

DeePMD-kit v2.1.5

When using the MLP file learned by deep-kit to run the molecular dynamics under the NPT ensemble with lammps, the temperature changes greatly.

lcurve.out generated by deep-kit

#  step      rmse_val    rmse_trn    rmse_e_val  rmse_e_trn    rmse_f_val  rmse_f_trn         lr
....         ....          ....         ....         ....         ....       ....         ....
1096100      7.91e-03    9.88e-03      2.20e-04    2.53e-04      7.69e-03    9.64e-03    3.7e-08
1096200      5.22e-03    1.54e-02      1.58e-04    6.12e-04      5.07e-03    1.49e-02    3.7e-08
1096300      7.99e-03    1.00e-02      2.28e-04    4.31e-04      7.76e-03    9.65e-03    3.7e-08
1096400      5.85e-03    5.04e-03      1.87e-04    4.13e-04      5.61e-03    4.61e-03    3.7e-08
1096500      5.60e-03    9.73e-03      2.26e-04    1.79e-04      5.36e-03    9.52e-03    3.7e-08
1096600      7.50e-03    4.25e-03      2.04e-04    1.31e-04      7.28e-03    4.14e-03    3.7e-08
1096700      6.33e-03    4.01e-03      2.10e-04    3.14e-04      6.09e-03    3.69e-03    3.7e-08
1096800      5.30e-03    9.99e-03      2.19e-04    5.49e-05      5.07e-03    9.81e-03    3.7e-08
1096900      5.56e-03    2.87e-03      1.71e-04    1.79e-04      5.39e-03    2.71e-03    3.7e-08
1097000      5.05e-03    4.59e-03      2.45e-04    1.64e-04      4.81e-03    4.45e-03    3.7e-08
1097100      7.17e-03    7.08e-03      2.56e-04    1.62e-04      6.82e-03    6.92e-03    3.7e-08
1097200      5.96e-03    1.57e-02      1.70e-04    7.13e-04      5.78e-03    1.51e-02    3.7e-08
1097300      7.61e-03    4.21e-03      2.58e-04    1.19e-04      7.30e-03    4.10e-03    3.7e-08
1097400      5.69e-03    8.07e-03      2.34e-04    1.01e-04      5.38e-03    7.92e-03    3.7e-08
1097500      6.18e-03    5.88e-03      1.92e-04    4.04e-04      5.98e-03    5.49e-03    3.7e-08
1097600      6.48e-03    3.43e-03      1.99e-04    1.88e-05      6.26e-03    3.36e-03    3.7e-08
1097700      5.87e-03    4.77e-03      1.69e-04    1.33e-04      5.68e-03    4.64e-03    3.7e-08
1097800      6.55e-03    1.22e-02      1.78e-04    3.88e-04      6.35e-03    1.18e-02    3.7e-08
1097900      7.01e-03    4.63e-03      1.83e-04    5.07e-04      6.79e-03    3.97e-03    3.7e-08
1098000      6.15e-03    5.10e-03      3.11e-04    4.85e-04      5.83e-03    4.53e-03    3.7e-08
1098100      5.72e-03    5.55e-03      1.72e-04    3.11e-04      5.52e-03    5.27e-03    3.7e-08
1098200      8.70e-03    8.04e-03      2.50e-04    3.34e-04      8.45e-03    7.76e-03    3.7e-08
1098300      6.07e-03    3.95e-03      2.33e-04    1.33e-04      5.80e-03    3.83e-03    3.7e-08
1098400      6.82e-03    1.05e-02      1.28e-04    3.44e-04      6.66e-03    1.02e-02    3.7e-08
1098500      6.46e-03    4.75e-03      2.72e-04    2.47e-04      6.12e-03    4.54e-03    3.7e-08
1098600      6.41e-03    9.10e-03      1.15e-04    2.40e-04      6.25e-03    8.87e-03    3.7e-08
1098700      5.03e-03    5.94e-03      1.26e-04    1.44e-04      4.90e-03    5.80e-03    3.7e-08
1098800      5.89e-03    7.18e-03      3.26e-04    2.34e-04      5.47e-03    6.97e-03    3.7e-08
1098900      6.53e-03    4.12e-03      1.39e-04    4.10e-04      6.35e-03    3.62e-03    3.7e-08
1099000      6.07e-03    1.33e-02      2.62e-04    3.03e-04      5.83e-03    1.30e-02    3.7e-08
1099100      7.99e-03    7.34e-03      1.93e-04    2.07e-04      7.72e-03    7.15e-03    3.7e-08
1099200      7.33e-03    6.61e-03      2.20e-04    5.20e-05      7.09e-03    6.49e-03    3.7e-08
1099300      6.71e-03    6.80e-03      1.00e-04    7.26e-05      6.56e-03    6.67e-03    3.7e-08
1099400      8.60e-03    3.82e-03      2.12e-04    1.93e-04      8.31e-03    3.66e-03    3.7e-08
1099500      1.04e-02    7.29e-03      2.70e-04    2.78e-04      1.01e-02    7.05e-03    3.7e-08
1099600      8.32e-03    7.03e-03      1.74e-04    2.22e-04      8.12e-03    6.83e-03    3.7e-08
1099700      6.55e-03    4.87e-03      2.09e-04    3.00e-04      6.34e-03    4.60e-03    3.7e-08
1099800      8.50e-03    6.40e-03      1.87e-04    2.38e-04      8.29e-03    6.20e-03    3.7e-08
1099900      6.77e-03    9.93e-03      3.14e-04    1.09e-04      6.45e-03    9.74e-03    3.7e-08
1100000      9.14e-03    4.81e-03      1.25e-04    1.82e-04      8.95e-03    4.66e-03    3.5e-08

in.lammps file

units           metal
boundary        p p p
atom_style      atomic

neighbor        2.0 bin
neigh_modify    every 10 delay 0 check no

read_data	crystal.lmp

pair_style	deepmd frozen_graph.pb
pair_coeff  * *	

velocity        all create 600 23456789

fix             1 all npt temp 300 300 0.5 iso 0 0 0.5
timestep        0.0005
thermo_style    custom step pe ke etotal temp press vol lx ly lz
thermo          100
dump		1 all custom 50 crystal.dump id type x y z 

run             1000000

write_data	S300K.dat
write_restart	S300K.rest

log file generated by lammps

   Step         PotEng         KinEng         TotEng          Temp          Press          Volume           Lx             Ly             Lz      
       186  -4233.8215      99.194235     -4134.6273      600            208.69833      51021.622      33.156116      34.06778       45.169655    
       200  -4233.7428      99.095748     -4134.6471      599.40427      194.03662      51024.606      33.156762      34.068444      45.170536    
       300  -4229.1065      93.227609     -4135.8789      563.90944      436.82836      51234.35       33.202132      34.115061      45.232344    
       400  -4219.3777      81.003336     -4138.3744      489.968        699.95335      52084.956      33.384868      34.302822      45.481291    
       500  -4209.2071      68.421181     -4140.7859      413.86184      746.83224      53794.747      33.74625       34.674141      45.973614    
       600  -4201.6201      58.945985     -4142.6741      356.54886      702.37997      55741.665      34.148545      35.087497      46.521674    
       700  -4196.8818      52.698594     -4144.1832      318.76002      646.26422      57528.87       34.509673      35.458555      47.01365     
       800  -4194.2004      48.718999     -4145.4814      294.68849      549.0667       59371.756      34.874302      35.83321       47.510396    
       900  -4192.774       46.238338     -4146.5357      279.68362      454.09924      61188.125      35.226374      36.194962      47.990035    
      1000  -4192.8746      45.56056      -4147.314       275.58392      355.12232      62801.406      35.533284      36.510311      48.408149    
      1100  -4192.2554      44.323233     -4147.9321      268.09965      286.50554      64210.293      35.79704       36.781319      48.767471    
      1200  -4195.2165      46.866393     -4148.3501      283.48256      217.08625      65433.917      36.022999      37.013491      49.075303    
      1300  -4193.3626      44.686718     -4148.6759      270.29828      179.61693      66537.645      36.224414      37.220445      49.349698    
      1400  -4195.8758      47.00852      -4148.8673      284.34225      111.48583      67506.035      36.399306      37.400145      49.587958    
      1500  -4206.9288      57.89739      -4149.0314      350.20618      127.72645      68340.122      36.548606      37.55355       49.791354    
      1600  -4206.9703      57.629874     -4149.3405      348.58804      34.444852      69095.828      36.682831      37.691465      49.974213    
      1700  -4208.908       59.087367     -4149.8207      357.40404     -10.696354      69728.727      36.794492      37.806197      50.126333    
      1800  -4216.9502      66.380345     -4150.5699      401.51736      8.9990631      70123.961      36.86388       37.877493      50.220863    
      1900  -4235.4791      83.651861     -4151.8272      505.98824      15.437296      70269.744      36.889408      37.903723      50.25564     
      2000  -4245.7403      91.47274      -4154.2676      553.29469     -99.812803      70382.715      36.909166      37.924025      50.282558    
      2100  -4255.4298      98.072364     -4157.3574      593.2141      -44.626453      70354.401      36.904216      37.918938      50.275814    
      2200  -4269.5943      108.87927     -4160.715       658.58223     -106.44838      70112.123      36.861805      37.875361      50.218036    
      2300  -4281.986       117.18263     -4164.8034      708.8071      -268.7748       69503.132      36.754768      37.765381      50.072216    
      2400  -4316.5482      146.00341     -4170.5448      883.13646     -136.15906      68593.147      36.593657      37.59984       49.852728    
      2500  -4338.2671      159.89045     -4178.3767      967.13556     -249.89682      67579.183      36.412448      37.413648      49.605862    
      2600  -4363.6826      175.87487     -4187.8077      1063.8211     -184.71517      66350.607      36.19044       37.185536      49.303413    
      2700  -4422.2797      221.31223     -4200.9675      1338.6599      53.64806       65475.257      36.030584      37.021285      49.085636    
      2800  -4438.4736      222.5129      -4215.9607      1345.9224     -280.43358      64879.823      35.92103       36.908719      48.936387    
      2900  -4488.9285      254.70584     -4234.2227      1540.6491     -633.27729      64132.599      35.782596      36.766478      48.747794    
      3000  -4539.3607      282.66869     -4256.692       1709.789      -461.94578      62689.872      35.512237      36.488685      48.379475    
      3100  -4593.265       310.79997     -4282.4651      1879.9478      101.62736      61041.572      35.198228      36.166042      47.951691    
      3200  -4603.9453      290.8025      -4313.1428      1758.9883     -751.03244      59941.911      34.985581      35.947548      47.661995    
      3300  -4724.801       378.01185     -4346.7891      2286.4949      435.59858      58936.378      34.788848      35.745406      47.393979    
      3400  -4758.0974      366.94103     -4391.1564      2219.5304     -346.62982      58774.893      34.757045      35.712728      47.350653    
      3500  -4772.9981      340.98259     -4432.0155      2062.5145     -1384.8723      58243.417      34.651963      35.604757      47.207497    
      3600  -4888.9992      411.76854     -4477.2307      2490.6803      694.66416      56609.946      34.324942      35.268744      46.761985    
      3700  -4932.0273      406.33576     -4525.6916      2457.8188     -862.42364      55456.96       34.090307      35.027658      46.442334    
      3800  -4988.5168      408.91995     -4579.5968      2473.4499     -772.66915      53924.087      33.773274      34.701908      46.01043     
      3900  -5042.3271      405.80037     -4636.5267      2454.5804     -1264.7808      52501.868      33.473707      34.394104      45.60232     
      4000  -5139.1072      442.61963     -4696.4876      2677.2904     -658.44148      51091.756      33.171301      34.083383      45.190342    
      4100  -5239.3961      477.11238     -4762.2837      2885.9281     -20.56002       49711.639      32.869891      33.773685      44.779721    
      4200  -5327.0323      491.68217     -4835.3501      2974.0569     -265.61739      48887.693      32.687277      33.58605       44.530941    
      4300  -5427.5443      514.17322     -4913.3711      3110.0994     -1088.4835      47756.197      32.433126      33.324911      44.184702    
      4400  -5484.4545      486.5254      -4997.9291      2942.865      -2145.2373      46429.924      32.130061      33.013513      43.771828    
      4500  -5588.5608      505.01791     -5083.5429      3054.7213     -1825.3733      44877.425      31.767878      32.641371      43.278414    
      4600  -5699.1822      525.5729      -5173.6093      3179.0531     -1804.5493      43577.289      31.458086      32.323061      42.856374    
      4700  -5788.8519      523.57874     -5265.2732      3166.9909     -2292.2788      41872.238      31.042328      31.895872      42.289973    
      4800  -5919.2809      564.01313     -5355.2678      3411.568      -1400.6388      40146.982      30.609991      31.451646      41.700986    
      4900  -6017.618       566.48859     -5451.1294      3426.5414     -1332.0001      38918.455      30.294521      31.127503      41.271211    
      5000  -6142.0575      582.77468     -5559.2829      3525.0517     -2322.9321      37811.475      30.004526      30.829533      40.876141    
      5100  -6263.8556      590.20537     -5673.6502      3569.998      -2761.7679      36565.34       29.671222      30.487065      40.422071    
      5200  -6442.6759      646.68357     -5795.9923      3911.6199     -976.65618      35369.857      29.344271      30.151125      39.976655    
      5300  -6517.7361      597.00244     -5920.7337      3611.1117     -2672.5763      34147.802      29.002347      29.799799      39.510841    
      5400  -6643.6148      601.24108     -6042.3737      3636.7502     -2212.4089      33078.796      28.696491      29.485533      39.094163    
      5500  -6748.4346      594.09449     -6154.3401      3593.5223     -4245.6458      31974.776      28.373619      29.153783      38.654304    
      5600  -6863.1658      593.75943     -6269.4063      3591.4956      1043.8279      30715.93       27.996267      28.766055      38.140224    
      5700  -6949.1427      566.00868     -6383.1341      3423.6385     -1638.3828      30188.308      27.835038      28.600393      37.920578    
      5800  -7045.9476      545.53178     -6500.4159      3299.7791     -3591.455       29493.738      27.619904      28.379344      37.627494    
      5900  -7169.8397      569.9283      -6599.9114      3447.3473      675.48132      28578.808      27.331299      28.082803      37.234317    
      6000  -7230.2803      520.27858     -6710.0017      3147.0292     -1913.6128      28179.935      27.203549      27.95154       37.060279    
      6100  -7307.9124      492.22617     -6815.6863      2977.3474     -1513.1879      27748.558      27.064024      27.808179      36.8702      
      6200  -7384.3853      472.4631      -6911.9222      2857.8058     -4121.7166      26953.084      26.802896      27.539871      36.514457    
      6300  -7480.7308      472.56432     -7008.1665      2858.4181     -1792.7772      25999.533      26.483014      27.211194      36.078671    
      6400  -7564.2485      464.68711     -7099.5614      2810.7708      345.14772      25607.781      26.349328      27.073832      35.896547    
      6500  -7630.8276      447.89584     -7182.9318      2709.2049     -3285.1659      25222.693      26.216581      26.937434      35.7157      
      6600  -7716.1596      447.41137     -7268.7482      2706.2744     -1890.6534      24430.127      25.939054      26.652277      35.337617    
      6700  -7793.413       442.95802     -7350.455       2679.3373     -1244.874       23934.643      25.762493      26.470861      35.097082    
      6800  -7842.3669      411.10432     -7431.2626      2486.6626     -1813.32        23388.847      25.565159      26.268101      34.828248    
      6900  -7902.0657      393.99631     -7508.0694      2383.1807      482.22792      22841.79       25.364265      26.061683      34.554562    
      7000  -7958.8605      386.3266      -7572.5339      2336.7886     -569.18056      22596.839      25.273271      25.968188      34.430599    
      7100  -8019.2836      380.73057     -7638.5531      2302.9397     -1439.7625      22354.034      25.182424      25.874843      34.306835    
      7200  -8079.9209      375.1731      -7704.7478      2269.324      -1972.657       21873.423      25.000641      25.688062      34.059187    
      7300  -8132.2537      358.42916     -7773.8246      2168.0443     -2905.1032      21076.508      24.693261      25.37223       33.640433    

my question is

1. Why can't I get a stable temperature on NPT ensemble , after doing thousands of iterations? Is it because the precision of my MLP is not enough? (but From the information in the lcurve.out file, I think the MLP has enough precision).
2. Or is it because the number of atoms in the structure is too small when I do molecular dynamics? (A total of 1280 atoms in my crystal.lmp file).
3. Or there is something wrong with my in.lammps parameter setting?
4. How to judge that the accuracy of my trained MLP is good enough? (have already known the method of https://github.com/deepmodeling/deepmd-kit/blob/master/doc/test/test.md)
5. Which step is wrong in this situation? the deep-kit step or the lammps step.

Thanks!

[owen-rett]

MLIAPs struggle to extrapolate (if you can consider them as able to extrapolate at all), and thus you need to include structures within your training set that bound the set of configurations that will be encountered during a NPT simulation. Looking at the lammps results, it seems that your potential struggles when the volume of the cell moves too far from your starting point, which indicates to me that you may need to include more structures with significant (volumetric or otherwise) strain applied to them within the training set. If you're doing active learning, I'd suggest ensuring that the final steps of active learning heavily resemble your eventual use case in terms of lammps settings.

See this FAQ within the documentation https://docs.deepmodeling.com/projects/deepmd/en/master/troubleshooting/md-energy-undulation.html

---

As for how to tell if the accuracy of your trained MLP is good enough, there's a number of methods ranging from the length of a stable simulation, force accuracy on a separate test set (eg check performance on structures from highish temperature AIMD), or checking vs several computed properties of interest (eg elastic constants, diffusion rates, phase transition temperatures, etc). It really depends on what you aim to use your potential for. See the following for examples of validating MLPs, though the methods they show are not the only ones that exist, nor are they the only reviews on this subject, just the first ones I remembered.

Morrow, Joe D., John LA Gardner, and Volker L. Deringer. "How to validate machine-learned interatomic potentials." arXiv preprint arXiv:2211.12484 (2022).

Stocker, Sina, et al. "How robust are modern graph neural network potentials in long and hot molecular dynamics simulations?." Machine Learning: Science and Technology 3.4 (2022): 045010.

[chunweizhu]

Thank you for your detailed answer and references, it helped me a lot.
It is true that the types of structures I included are too few, resulting in weak generalization of the model.

[AnuragKr]

@chunweizhu Which methodology did you use finally to validate Model generalization?

[chunweizhu]

Like owen-rett says that it really depends on what you aim to use your potential for. In my case, the weak generalization of the model will lead to the destruction of a stable structure in the process of running molecular dynamics.

[owen-rett]

A common measure of generalizability that I see in published work is out of distribution force and energy errors. E.g. see in OC22 [1] where they measure model performance by checking the performance on compositions that aren't included in the training dataset. I'm mentioning force errors because forces are generally harder to predict than energies, and also because having well calibrated force predictions are key for performing molecular dynamics. The smaller the force errors you have, generally the longer stable molecular dynamics simulations you can run.

Out of distribution really just means the data is gathered using a different method than the rest of your training dataset. This could be gathering your data using active learning, and testing on ab initio molecular dynamics, testing on a phase/composition that isn't included in the training dataset, or some other manner. This is important for testing generizability because well, its testing to see if your model can generalize on data that doesn't necessarily resemble that of your training set.

Out of distribution data isn't the only way I see people show model performance, for example I've seen a lot of works where they'll show performance on training set data + validation set data, where the latter isn't directly trained upon, but instead is used to pick the best performing epoch (depending on which NNP they're using), and typically is drawn from the same distribution as the training dataset. This method is frankly a bit easier to do, as it doesn't require the generation of a separate test set, and instead data can simply be randomly split into training and validation sets. I should note that showing performance on the validation dataset is well, valid, but doesn't demonstrate generizability quite as persuasively. In either case, the structure of your training dataset should adequately cover the range of configurations you'd expect to see during your eventual use case.

Again, the exact method for measuring model performance will heavily depend on what you're aiming to use the resulting potential for.

Edit: I forgot that I never put the explanation for [2]. Its a paper I read a bit ago that I vaguely remember talking about the importance of testing against data that you aren't necessarily training on. That said I may have put the wrong paper here.

[1] Tran, Richard, et al. "The open catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysis." arXiv preprint arXiv:2206.08917 (2022).

[2] Herr, John E., et al. "Metadynamics for training neural network model chemistries: A competitive assessment." The Journal of chemical physics 148.24 (2018): 241710.

[AnuragKr]

Thanks @owen-rett, for writing a comprehensive answer. It cleared all the doubts. I aim to draw an RDF graph using DeePMD and compare it with the DFT RDF graph. Force error will be the correct metric for me to have a stable molecular dynamics simulation.