Strange behavior in validation/training loss when virial component is included.

[fmatusalem]

To test the inclusion of virial component I'm training a network using a set of 1000 configurations for bulk Au, with 180 atoms. It is only for testing purpose. The configurations was generated from a classical MD and the energies, forces and virial calculated using DFT in SIESTA code.

The behaviour of validation/training loss (rmse_val/rmse_trn) without including virial is OK. At the final step the validation/training loss (rmse_val/rmse_trn) converges to the same order of magnitude of the other components.

If the virial is included I got:

I was expecting the validation loss get at the final step the same order of magnitude of the other components. Why this is not happening?

Question : How the validation/training loss (rmse_val/rmse_trn) is calculated from the other error components in lcurve.out file? Equation 1 from PRL 120, 143001 (2018) and Equation 19 from Comp. Phys. Comm., 228, 178-184, 2018, present differences in their formulation. I have tried different combinations and did not recover the rmse_val/rmse_trn result in lcurve.out.

Thanks for the help.

Bellow the input.json file.

{
"_comment": "that's all",
"model": {
"type_map": [
"Au"
],
"type_embedding": {
"neuron": [
2,
4,
8
],
"resnet_dt": false,
"seed": 3876529487
},
"descriptor": {
"type": "se_e2_a",
"sel": "auto",
"rcut_smth": 0.5,
"rcut": 7.0,
"neuron": [25,50,100],
"axis_neuron": 16,
"resnet_dt": false,
"seed": 38291751,
"_comment": " that's all"
},
"fitting_net": {
"neuron": [
240,
240,
240
],
"resnet_dt": true,
"seed": 3218943636,
"_comment": " that's all"
},
"_comment": " that's all"
},
"learning_rate": {
"start_lr": 1.0e-03,
"decay_steps": 5000,
"_comment": " that's all",
"stop_lr": 1.0e-08
},
"loss": {
"start_pref_e": 0.02,
"limit_pref_e": 1,
"start_pref_f": 1000,
"limit_pref_f": 1,
"start_pref_v": 0.02,
"limit_pref_v": 1,
"_comment": " that's all"
},
"training": {
"stop_batch": 100000,
"_comment": "that's all",
"disp_file": "lcurve.out",
"disp_freq": 100,
"numb_test": 5,
"save_freq": 100000,
"save_ckpt": "model.ckpt",
"disp_training": true,
"time_training": true,
"profiling": false,
"profiling_file": "timeline.json",
"seed": 1271622617,
"training_data": {
"systems": [
"./pbe/train"
],
"batch_size": "auto",
"_comment": "that's all "
},
"validation_data":{
"systems": [
"./pbe/test"
],
"batch_size": "auto",
"_comment": "that's all "
}
}
}