Can I ask you a question about fine-tuning the dpa3 model?

[hyeon4977]

https://github.com/janosh/matbench-discovery/tree/main/models/deepmd

I want to train a pre-trained model based on the materials project in the link. But I am having an error in the process. Am I trying to do something that is impossible?

[anyangml]

What exactly is the error you are facing? The checkpoint submitted to Matbench-discovery is a frozen .pth file. The init-frz-model finetuning mode might be deprecated already for the Pytorch backend. You can use this model instead, it's the original checkpoint before finetuning for Matbench-Discovery.

[hyeon4977]

Thank you for your kind reply. Isn't the MP_traj_v024_alldata_mixu head in the OpenLAM DB a pre-trained model based on the Materials project?

[anyangml]

Thank you for your kind reply. Isn't the MP_traj_v024_alldata_mixu head in the OpenLAM DB a pre-trained model based on the Materials project?

I do not think you can directly finetune the MP_traj_v024_alldata_mixu head from a frozen checkpoint. If you want to use the MP_traj_v024_alldata_mixu head from the Openlam model (i.e. the non-compliant model), feel free to use the link i shared earlier. We further finetuned that head with a huber loss before our submission. If you want the compliant model, we would have to upload the raw ckpt file so that you can finetune it.

[hyeon4977]

I downloaded DPA-3.1-3M.pt from the link you provided and fine-tuned it with the command below. My Vasp aimd results are mapped to input_fine.json, but I'd like to ask if the following steps are correct.

BRANCH=MP_traj_v024_alldata_mixu
dp --pt train input_fine.json --finetune DPA-3.1-3M.pt --model-branch $BRANCH --output out_finetune.json

[anyangml]

I downloaded DPA-3.1-3M.pt from the link you provided and fine-tuned it with the command below. My Vasp aimd results are mapped to input_fine.json, but I'd like to ask if the following steps are correct.

BRANCH=MP_traj_v024_alldata_mixu dp --pt train input_fine.json --finetune DPA-3.1-3M.pt --model-branch $BRANCH --output out_finetune.json

The finetune commands are correct. Note: this model-branch is not what we submitted to matbench-discovery. That model was further finetuned with Huber loss on MPTrj dataset.

[hyeon4977]

Thank you so much for your help!