A question about the activation function Gelu

[haku0462]

Hi there,

I am running the Dpgen now. The version of Deepmd for training is "Anaconda3-2022.05-Linux-x86_64.sh". In the iter step, I always received the error massage
"No gradient defined for operation'layer_2_type_0/Gelu' (op type: Gelu). In general every operation must have an associated @tf.RegisterGradient for correct autodiff, which this op is lacking. If you want to pretend this operation is a constant in your program, you may insert tf.stop_gradient. This can be useful to silence the error in cases where you know gradients are not needed, e.g. the forward pass of tf.custom_gradient. Please see more details in https://www.tensorflow.org/api_docs/python/tf/custom_gradient."

I have searched for some similar questions and found the answer "#1918" which said Use GeluCustom as operator name. But I was confused, should I change the "gelu" in param.json to "gelugustom"? which I have tried and failed again. Or should I download a new version of Deepmd before doing this modification?

Other question is if I need to upgrade deepmd to a new version, do I need to delete the old one to avoid the conflict? Or it will auto recover everything by using this Easy install the latest development version

[haku0462]

Or I need to modify the code in _gelu.py by myself?

[njzjz]

It seems you just need to install the version later than #1918, so #1918 can be applied. Btw: I don't understand why you gave an Anaconda version.

[haku0462]

Thank you for your response. The problem has been resolved!

Regarding the Anaconda error, I didn't realize I copied the wrong file name until your notice. Awkward😅