Question about feature 'resnet_dt' and 'useBN' #1223
Unanswered
Franklalalala
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Going through papers and docs, I knew that, deepmd builds embedding net and fitting net with resnet style MLP.
Reading the src code deepmd\utils\network.py, I found a button called 'resnet_dt' and notation says 'Using time-step in the ResNet construction', which confused me. Here is the code starting in line 206:
if resnet_dt :
xx += hidden * idt
else :
xx += hidden
I've read the blogs about residual net, but still couldn't understand the notation. What's the meaning about time-step? It seems like a weight between identity and mapped identity, since it's a randommly initialized constant.
On the same file, I see the button 'useBN', apparently it has been used. It's well known that a nomalization layer can avoid gradient vannishing etc... Is there any plan to reconsider the module? Or rebuild the structure of the training nets, like cnn, rnn? Just out of curiosity.
Beta Was this translation helpful? Give feedback.
All reactions