Trainable parameters in CGConv?

[errhernandez]

Hi! I'm using a CGConv layer in a model, and I thought I understood this kind of layer, but in trying to find the number of trainable parameters, I realise that I don't. Here is how I define my model:

nNodeFeatures = 55
nEdgeFeatures = 1

self.graphconv = CGConv( nNodeFeatures, dim = nEdgeFeatures )

Then I pass the output of this layer through a linear layer:

self.linear = Linear( nNodeFeatures, 1 )

With a model defined with just these two layers, I now loop over the parameters, and I get the following:

for param in model.parameters():
print( param.size(), param.requires_grad)

This is what I get, and what prompts my question:

torch.Size([55, 111]) True <--- Matrix Wf (I understand)
torch.Size([55]) True <--- bias bf (I understand)
torch.Size([55, 111]) True <--- Matrix Ws (I understand)
torch.Size([55]) True <--- bias bs (I understand)
torch.Size([55]) True <--- What is this? Looks like another bias (I don't understand!)
torch.Size([55]) True <--- What is this? Yet another bias? (I don't understand!)
torch.Size([1, 55]) True <--- linear layer weight matrix (I understand)
torch.Size([1]) True <--- linear layer bias (I understand)

So, my question is: there appear to be two additional trainable tensors that I cannot figure out from the documentation on CGConv; just for my peace of mind, I would like to understand what these are. Thanks!

[wsad1]

self.bn = BatchNorm1d(channels[1]) this line in CGConvs init, gets executed even if batch_norm variable is false, but in forward sefl.bn gets applied only if self.batch_norm is true.
So the two parameters of size 55, are the two trainable parameters of batch norm, but these parameters don't get trained because they aren't used in the networks forward pass.
I'll create a PR to fix this in master in a while.

[errhernandez]

Thanks for your reply, Jinu, but I must confess it doesn't illuminate me (maybe it is because I am still new to all of this): what is batch_norm supposed to do? by default it is set to false, but what is its effect if set to true? I had noticed this in the documentation, but did not understand from it (neither looking at the source). Thanks again.

[wsad1]

In short, if batch_norm is set to true then the output of this layer gets normalized or standardized for each mini-batch. After getting normalized the values get scaled and shifted using 2 learnable parameters.
For a more detailed explanation you could refer to this video by Andrew Ng or the official pytorch documentation.

[errhernandez]

Thank you so much, Jinu; this clarifies the issue. Thanks!