GCNConv input and output have vastly different scales

[Edenzzzz]

Hi, I'm using GCNConv on some image latent code with dim 4096 (flattened), with mean and std both around 1.0. But after a 2-layer GCN the mean is 16 and the std is 705. Normalization is enabled, as shown in the code. Any comment pointing out what I'm doing wrong here would be deeply appreciated.

import torch

import torch.nn as nn
import torch.nn.functional as F

from torch_geometric.data import Data
from torch_geometric.nn import GCNConv

class GCN(torch.nn.Module):
    def __init__(self, indim, latentdim, outdim):
        super().__init__()
        self.conv1 = GCNConv(indim, latentdim, cached=True, normalize=True, bias=False)
        self.conv2 = GCNConv(latentdim, outdim, cached=True, normalize=True, bias=False)
        nn.init.kaiming_uniform_(self.conv1.lin.weight, mode='fan_in', nonlinearity='relu')
        nn.init.kaiming_uniform_(self.conv2.lin.weight, mode='fan_in', nonlinearity='relu')

    def forward(self, data, training):
        x, edge_index = data.x, data.edge_index

[EdisonLeeeee]

Maybe you should add some BatchNorm layers?

[rusty1s]

Please note that GCNConv is not really designed for operating on images/grid. I feel SplineConv is a better choice here to take the position of nodes into account - See here for an example.