Is there a correct way to use GCNConv layers with no features?

[sbonner0]

I've been thinking about the correct way to use GCNConv layers where there is no input feature matrix provided. I have been using a one-hot encoded input as per the original Kipf paper: data.x = torch.sparse.torch.eye(data.num_nodes)

Another option would be to declare a separate embedding matrix and use this as features in the GCN: emb = nn.Embedding(data.num_nodes, embedding_dim).

How obviously both of these cause a large increase in the number of model parameters. Is there some other way to use GCNConv with no input feature which doesn't cause such a large increase in parameters. If not, what would be the best way to combat this?

[rusty1s]

I don't think there is a clear recommended way to do this. I think the Embedding approach is better than using torch.eye since it scales to larger graphs via scalability techniques such as NeighborSampler as well, and avoids the need of creating a sparse diagonal input feature matrix.

But you are right. These approaches will create a huge number of parameters and only work in transductive learning scenarios. IMO, there are two options to limit the number of parameters:

1. Use synthetic input features instead, such as one-hot decoded node degrees (torch_geometric.transforms.OneHotDegree). This is commonly done in graph classification and works well there.
2. Use basis decomposition to learn initial node embeddings, e.g.:

emb = ... # Parameter of shape [num_bases, in_channels]
alpha = ... # Parameter of shape [num_nodes, num_bases)

x = (alpha.unsqueeze(-1) * emb.unsqueeze(0)).view(num_nodes, in_channels)

[sbonner0]

Thanks, great and clear explanation.