What is shape of the node embedding if no feature is provided

[icedpanda]

Hi,

What is the feature size of a node in RGCNConv if no feature is given?

# initialized node features
# let's say I have features from nn.embedding and its trainable
# method A
node_embed = nn.Embedding(10, 32) # (num_node, n_node_feat)
rgcn = RGCN(in_channels=32, out_channels=128, num_relations=n_relation)
x = rgcn(node_embed.weight, edge_index, edge_type)   # (10, 128)

# however, what would be the size of initialized node feature in rgcn?
# in_channels ->10 nodes here
# In case no input features are given, this argument should correspond to the number of nodes in your graph.
# method B
rgcn = RGCN(in_channels=10, out_channels=128, num_relations=n_relation)
x = rgcn(x=None, edge_index, edge_type) # (10, 128）

Although both approaches have the same number of nodes and out_channels, the B has much more parameters (about 5x-6x more when I have 30k+ nodes) than A. Also, B performs better than A; even I had added some embedding translation pretraining to A

So I guess the gap here would be the size of initialized node features. But I couldn't find any documentation regards this.

Is there any preferred situation using B over A or vice versa?

Thanks

[rusty1s]

They are both similar. I think A is the preferred approach since B will learn an embedding matrix per edge type which is not really necessary.

[icedpanda]

Hi @rusty1s, thanks for the prompt relay, could you please explain in more detail?

[rusty1s]

RGCN trains a weight matrix of shape [in_channels, out_channels] for each edge type. In the case that you pass in x=None, that means that you learn an embedding per node for each edge type. On the other side, if you first embed your nodes with a single embedding layer, you only need to learn low-dimensional weight matrices in the first layer.

[icedpanda]

so basically, each node would have an embedding in the corresponding relation space (edge type) ?

Like [n_relation_this_node_has, in_channels, out_channels] for a node.

[rusty1s]

Yes, exactly!