how to combine GNN with other CNN networks

[MRBMELMH]

Hello,

I would like to ask a question of combining using pyg and pytorch. If i design a model that includes both GNN and CNN, for example,

class GCN2D(torch.nn.Module):
    def __init__(self, num_features, num_classes, num_nodes):
        super(GCN2D, self).__init__()
        self.conv1 = GCNConv(num_features, 64)  
        self.conv2 = nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1)  
        self.fc = nn.Linear(16 * 64 * 64, num_classes) 

        # define learnable adjacency matrix
        self.adj = Parameter(torch.Tensor(num_nodes, num_nodes))
        self.reset_parameters()

    def reset_parameters(self):
        xavier_normal_(self.adj)

    def forward(self, data):
        x, batch = data.x, data.batch
        edge_index = self.adj.nonzero(as_tuple=False).t()
        edge_weight = self.adj[self.adj != 0]
        x = self.conv1(x, edge_index, edge_weight=edge_weight)
         x = self.conv2(x)

before executing ** x = self.conv2(x)**, how to reshape the output of GCNconv to make it be the suitable input of nn.Conv2d? besides, if I define two blocks, each block contains the " self.conv1 = GCNConv(num_features, 64) and self.conv2 = nn.Conv2d(1, 16, kernel_size=3, stride=1, padding=1)" , how can I transfom the shape of input from GCNConv to Conv2d and from Conv2d to GCNConv?

Thanks & Regards

[rusty1s]

That depends a bit on what you want your Conv2d to do. Are you sure you don't want to use Conv1d instead (since PyG does not return a grid output).

The output from GCNConv is a matrix of shape [num_nodes, num_features], which you could theoretically reshape into [num_nodes, grid_x, grid_y, num_features_divided], but I don't understand the purpose of this yet.