Heterogenous GCN Design Advice

[ChrisJH1]

Hello

I am new to GCN design and creation and hoping for some guidance if possible.

Firstly, are there any particular model examples to study in the field of heterogenous graph regression?

I am struggling to find any example implementation. Particularly those with edge features.

My particular challenge is looking at the prediction of house prices. I have created a preliminary HeteroData graph (more node types to be added if successful) shown below:

          HeteroData(
            Property={
              x=[82698, 2],
              y=[82698],
              train_mask=[82698],
              val_mask=[82698],
              test_mask=[82698]
            },
            Station={ x=[677, 1] },
            City={ x=[1, 1] },
            (Property, Distance, Station)={
              edge_index=[2, 413490],
              edge_attr=[413490]
            },
            (Property, Distance, City)={
              edge_index=[2, 82698],
              edge_attr=[82698]
            }
          )

Really simply, the y is the property price whilst the x is the area and property type. The other nodes are train stations and cities (although I have a single city for now)

For both edge types the attribute is the Euclidean distance

I have attempted various HGTConv implementations without success but I know I am likely missing fundamentals.

Any guidance would be greatly appreciated.

Thanks!

[rusty1s]

You can use HeteroConv to perform heterogeneous message passing with underlying message passing operators that support edge weights/edge features, e.g., GraphConv:

conv = HeteroConv({
    edge_type: GraphConv((-1, -1), 64) for edge_type in data.edge_types
}, aggr='sum')

conv(data.x_dict, data.edge_index_dict, edge_weight_dict=data.edge_weight_dict)

[rusty1s]

I think the issue is that you have no edge types pointing to Property. data = ToUndirected()(data) should fix this.

[ChrisJH1]

Thanks @rusty1s . That has solved the key issues and completely makes sense. It has now uncovered a mat multiplication issue that I will investigate and come back with progress.

[ChrisJH1]

Thanks @rusty1s , that solved it in the end. Interestingly my single city node (the centre point of London) was causing issues.

All 80k property nodes had an edge to the city centre with the weight as the Euclidean distance. When reviewing the x_dict produced in the forward, i noticed the entire city tensor was returning as nulls. Should I have this setup differently? Like perhaps 80k identical nodes, one for each of the properties?

Either way, for Hetero graph:

HeteroData( Property={ x=[82698, 2], y=[82698], train_mask=[82698], val_mask=[82698], test_mask=[82698] }, Station={ x=[677, 1] }, (Property, Distance, Station)={ edge_index=[2, 413490], edge_weight=[413490] } )

The following setup works. Although the current model is terrible as I need to add further features.

`
class HeteroGNN(torch.nn.Module):
def init(self, metadata, hidden_channels, out_channels, num_layers):
super().init()
self.convs = torch.nn.ModuleList()

    for _ in range(num_layers):
        conv = HeteroConv({edge_type: GraphConv((-1, -1), hidden_channels) for edge_type in data.edge_types
                          }, aggr='sum')
        self.convs.append(conv)

    self.lin = Linear(hidden_channels, out_channels)

def forward(self, x_dict, edge_index_dict):
    for conv in self.convs:
        x_dict = conv(x_dict, edge_index_dict, edge_weight_dict=data.edge_weight_dict)
        x_dict = {key: F.leaky_relu(x) for key, x in x_dict.items()}
    return self.lin(x_dict['Property'])`

model = HeteroGNN(data.metadata(), hidden_channels=64, out_channels=1, num_layers=2) optimizer = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=0.001)

def train(): model.train() optimizer.zero_grad() out = model(data.x_dict, data.edge_index_dict) mask = data['Property'].train_mask target = data['Property'].y[mask].unsqueeze(1) loss = F.mse_loss(out[mask], target) loss.backward() optimizer.step() return float(loss)

`@torch.no_grad()
def test():
model.eval()
pred = model(data.x_dict, data.edge_index_dict).argmax(dim=-1)

accs = []
for split in ['train_mask', 'val_mask', 'test_mask']:
    mask = data['Property'][split]
    acc = r2_score(data['Property'].y[mask],pred[mask])
    accs.append(float(acc))
return accs`

for epoch in range(1, 101): loss = train() train_acc, val_acc, test_acc = test() print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}, Train: {train_acc:.4f}, ' f'Val: {val_acc:.4f}, Test: {test_acc:.4f}')

[rusty1s]

I see. A node that is connected to 80k other nodes isn't that informative, as it probably cannot aggregate any meaningful information. What about you simply add the distance to london as a node feature to each node, and remove the london node completely.

[ChrisJH1]

Thanks @rusty1s great point