HeteroData conversion for an RDF based graph

[igrangel]

Hi PyG Community,

Based on the example described here, I need to create an Heterogeneous Graph out of an RDF graph. The RDF graphs comprises two classes, one object property, two datatype properties, and three instances (see Figure).
I want to ensure that the classes, properties, and instances are correctly modelled here according to PyG.

In this case I have the following transformation using the HeteroData class:

data = HeteroData()
data[':Class1'].x = [1, 1] # [num_class1, num_features_class1 - property :hasName]
data[':Class2'].x = [2, 2] # [num_class2, num_features_class2 - property :hasTitle]
data[':Class1', ':isConnectedTo', ':Class2'].edge_index = [2, 2] # [2, property :isConnectedTo used twice]

Are my assumptions correct here?

[rusty1s]

That looks correct :)

I think that both class1 and class2 only have a single feature (hasName, hasTitle), so:

data[':Class1'].x = [1, 1]  # [num_class1, num_features_class1 - property :hasName]
data[':Class2'].x = [2, 1]  # [num_class2, num_features_class2 - property :hasTitle]

[igrangel]

Hi again,

I have the following code for creating a custom dataset based on the depicted RDF graph above.

def process(self):
      data = HeteroData()
       # get all the node types, in this case, :Class1 and :Class2 and the number of their instances
        node_types = self.get_node_types()
        for node_type in node_types:
            data[node_type].x = torch.randn(self.get_num_class_features(node_type))
        # get the links and the number between the nodes
        df_links_numbers = self.get_links_numbers()
        for index, row in df_links_numbers.iterrows():
            data[row['subject_class'], row['object_property'], row['object_class']].edge_index = [2, row['op_count']]
        print(data)
        torch.save(data, self.processed_paths[0])

And I have the following output:

HeteroData(
  :Class1={ x=[1, 1] },
  :Class2={ x=[2, 1] },
  (:Class1, :isConnectedTo, :Class2)={ edge_index=[2] }
)

So, in theory, the classes are correct but for some reason the edge_index does not get the value.

In this line here

data[row['subject_class'], row['object_property'], row['object_class']].edge_index = [2, row['op_count']]

The number row['op_count'] has the right value, i.e., 2 but I can't assign it properly even hardcoding it.

@rusty1s would be great to have your input, suggestion here.

Best,

[rusty1s]

You are saving a list of ints as edge_index ([2, 2]), not an actual long tensor that is holding node index pairs of shape [2, num_edges].

[igrangel]

@rusty1s thanks a lot for your answer. Indeed, now I'm getting the right result using a tensor as you described.

HeteroData(
  :Class1={ x=[1, 1] },
  :Class2={ x=[2, 1] },
  (:Class1, :isConnectedTo, :Class2)={ edge_index=[2, 2] }
)

In these cases, do I need to work also in spliting to train, validation, and test, as in here? Or this can be done later when the GNN model is created? Do you have any recommendation here to complete this kind of datasets?
I have done this automatically so it can be extended to any RDF dataset that can be queried using SPARQL.

[rusty1s]

If you do not have any pre-defined splitting strategy, you can do the splitting part in a later stage via the RandomNodeSplit transform, see here.

[igrangel]

@rusty1s I really appreciate your feedback, thanks a lot.
Our intention is to perform link prediction using these kind of graph, i.e., heterogeneous and directed graph. In addition, our links have types, i.e., object properties.
Knowing that will you recommend to follow this example for the link prediction task?

[rusty1s]

We also have an open PR regarding heterogeneous link prediction, see #3350.

[shahfasal]

Thanks for your response
@rusty1s We looked into code #3350 and are trying to use it for our use case as mentioned above... We have a few questions regarding the code implementation

1. In examples/hetero/hetero_link_pred.py we notice data is converted ToUndirected()(data) why is it done so? We have a directed graph do we need to convert it ToUndirected()?
2. In the same file line 43 we see <del data['movie', 'rev_rates', 'user'].edge_label>, I tried to locate this "rev_rates" relation while generating data in "torch_geometric/datasets/movie_lens.py" but I could not find it so why is it used here and what is the purpose of it?
3. In RandomLinkSplit "rev_edge_types=[('movie', 'rev_rates', 'user')]" is used, is it compulsory to create "rev relation" for every relationship we have in our KG?

[rusty1s]

1. Adding reverse edges is usually done in a GNN scenario to allow message passing flow in both directions. Other-wise, only destination nodes will receive information. Calling ToUndirected() in heterogeneous graphs will add the rev_{rel} relation.
2. By default, ToUndirected() will copy all edge attributes to its reverse edge type as well. Line 43 just removes the label information from the reverse type.
3. You only need to specify for links across you plan to split to prevent data leakge. I agree that this might be cumbersome if you are working in a knowledge graph with lots amounts of relations. One way around this is to autogenerate this list:

edge_types = data.edge_types
data = T.ToUndirected()(data)
rev_edge_types = [(dst, f'rev_{rel}', src) for src, rel, dst in edge_types]

[shahfasal]

Hi @rusty1s thanks for your response. Understood it clearly.. Further we tried running the code on the below data set

HeteroData(
  :Class1={ x=[2, 2] },
  :Class2={ x=[6, 1] },
  (:Class1, :class1ConnectClass1, :Class2)={ edge_index=[2, 6] },
  (:Class2, rev_:class1ConnectClass1, :Class1)={ edge_index=[2, 6] }
) 

we got the error as follows

File "D:model-test.py", line 79, in forward
    z = torch.cat([z_dict[':Class1'][row], z_dict[':Class2'][col]], dim=-1)
TypeError: 'NoneType' object is not subscriptable

On debugging in class Model under method forward z_dict = self.encoder(x_dict, edge_index_dict) we get z_dict null as shown
{':Class1': None, ':Class2': None}
This is our model based on the example hetero_link_pred

parser = argparse.ArgumentParser()
parser.add_argument('--use_weighted_loss', action='store_true',
                    help='Whether to use weighted MSE loss.')
args = parser.parse_args()

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

dataset = DatasetPyGRDF('data/')
data = dataset[0].to(device)
data[':Class1'].x = torch.eye(data[':Class1'].num_nodes, device=device)
del data[':Class1'].num_node

data = ToUndirected()(data)
print("ToUndirected",data)
del data[':Class2', 'rev_:class1ConnectClass1', ':Class1'].edge_label

train_data, val_data, test_data =  T.RandomLinkSplit(
num_val=0.05,
num_test=0.1,
neg_sampling_ratio=0.5,
edge_types=[(':Class1', ':class1ConnectClass1', ':Class2')],
rev_edge_types=[(':Class2', 'rev_:class1ConnectClass1', ':Class1')]
)(data)


if args.use_weighted_loss:
    weight = torch.bincount(train_data[':Class1',':Class2'].edge_label)
    weight = weight.max() / weight
else:
    weight = None

def weighted_mse_loss(pred, target, weight=None):
    weight = 1. if weight is None else weight[target].to(pred.dtype)
    return (weight * (pred - target.to(pred.dtype)).pow(2)).mean()

class GNNEncoder(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = SAGEConv((-1, -1), hidden_channels)
        self.conv2 = SAGEConv((-1, -1), out_channels)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        x = self.conv2(x, edge_index)
        return x

class EdgeDecoder(torch.nn.Module):
    def __init__(self, hidden_channels):
        super().__init__()
        self.lin1 = Linear(2 * hidden_channels, hidden_channels)
        self.lin2 = Linear(hidden_channels, 1)

    def forward(self, z_dict, edge_label_index):
        row, col = edge_label_index

        z = torch.cat([z_dict[':Class1'][row], z_dict[':Class2'][col]], dim=-1)

        z = self.lin1(z).relu()
        z = self.lin2(z)
        return z.view(-1)

class Model(torch.nn.Module):
    def __init__(self, hidden_channels):
        super().__init__()
        self.encoder = GNNEncoder(hidden_channels, hidden_channels)
        self.encoder = to_hetero(self.encoder, data.metadata(), aggr='sum')
        self.decoder = EdgeDecoder(hidden_channels)

    def forward(self, x_dict, edge_index_dict, edge_label_index):
        z_dict = self.encoder(x_dict, edge_index_dict)
        return self.decoder(z_dict, edge_label_index)


model = Model(hidden_channels=32).to(device)

# Due to lazy initialization, we need to run one model step so the number
# of parameters can be inferred:
with torch.no_grad():
    model.encoder(train_data.x_dict, train_data.edge_index_dict)

optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
@torch.no_grad()
def test(data):
    model.eval()
    pred = model(data.x_dict, data.edge_index_dict,
                 data[':Class1',':Class2'].edge_label_index)
    pred = pred.clamp(min=0, max=5)
    target = data[':Class1',':Class2'].edge_label.float()
    rmse = F.mse_loss(pred, target).sqrt()
    return float(rmse)

def train():
    model.train()
    optimizer.zero_grad()

    pred = model(train_data.x_dict, train_data.edge_index_dict,
                 train_data[':Class1',':Class2'].edge_label_index)
    target = train_data[':Class1',':Class2'].edge_label
    loss = weighted_mse_loss(pred, target, weight)
    loss.backward()
    optimizer.step()
    return float(loss)

for epoch in range(1, 30):
    loss = train()
    train_rmse = test(train_data)
    val_rmse = test(val_data)
    test_rmse = test(test_data)
    print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}, Train: {train_rmse:.4f}, '
          f'Val: {val_rmse:.4f}, Test: {test_rmse:.4f}')

Can you please help us with this issue

[rusty1s]

Sorry for the late reply. It looks like the : and - in the node and edge type names resulted in that bug. I fixed it in master, see b63a660.

If you further delete the line

del data['if-gnn:Class2', 'rev_:isConnectedTo', 'if-gnn:Class1'].edge_label

(since it will access a non-existing edge type), the code will crash with an index error. This looks like a problem with your data, as the following assert crashes as well:

for edge_type in data.edge_types:
    src, _, dst = edge_type
    assert data[edge_type].edge_index[0].max().item() < data[src].num_nodes
    assert data[edge_type].edge_index[1].max().item() < data[dst].num_nodes

[shahfasal]

@rusty1s thanks for your response... The changes you have pushed how do we get them? Do we have to install the torch-geometric package again? or manually make these changes in the code?

[rusty1s]

I recommend to install directly from master:

pip install git+https://github.com/pyg-team/pytorch_geometric.git

[shahfasal]

Thanks, @rusty1s this was a long discussion and you helped us all along... Sorry for the late reply but with your git changes and minor fix from our side we were able to run the code... Hope the fix you did in master will be released in the next release.
Thanks a lot once again

[rusty1s]

Super :)