heterogeneous graphs from own datasets, explainer

[Lucia61-star]

Sorry to bother you all, I've recently been building a heterogeneous graph using my own dataset and trying to add interpretability to the model.The data set contains user characteristics, product characteristics, invitation relationships between users, and transaction relationships between users and products.The heterogeneous graph data is as follows:
HeteroData( user={ node_id=[2054], x=[2054, 20], }, item={ node_id=[50], x=[50, 9], }, (user, social, user)={ edge_index=[2, 841] }, (user, buy, item)={ edge_index=[2, 5853] }, (item, rev_buy, user)={ edge_index=[2, 5853] } )
I try to update the user's embedding using a network of user-to-user relationships and a network of user-to-product transactions. The product-to-user transaction network is used to update product embeddings, and ultimately attempts to obtain the probability of transactions between users and products.The model code is as follows:

transform = T.RandomLinkSplit(
    num_val=0.1,
    num_test=0.1,
    disjoint_train_ratio=0.3,
    neg_sampling_ratio=3.0,
    add_negative_train_samples=False,
    edge_types=('user','buy','item'),
    rev_edge_types=('item','rev_buy','user'),)

train_data, val_data, test_data = transform(data)
edge_label_index = train_data["user", "buy", "item"].edge_label_index
edge_label = train_data["user", "buy", "item"].edge_label

class ItemGNNEncoder(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = SAGEConv(-1, hidden_channels)
        self.conv2 = SAGEConv(hidden_channels, hidden_channels)
        self.lin = Linear(hidden_channels, out_channels)
    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        x = self.conv2(x, edge_index).relu()
        return self.lin(x)

class UserGNNEncoder(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = SAGEConv((-1, -1), hidden_channels)
        self.conv2 = SAGEConv((-1, -1), hidden_channels)
        self.conv3 = SAGEConv((-1, -1), hidden_channels)
        self.lin = Linear(2*hidden_channels, out_channels)
    def forward(self, x_dict, edge_index_dict):
        user_social = self.conv1(
            x_dict['user'],
            edge_index_dict[('user','social','user')],
        ).relu()
        user_item = self.conv2(
            (x_dict['item'], x_dict['user']),
            edge_index_dict[('item','rev_buy','user')],
        ).relu()
        user_x = torch.cat([user_social, user_item], dim=1)
        return self.lin(user_x)

class EdgeDecoder(torch.nn.Module):
    def __init__(self, hidden_channels):
        super().__init__()
        self.lin1 = Linear(2 * hidden_channels, hidden_channels)
        self.lin2 = Linear(hidden_channels, 1)
        self.sigmoid = nn.Sigmoid()
    def forward(self, z_src, z_dst, edge_label_index):
        row, col = edge_label_index
        z = torch.cat([z_src[row], z_dst[col]], dim=-1)
        z = self.lin1(z).relu()
        z = self.lin2(z)
#         print(z)
        return self.sigmoid(z.view(-1))

class Model(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
#         self.user_emb = Embedding(num_users, hidden_channels)
        self.user_encoder = UserGNNEncoder(hidden_channels, out_channels)
        self.item_encoder = ItemGNNEncoder(hidden_channels, out_channels)
        self.decoder = EdgeDecoder(hidden_channels)
    def forward(self, x_dict, edge_index_dict, edge_label_index):
        z_dict = {}
#         x_dict['user'] = self.user_emb(x_dict['user'])
        z_dict['user'] = self.user_encoder(x_dict, edge_index_dict)
        z_dict['item'] = self.item_encoder(x_dict['user'],edge_index_dict[('user','buy','item')],
        )
        return self.decoder(z_dict['user'], z_dict['item'], edge_label_index)

device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = Model(hidden_channels=64, out_channels=64)
model = model.to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=0.0003)
loss_function = nn.BCELoss(reduction='mean')

def train():
    model.train()
    optimizer.zero_grad()
    out = model(
        train_data.x_dict,
        train_data.edge_index_dict,
        train_data['user','buy','item'].edge_label_index,
    )
    loss = loss_function(out, train_data['user','buy','item'].edge_label)
    loss.backward()
    optimizer.step()
    return float(loss)


@torch.no_grad()
def test(data):
    model.eval()
    out = model(
        data.x_dict,
        data.edge_index_dict,
        data['user','buy','item'].edge_label_index,
    ).sigmoid()
#     print(out)
    out = torch.round(out)
    f1 = f1_score(data['user','buy','item'].edge_label.cpu().numpy(),out.cpu().numpy())
    return f1

for epoch in range(1, 10):
    loss = train()
    train_f1 = test(train_data)
    val_f1 = test(val_data)
    test_f1 = test(test_data)
    print(f'Epoch: {epoch:04d}, Loss: {loss:.4f}, Train: {train_f1:.4f}, '
          f'Val: {val_f1:.4f}, Test: {test_f1:.4f}')

I don't know if it's wrong to build a heterogeneous graph model like this, because the loss decreases during training, but the f1 index stays the same?
Because I am a beginner, I don’t know much about heterogeneous graphs. Thank you for your answers.

[Lucia61-star]

now i update my model,andNegative sampling has been added, and the model has started training. The model is as follows:

class GNN(torch.nn.Module):
    def __init__(self, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = SAGEConv((-1, -1), hidden_channels)
        self.conv2 = SAGEConv((-1, -1), out_channels)

    def forward(self, x, edge_index):
#         x = data.x_dict
#         x = self.conv1(x, edge_index)
        x = self.conv1(x, edge_index).relu()
        x = self.conv2(x, edge_index)
        return x

class EdgeDecoder(torch.nn.Module):
    def __init__(self):
        super().__init__()
     
    def forward(self, z_dict, edge_label_index):
#         z_dict = data.x_dict
        row, col = edge_label_index
       
        z_src = z_dict["user"][row]
        z_dst = z_dict["item"][col]
#         print(z_src)
#         print(torch.sum(z_src * z_dst, dim=-1))
        
        return torch.sum(z_src * z_dst, dim=-1)

class Model(torch.nn.Module):
    def __init__(self, hidden_channels):
        super().__init__()
        self.encoder = GNN(hidden_channels, hidden_channels)
        self.encoder = to_hetero(self.encoder, data.metadata(), aggr='sum')
        self.decoder = EdgeDecoder()
        self.linear = torch.nn.Linear(hidden_channels, 2)

        

    def forward(self, data,edge_label_index):
        x_dict = data.x_dict
        edge_index_dict = data.edge_index_dict
        z_dict = self.encoder(x_dict, edge_index_dict)
#         decoder_output = self.decoder(z_dict, edge_label_index)
# #         reshaped_output = decoder_output.unsqueeze(-1)  
#         linear_output = self.linear(decoder_output)
#         return linear_output.squeeze(-1)
        
        return self.decoder(z_dict, edge_label_index)

model = Model(hidden_channels=32).to(device)

# with torch.no_grad():
#     model(train_data.x_dict, train_data.edge_index_dict, train_data['user', 'buy', 'item'].edge_label_index)
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
# loss_fn = torch.nn.BCEWithLogitsLoss()
loss_fn = torch.nn.BCELoss()
# loss_fn = torch.nn.functional.binary_cross_entropy_with_logits(reduction='mean')


def train():
    model.train()
    optimizer.zero_grad()
    edge_label_index, edge_label = neg_sampling(train_data)
    
#     pred = model(train_data.x_dict, train_data.edge_index_dict, train_data['user', 'buy', 'item'].edge_label_index).sigmoid()
    pred = model(train_data, edge_label_index).sigmoid()

#     print(pred)
    target = edge_label
    l = loss_fn (pred, target)
    l.backward()
    optimizer.step()
    return float(l)

@torch.no_grad()
def evaluate(data):
    model.eval()
    pred = model(data, data['user', 'buy', 'item'].edge_label_index).sigmoid()

    target = data['user', 'buy', 'item'].edge_label
#     print(pred)
    pred = torch.round(pred)
#     print(target)
    l = esm(target,pred)
#     l = loss_fn(pred, target)
    return l

epo_loss = []
all_res = pd.DataFrame()

for epoch in range(1, 170):
    loss = train()
    epo_loss.append(loss)
#     val_loss = evaluate(val_data)
    test_loss = evaluate(test_data)
    new_data = test_loss['scores'].to_frame().T
    all_res = pd.concat([all_res,new_data])

But when I connected to the heterogeneous graph interpreter, the following error occurred:

explainer = Explainer(
    model=model,
    algorithm=CaptumExplainer('IntegratedGradients'),
    explanation_type='model',
    node_mask_type='attributes',
    edge_mask_type='object',
    model_config=dict(
        mode = 'binary_classification',
        task_level='edge',
        return_type='probs'),
    threshold_config=dict(
        threshold_type='topk',
        value=5),
    )
index = torch.tensor([0])
explanation = explainer(
    train_data.x_dict,
    train_data.edge_index_dict,
    index = index,
    edge_label_index = edge_label_index,
   )

TypeError Traceback (most recent call last)
Cell In[130], line 17
15 index = torch.tensor([0])
16 # index = torch.arange(0,100).int()
---> 17 explanation = explainer(
18 train_data.x_dict,
19 train_data.edge_index_dict,
20 index = index,
21 edge_label_index = edge_label_index,
22 )
24 print(explanation.node_mask_dict)

File ~.conda\envs\pytorch\lib\site-packages\torch_geometric\explain\explainer.py:195, in Explainer.call(self, x, edge_index, target, index, **kwargs)
191 if target is not None:
192 warnings.warn(
193 f"The 'target' should not be provided for the explanation "
194 f"type '{self.explanation_type.value}'")
--> 195 prediction = self.get_prediction(x, edge_index, **kwargs)
196 target = self.get_target(prediction)
198 if isinstance(index, int):

File ~.conda\envs\pytorch\lib\site-packages\torch\utils_contextlib.py:115, in context_decorator..decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)

File ~.conda\envs\pytorch\lib\site-packages\torch_geometric\explain\explainer.py:115, in Explainer.get_prediction(self, *args, **kwargs)
112 self.model.eval()
114 with torch.no_grad():
--> 115 out = self.model(*args, **kwargs)
117 self.model.train(training)
119 return out

File ~.conda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)

File ~.conda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None

TypeError: Model.forward() got multiple values for argument 'edge_label_index'

I'm not very familiar with the heterogeneous graph interpreter, can you help me explain this error? Thank you so much

[rusty1s]

Please note that your model should be defined as

    def forward(self, x_dict, edge_index_dict ,edge_label_index):

[Lucia61-star]

Thank you for your answer. I changed the model structure and ran it successfully.But for the results output by the interpreter, such as explanation.node_mask_dict and explanation.edge_mask_dict, I don't quite understand their specific meaning or practical significance. Is there any reference document about this interpreter that can help me understand the specific meaning of its output results?

[rusty1s]

I suggest to take a look at https://captum.ai/docs/extension/integrated_gradients.

In general, the higher the score in node_mask or edge_mask, the higher the importance of this node or edge is for the final prediction.

[Lucia61-star]

Thank you very much!