About using Hypergraph Attention to reproduce the accuracy on the Cora dataset

[HPUtx813]

Hi, I was recently reading the paper "Hypergraph Convolution and Hypergraph Attention" and I would like to thank PyG for providing the source code of this model. I want to try to use Hypergraph Attention to do experiments on Cora to reproduce the correct rate of 82.61% in the original paper.
I found that I had to pass in a variable called hyperedge_attr in the forward function. I don't know where this variable comes from, I tried using the mean of the nodes connected by the hyperedge as the feature of the hyperedge, but this works poorly, does anyone know how to do this?
Below is the source code of my model. Thanks again!

import matplotlib.pyplot as plt
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch_geometric
import torch_geometric.nn as pyg_nn
import torch_scatter
from torch_geometric.datasets import Planetoid


class HGNN(nn.Module):
    def __init__(self, **kwargs):
        super().__init__()
        self.hgnn1 = pyg_nn.HypergraphConv(1433, 256, **kwargs)
        self.hgnn2 = pyg_nn.HypergraphConv(256, 7, **kwargs)
        self.dropout = nn.Dropout(0.2)

    def get_hyperedge_attr(self, x, edge_index):
        hyperedge_attr = torch_scatter.scatter_mean(x[edge_index[0]], edge_index[1], dim=0)
        return hyperedge_attr

    def forward(self, x, edge_index):
        x = self.hgnn1(x, edge_index, hyperedge_attr=self.get_hyperedge_attr(x, edge_index))
        x = F.relu(x)
        x = self.dropout(x)
        x = self.hgnn2(x, edge_index, hyperedge_attr=self.get_hyperedge_attr(x, edge_index))
        return x


def get_h():
    dataset = Planetoid(root='../data', name="Cora")
    edge_index = torch_geometric.utils.add_self_loops(dataset.data["edge_index"])[0]
    train_mask = dataset.data["train_mask"]
    test_mask = dataset.data["test_mask"]
    features = dataset.data["x"]
    labels = dataset.data["y"]
    return edge_index, train_mask, test_mask, features, labels


@torch.no_grad()
def test(model, adj_matrix, features, labels, test_mask):
    model.eval()
    total = test_mask.sum()
    outputs = model(features, adj_matrix)
    predicts = torch.argmax(outputs[test_mask], dim=1)
    correct = torch.sum(predicts == labels[test_mask])
    acc = correct.item() / total
    return acc


def train(model, epochs, lr, device="cuda:0", dtype=torch.float16, optimizer="SGD", weight_decay=5e-4):
    device = torch.device(device)
    # prepare data
    edge_index, train_mask, test_mask, features, labels = get_h()
    edge_index = edge_index.to(device, torch.long)
    features = features.to(device, dtype)
    labels = labels.to(device, torch.long)

    # prepare model
    model = model.to(device, dtype)

    # Prepare loss function and optimizer
    loss_fn = nn.CrossEntropyLoss()
    if optimizer == "SGD":
        optimizer = torch.optim.SGD(model.parameters(), lr=lr, weight_decay=weight_decay)
    elif optimizer == "Adam":
        optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=weight_decay)
    else:
        raise ValueError("optimizer must be SGD or Adam")
    best_acc = 0
    loss_array = []
    acc_array = []
    for epoch in range(epochs):
        model.train()
        optimizer.zero_grad()
        output = model(features, edge_index)
        loss = loss_fn(output[train_mask], labels[train_mask])
        loss.backward()
        optimizer.step()
        train_acc = test(model, edge_index, features, labels, train_mask)
        acc = test(model, edge_index, features, labels, test_mask)
        if acc > best_acc:
            best_acc = acc
        print("epoch: %d, loss: %.4f acc: %.3f train_acc: %.3f" % (epoch, loss.item(), acc, train_acc))
        loss_array.append(loss.item())
        acc_array.append(acc)
    print("best acc: %.3f" % best_acc)
    plt.plot(loss_array, label="loss")
    plt.plot(acc_array, label="acc")
    plt.legend()
    plt.savefig("loss_acc.png")
    plt.show()
    return loss_array, acc_array


if __name__ == '__main__':
    # model = HGNN(head=8, concat=False, use_attention=False, dropout=0.2) # acc: 81%
    model = HGNN(head=8, concat=False, use_attention=True, dropout=0.2)  # low accuracy
    train(model, 1000, 0.005, "cuda:0", dtype=torch.float32, optimizer="Adam")

[rusty1s]

The paper sadly doesn't give that much detail about how it actually constructs its hypergraph. Did you try to look at its source code? For constructing edge_index, you will need to convert this into an incidence matrix in which case each node maps to a hyperedge, e.g., the sparse indices of the matrix of shape [num_nodes, num_edges].

[rusty1s]

In the worst case, you can construct hyperedge_attr from your features, e.g., via x[hyperedge_index[0]].

[HPUtx813]

Thanks for your reply. I tried it and it doesn't seem to work(maybe I'm wrong). From beginning to end, I was just interested in how the author of the paper trained Hypergraph Attention on the Cora data set and obtained a relatively good result. As far as I know, the Cora dataset does not provide edge features, so I'm curious how the paper author got it.
It's a pity that the authors of the original paper did not provide the source code of their training model (I can't find the source code). this is my code.If I have any mistakes please tell me.😟
hypergraph-attention.zip

[rusty1s]

In the code, you are applying scatter_mean to obtain hyperedge_attr. This should work if you are using hyperedge_attr = x[edge_index[0]] instead. Can you confirm?

[HPUtx813]

I have already tried this in the uploaded HGNN.py file. It doesn't work.

[rusty1s]

Yeah, sorry. Looks like I was wrong. You indeed need to use scatter_mean since hyperedge_attr refers to the number of hyperedges in your data, i.e. hyperedge_index[1].max() + 1.