Detail of graph varational autoencoder KL loss

[xiachenrui]

Hi, all

I found the PyG give an example of GVAE, and it calculated KL loss in the following way

    if args.variational:
        loss = loss + (1 / train_data.num_nodes) * model.kl_loss()

But I think it show be divided by the dimension of node features rather than the node numbers , such as

    if args.variational:
        loss = loss + (1 / train_data.data.shape[1]) * model.kl_loss()

Please let me know your ideas

Best !

Chenrui

[rusty1s]

This is directly taken from here. Since the KL loss is operating on a dense N^2 matrix, I think this is correct.