Data loading with padding of an average number of nodes

[shrimonmuke0202]

How can I fix the padding of edge_index in the data loader for a fixed dimension?
For e.g: if my maximum number of nodes in the graph is 46 but I want all the edge indexes with an average number of nodes which is around 43. In this case, how can I fix the padding length of each batch with a fixed size 43?

[rusty1s]

Can you give me an example on what you are trying to do exactly? If you want to have all graphs to be of size 43, you need to either pad/filter out features in x and edges in edge_index individually:

def fix_dimension(data):
    if data.num_nodes < 43:
        data.x = torch.cat([data.x, torch.zeros(43 - data.num_nodes, data.x.size(-1))], dim=0)
        mask = (data.edge_index[0] < 43) & (data.edge_index[1] < 43)
        data.edge_index = data.edge_index[:, mask]
    elif data.num_nodes > 43:
       data.x = data.x[:43]
    return data

dataset = MyDataset(transform=fix_dimension)

[shrimonmuke0202]

I want the shape of all x and adj must be the same in all batch

[rusty1s]

I see. Doing

def fix_dimension(data):
    if data.num_nodes < 28:
        data.x = torch.cat([data.x, torch.zeros(28 - data.num_nodes, data.x.size(-1))], dim=0)
        mask = (data.edge_index[0] < 28) & (data.edge_index[1] < 28)
        data.edge_index = data.edge_index[:, mask]
    elif data.num_nodes > 28:
       data.x = data.x[:28]
    data.num_nodes = 28
    return data

should fix this.

[shrimonmuke0202]

If I take 22 since the minimum number of nodes is 22 in the graph then the code is not executed in the google colab i.e session crash occurs,

why this happens

[shrimonmuke0202]

The session does not crash for 26 but for 22 the session crash occurs

[rusty1s]

Oh yeah, I should not write code without testing it :D The correct code is

import torch
from torch_geometric.data import DataLoader
from torch_geometric.datasets import TUDataset
from torch_geometric.utils import to_dense_batch, to_dense_adj

N = 22


def fix_dimension(data):
    if data.num_nodes < N:
        data.x = torch.cat(
            [data.x, torch.zeros(N - data.num_nodes, data.x.size(-1))], dim=0)
    elif data.num_nodes > N:
        data.x = data.x[:N]
        mask = (data.edge_index[0] < N) & (data.edge_index[1] < N)
        data.edge_index = data.edge_index[:, mask]
    data.num_nodes = N
    print(data.edge_index.max())
    return data


dataset = TUDataset('/tmp/MUTAG', name='MUTAG', transform=fix_dimension)
loader = DataLoader(dataset, batch_size=32, drop_last=True)

for data in loader:
    x, _ = to_dense_batch(data.x, data.batch)
    adj = to_dense_adj(data.edge_index, data.batch)
    print(x.shape, adj.shape)

[shrimonmuke0202]

Thank you this code works.

…

On Fri, Aug 20, 2021 at 9:26 PM Matthias Fey ***@***.***> wrote: Oh yeah, I should not write code without testing it :D The correct code is import torchfrom torch_geometric.data import DataLoaderfrom torch_geometric.datasets import TUDatasetfrom torch_geometric.utils import to_dense_batch, to_dense_adj N = 22 def fix_dimension(data): if data.num_nodes < N: data.x = torch.cat( [data.x, torch.zeros(N - data.num_nodes, data.x.size(-1))], dim=0) elif data.num_nodes > N: data.x = data.x[:N] mask = (data.edge_index[0] < N) & (data.edge_index[1] < N) data.edge_index = data.edge_index[:, mask] data.num_nodes = N print(data.edge_index.max()) return data dataset = TUDataset('/tmp/MUTAG', name='MUTAG', transform=fix_dimension)loader = DataLoader(dataset, batch_size=32, drop_last=True) for data in loader: x, _ = to_dense_batch(data.x, data.batch) adj = to_dense_adj(data.edge_index, data.batch) print(x.shape, adj.shape) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2995 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ7MTCRHFWTLOEECOTUGOWDT5Z3JVANCNFSM5CNU22HQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .