About result from custum dataloader

[Rojina99]

I have created a custom data loader to load pytorch geometric dataset

The original is a list of five (keeping it small for testing ) data looks like

[Data(x=[20, 75], edge_index=[2, 38]), Data(x=[20, 75], edge_index=[2, 36]), Data(x=[20, 75], edge_index=[2, 36]), Data(x=[12, 75], edge_index=[2, 22]), Data(x=[12, 75], edge_index=[2, 22])]

I am loading the dataset as

dataset = CustomDataset(states_pytorch_geom, states1_pytorch_geom, output_list) custom_loader = DataLoader(dataset, batch_size=4, shuffle=False)

Then I load the dataset in the loop as:

for data in custom_loader:
print(data['data'].x)
print(data['data'].edge_index)

The result shape of dataset in batch looks like

data['data'].x.shape
torch.Size([72, 75])

and for edge index looks like

data['data'].edge_index.shape
torch.Size([2, 132])

Since, the data is of shape 72,75 and shuffle is False I originally expected the edge index to have maximum index till 71. But the result I get looks like

tensor([[  0,   1,   1,   2,   2,   3,   3,   4,   4,   4,   5,   5,   6,   7,
           8,   9,   9,  10,  10,  10,  11,  11,  12,  12,  13,  13,  13,  14,
          14,  14,  15,  15,  16,  17,  18,  18,  19,  19,  41,  42,  42,  43,
          43,  43,  44,  44,  45,  45,  46,  46,  46,  47,  47,  48,  48,  49,
          49,  49,  50,  51,  51,  52,  53,  54,  54,  55,  55,  55,  56,  57,
          57,  58,  58,  59,  81,  82,  82,  83,  83,  83,  84,  84,  85,  85,
          86,  86,  86,  87,  87,  88,  88,  89,  89,  89,  90,  91,  91,  92,
          93,  94,  94,  95,  95,  96,  96,  96,  97,  98,  98,  99, 120, 121,
         121, 121, 122, 122, 123, 123, 123, 124, 125, 126, 127, 127, 128, 128,
         129, 129, 130, 130, 131, 131],
        [  1,   0,   2,   1,   3,   2,   4,   3,   5,   7,   4,   6,   5,   4,
           9,   8,  10,   9,  11,  19,  10,  12,  11,  13,  12,  14,  18,  13,
          15,  17,  14,  16,  15,  14,  13,  19,  10,  18,  42,  41,  43,  42,
          44,  48,  43,  45,  44,  46,  45,  47,  49,  46,  48,  43,  47,  46,
          50,  51,  49,  49,  52,  51,  54,  53,  55,  54,  56,  57,  55,  55,
          58,  57,  59,  58,  82,  81,  83,  82,  84,  88,  83,  85,  84,  86,
          85,  87,  89,  86,  88,  83,  87,  86,  90,  91,  89,  89,  92,  91,
          94,  93,  95,  94,  96,  95,  97,  98,  96,  96,  99,  98, 121, 120,
         122, 126, 121, 123, 122, 124, 125, 123, 123, 121, 128, 131, 127, 129,
         128, 130, 129, 131, 127, 130]])

Can someone let me know why the index are jumping from 59 to 81 not 61 ?

[rusty1s]

What happens if you run

for data in dataset:
    data.validate()

It looks like your edge_index may point to invalid indices.

[Rojina99]

My apologies, when I did

data['data'].validate()

I got error

*** ValueError: 'edge_index' contains larger indices than the number of nodes (72) in 'DataBatch' (found 131)

And when I did same for data1

data['data1'].validate()

I got TRUE.

I am quite not sure what is causing this issue.

[rusty1s]

How do you construct edge_index in the first place? It looks that something has gone wrong there.

[Rojina99]

I have an adjacency matrix from which I am generating edge index as

edge_index = torch.tensor(adjacency_matrix, dtype=torch.long).nonzero().t().contiguous()

[rusty1s]

Can you show me a minimal example to reproduce? I think this should work just fine as long as adjacency_matrix has the same size as your node features.

[Rojina99]

Sorry I could not respond earlier. Thank you so much for you help. Your insight on adjacency matrix help me to figure out the error. The issue is not related to custom data loader but the fact I am using the dataset somewhere else in code before passing to custom data loader so it got changed in the other function. I used deepcopy to solve the issue.