Problem creating a custom dataset for PyTorch geometric

[fedeotto]

Hello,

After having created my dataset on top of inorganic chemical compositions I have a dataset object which returns a tuple made by a feature matrix (just some one-hot-encoding of elements in this case) for a given composition and a target value.

dataset.__getitem__(0)

(tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0.],
         [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0.],
         [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
          0., 0., 0., 0.]]),
 tensor([0.4000]))

Now, to feed these data through a GCN model, at least in my understanding, I have to convert the current dataset in a new version that these (already implemented) models can read. So I'm simply doing

data_list = [Data(x=train, y=target) for train,target in dataset]

[Data(x=[3, 112], y=[1]),
 Data(x=[2, 112], y=[1]),
 Data(x=[2, 112], y=[1]),
 Data(x=[3, 112], y=[1]),
 Data(x=[2, 112], y=[1]),
 Data(x=[2, 112], y=[1]),
 Data(x=[3, 112], y=[1]),
 Data(x=[3, 112], y=[1]),
 Data(x=[4, 112], y=[1]),
 Data(x=[4, 112], y=[1]),
 Data(x=[2, 112], y=[1]),
 Data(x=[4, 112], y=[1]),
...

and finally creating a DataLoader object simply doing:

data_loader = DataLoader(data_list, batch_size=1)

The problem is that when I try to iterate over data_loader,

for train, target in data_loader:
print(train)
break

I get the following error:

ValueError                                Traceback (most recent call last)
<ipython-input-63-78597716eee4> in <module>
----> 1 for i,j in data_loader:
      2     print(i)
      3     break

ValueError: too many values to unpack (expected 2)

[wsad1]

Iterating through the Dataloader object should return a single Data object. So the following should work.

for data in data_loader:
    train = data.x
    target = data.y

[fedeotto]

Thanks! yes it's a bit different than using DataLoader for usual ANNs or CNNs in torch.

What I still don't understand is that if I look now at train.shape what I actually expect is having a 3-dimensional tensor with shape

[batch_size, num_graphs, num_features]

but instead it returns something like

[274, 112]

Now 112 represents features length, but what is 274? Where is the batch_size?