AttributeError: 'list' object has no attribute 'size'

[yancychy]

Hi, my datalist is a list of tuple objects (graph1, graph2).
Then I create dataloader by

all_loader = DataLoader(dataL, batch_size=batch_size, shuffle=False)
print(len(all_loader))

When I ran the code, it reported error AttributeError: 'list' object has no attribute 'size'. My previous code was running without any error. But currently, it failed. I appreciate if you could give any suggestion.

for data in all_loader:
       data1, data2 = data

The errors are as follows:

    for data in loader:  # Iterate in batches over the training dataset.
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 677, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/loader/dataloader.py", line 35, in __call__
    return [self(s) for s in zip(*batch)]
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/loader/dataloader.py", line 35, in <listcomp>
    return [self(s) for s in zip(*batch)]
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/loader/dataloader.py", line 20, in __call__
    return Batch.from_data_list(batch, self.follow_batch,
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/data/batch.py", line 76, in from_data_list
    batch, slice_dict, inc_dict = collate(
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/data/collate.py", line 85, in collate
    value, slices, incs = _collate(attr, values, data_list, stores,
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/data/collate.py", line 132, in _collate
    slices = cumsum([value.size(cat_dim or 0) for value in values])
  File "/Users/cheny48/miniconda/lib/python3.9/site-packages/torch_geometric/data/collate.py", line 132, in <listcomp>
    slices = cumsum([value.size(cat_dim or 0) for value in values])
AttributeError: 'list' object has no attribute 'size'

python 3.9.12
torch 2.0.1
torch-geometric 2.3.1

[akihironitta]

I think PyG's DataLoader doesn't support "a list of tuple objects", so you'll have to write your own collate_fn.

• https://pytorch-geometric.readthedocs.io/en/2.3.1/modules/loader.html#torch_geometric.loader.DataLoader
• https://pytorch.org/docs/2.0/data.html?highlight=collate_fn

[rusty1s]

This should be already handled within our collate_fn, see https://github.com/pyg-team/pytorch_geometric/blob/master/torch_geometric/loader/dataloader.py#L12.

[yancychy]

Thanks. It works for tuple list.

[rusty1s]

Do you have a small example to reproduce? The DataLoader should be able to handle lists of tuples. The following code works just fine for me:

import torch

from torch_geometric.data import Data
from torch_geometric.loader import DataLoader

data = Data(x=torch.randn(5, 4))

data_list = [(data, data), (data, data)]

loader = DataLoader(data_list, batch_size=2)
for batch in loader:
    print(batch)

[yancychy]

Thanks. This code works.

When I ran my code, it works when I set bachsize = 1. But it failed when I set bachsize>1. I guess my data format may be wrong.

print(augList[0])

#(Data(x=[10, 36], edge_index=[2, 40], edge_attr=[40], y=-6.801296710968018, yL=0.0, weightV=0.007874015718698502), 
#Data(x=[10, 36], edge_index=[2, 40], edge_attr=[40], y=-6.801296710968018, yL=0.0, weightV=0.007874015718698502))

from torch_geometric.data import Data
from torch_geometric.loader import DataLoader

all_loader = DataLoader(augList, batch_size=batch_size, shuffle=False)
print(len(all_loader))

for batch in all_loader:
    print(batch)
    break

[rusty1s]

Can you share/upload a sample of your data so that I can take a look?

[yancychy]

Yes. I shared the sample data to your email by google drive.
I tested it again. I can work at batch size 1-8. But failed on 9 and after.

import torch
import torch_geometric
from torch_geometric.data import Data
from torch_geometric.loader import DataLoader

subList = torch.load('subList64.pt')

batch_size = 16
all_loader = DataLoader(subList, batch_size=batch_size, shuffle=False)
print(len(all_loader))

for data in all_loader:  # Iterate in batches over the training dataset.
    print(len(data))
    d1, d2 = data
    print(d1)
    print(d2)
    break

[rusty1s]

The problem is that one of your data objects contains aid as a list rather than as a tensor. I fixed this via

subList = torch.load('subList64.pt')
for data in subList:
    for d in data:
        if not isinstance(d.aid, torch.Tensor):
            d.aid = torch.tensor(d.aid)

[yancychy]

Thank very much. Yes. I did not use tensor format for it.