BatchSampler mistakes batch size as channel number

[lxmedai]

import torch
from torch.utils.data import SequentialSampler, BatchSampler, DataLoader

def data_loader(dataset, indices_subset, batch_size=5, drop_last=False):
    pin_memory = torch.cuda.is_available()
    return DataLoader(dataset, sampler=BatchSampler(SequentialSampler(indices_subset), 
                                  batch_size=batch_size, drop_last=drop_last), pin_memory=pin_memory)

I use the batch sampler for my data loader and it yields each batched data with size of (1, 5, 256, 256, 256) if I run the enumerator and check the data size like:
for i, (X, _) in enumerate(dataloader): print(X.shape)
I however would like it to be size to be (5, 1, 256, 256, 256) as channel number should be 1 and the batch size should be 5 instead.
I can permute the data after calling emumerate to get right size but I don’t want to modify the standardized module of SupervisedTrainer (imported from monai.engines) for the downstream processing which I intend to use for my project. How should I customize the dataloader definition so that it will provide batched input data of the right size to a 3D classification model by itself and I don’t need to modify any imported packages which I use in the downstream? I thank anyone who would like to help out in advance!

[wyli]

the code example looks like a pure pytorch implementation, could you provide more details, I guess the dataset is some dataset + transforms from monai?

[wyli]

the data shape becomes naturally (5, 256, 256, 256), but then it creates a problem for my downstream using SupervisedTrainer (imported from monai.engines) and I get a RuntimeError complaining about the incompatibility with the weight size

I think that's the direction to go, if the __getitem__ of the dataset instead returning a 1x256x256x256 array, then I think SupervisedTrainer and the network will receive shape(5, 1, 256, 256, 256), which should be compatible with 3D networks. Batchsize=5,numchannels=1,spatial dims=256,256,256.

[lxmedai]

How would you suggest to modify the code for the __getitem__ method? Now the returned 'inputs' of the __getitem__ method provides me with a batch instead of one single image.

[wyli]

I think you can start from this example and modify it accordingly:

import numpy as np
from torch.utils.data import BatchSampler, DataLoader, Dataset, SequentialSampler

class MyDataset(Dataset):
    def __getitem__(self, index):
        return np.zeros((1, 256, 256, 256)), np.zeros((2, 256, 256, 256))

def data_loader(dataset, indices_subset, batch_size=5, drop_last=False):
    return DataLoader(
        dataset,
        batch_sampler=BatchSampler(SequentialSampler(indices_subset), batch_size=batch_size, drop_last=drop_last),
    )

for i, (x, y) in enumerate(data_loader(MyDataset(), range(50))):
    print(x.shape, y.shape)

torch.Size([5, 1, 256, 256, 256]) torch.Size([5, 2, 256, 256, 256])
torch.Size([5, 1, 256, 256, 256]) torch.Size([5, 2, 256, 256, 256])

it's a generic pytorch question, I'd suggest please post further questions to the pytorch forum, so that potentially more users may benefit from the Q&A.

[lxmedai]

I tested adding inputs = torch.unsqueeze(inputs, 1) after the line of inputs = self.transform(self.inputs[index]), and now it is working. I have previously also posted my question there and the current solution has also been posted in pytorch forum. Thanks for your help!