Optimize my use of Dataset and DataLoader at inference

[pcanas]

My team is currently using the following code snippet to generate a monai.data.DataLoader for inference:

        ds_p = monai.data.Dataset([deepcopy(image) for _ in range(5)], transform=get_tta_tfms())
        dl_p = monai.data.DataLoader(ds_p, batch_size=1, num_workers=1, shuffle=False)

Where get_tta_tfms() is a function that retrieves a Compose of monai transforms.

I would like to know if there is a better way of achieving the same result for optimal performance.

Thanks!

[KumoLiu]

Hi @pcanas, sorry for I didn't get your question. You can simply set transform=Compose(...) in the Dataset.
Thanks!

[pcanas]

No, that is not what I wanted to ask. My point is that creating a Dataset of 5 deep copied image just to later send them 1 by 1 into the DataLoader, it does not sound efficient. I remember with pytorch you could just use the collate_fn argument and achieve similar results.

My question is, how can I create this DataLoader in the most efficient way.

[KumoLiu]

Hi @pcanas, I think you can use RepeatChanneld and SplitDimd to achieve the same result.

MONAI/monai/transforms/utility/dictionary.py

Line 332 in 6f8e63f

class RepeatChanneld(MapTransform):

MONAI/monai/transforms/utility/dictionary.py

Line 382 in 6f8e63f

class SplitDimd(MapTransform, MultiSampleTrait):

Hope it can help you, thanks!

[pcanas]

Hi @KumoLiu thanks!
Could you provide a code example?

[KumoLiu]

Here is a simple demo.

trans = Compose([
         LoadImaged(...),
         EnsureChannelFirstd(...),
         RepeatChanneld(..., repeats=5),
         SplitDimd(..., dim=0, list_output=True)
])
ds_p = monai.data.Dataset([{"img": image_path}], transform=trans)

BTW, you can also specify your own collate_fn in MONAI.

MONAI/monai/data/dataloader.py

Lines 86 to 87 in 6f8e63f

	if "collate_fn" not in kwargs:
	kwargs["collate_fn"] = list_data_collate

[pcanas]

I am not sure whether this achieves the same results... This would create the same transformed image 5 different times, whereas the original code creates 5 different transformed images.

Am I right?

[KumoLiu]

You can add any other transforms for each key after SplitDimd(maybe with list_output=False), then you will get five different results.

[pcanas]

I see, thank you!!