Using HDF-Datasets with MONAI

[ysuter]

I would like to use custom HDF5 datasets with h5py for speed. I previously used this with PyTorch and pymia, but I have trouble figuring out how I can fit that into the dataset module of MONAI. I only tried to put the initialized hdf dataset into data.Dataset and data.IterableDataset. I am grateful for any hints in the right direction :)

[KumoLiu]

Hi @ysuter, in monai we often use LoadImage to load data which I think doesn't support hdf5 now. You could load data with h5py first then use Dataset in monai. A typical input data like this:

MONAI/monai/data/dataset.py

Lines 69 to 75 in ab800d8

	For example, typical input data can be a list of dictionaries::
	[{ { {
	'img': 'image1.nii.gz', 'img': 'image2.nii.gz', 'img': 'image3.nii.gz',
	'seg': 'label1.nii.gz', 'seg': 'label2.nii.gz', 'seg': 'label3.nii.gz',
	'extra': 123 'extra': 456 'extra': 789
	}, }, }]

Also, you could try to write a custom transform may be called LoadHdf5, then combine all the transforms.
For how to write custom transform, you could refer to this discussion. #4809

Hope these can help you, thanks!

[youwei1-sudo]

Hi, @KumoLiu. So, if I understand correctly, the process would involve first loading the h5py file into a temporary object(or create temporary files). Then, we would generate an indexical key for each image and label pair within the h5py object, which will serve as the input for the MONAI Dataset (DS) object. Finally, we would implement a transform that takes the temporary object and assigns actual values to the image and label pairs, similar to the LoadImage transform. However, in this case, instead of loading from files using paths, we would be loading from the previously dereferenced h5py object.

I would be very appreciate for any feedbacks.

[KumoLiu]

Hi @youwei1-sudo, yes you are right.