How to use monai.transforms (Dictionary Transforms) when there are more than 2 keys?

[jessie-chen99]

Hi! Thanks for your great work MONAI!
environment:

linux/ubuntu 
pytorch=1.13.0
monai=1.0.0

I already know how to solve a dataset like {"image1":xxx, "label1":xxx} and did not get any error, but still don't know how to do on dataset like {"image1":xxx, "image2":xxx, "label1":xxx}.

I have a dicom datasets which is like:

-trainset
    -img1 #dicom image
    -img2 #dicom image
    -label1 #segmentation annotation

ps. img1 and img2 are the raw image from same patient but different view/shape, img1 and label1 have the same shape.

I pack my input as:

train_files =  [{'image1': image_1, 'image2': image_2, 'label1': label_1} for image_1, image_2, label_1 in zip(img1_train_files, img2_train_files, label1_train_files)]

data transforms I used are:

    train_transform = transforms.Compose(
        [
            transforms.LoadImaged(keys=["image1", "image2", "label1"], image_only=True),
            transforms.RandCropByPosNegLabeld(
                keys=["image1", "image2", "label1"],
                label_key="label_1",
                spatial_size=(args.roi_x, args.roi_y),
                pos=1, 
                neg=1, 
                num_samples=args.num_samples,
                image_key="image1",
                image_threshold=0,
            ),
            transforms.RandFlipd(keys=["image1", "image2", "label1"], prob=args.RandFlipd_prob, spatial_axis=0),
            transforms.RandFlipd(keys=["image1", "image2", "label1"], prob=args.RandFlipd_prob, spatial_axis=1),
            transforms.RandRotate90d(keys=["image1", "image2", "label1"], prob=args.RandRotate90d_prob, max_k=3),
            transforms.ToTensord(keys=["image1", "image2", "label1"]),
        ]
    )

But I got some errors and don't know how to change my code.

2023-08-24 08:17:36,427 - > collate dict key "label_1" out of 3 keys
2023-08-24 08:17:36,444 - >> collate/stack a list of tensors
train: idx, data1, data2, lable 0 (10, 1, 512, 512) (10, 1, 512, 512) (10, 10, 512, 512)
Traceback (most recent call last):
  File "utils/data_utils_dual.py", line 203, in <module>
    for idx, batch_data in enumerate(loader[0]):
  File "utils/data/dataloader.py", line 668, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1361, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1387, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 475, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "/opt/monai/monai/data/utils.py", line 466, in list_data_collate
    ret = collate_meta_tensor(data)
  File "/opt/monai/monai/data/utils.py", line 435, in collate_meta_tensor
    return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0}
  File "/opt/monai/monai/data/utils.py", line 435, in <dictcomp>
    return {k: collate_meta_tensor([d[k] for d in batch]) for k in elem_0}
  File "/opt/monai/monai/data/utils.py", line 429, in collate_meta_tensor
    collated = default_collate(batch)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 140, in default_collate
    out = elem.new(storage).resize_(len(batch), *list(elem.size()))
  File "/opt/monai/monai/data/meta_tensor.py", line 249, in __torch_function__
    ret = super().__torch_function__(func, types, args, kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/_tensor.py", line 1089, in __torch_function__
    ret = func(*args, **kwargs)
RuntimeError: Trying to resize storage that is not resizable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/opt/monai/monai/data/utils.py", line 479, in list_data_collate
    raise RuntimeError(re_str) from re
RuntimeError: Trying to resize storage that is not resizable

Exception in thread Thread-13:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/opt/conda/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 28, in _pin_memory_loop
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 305, in rebuild_storage_fd
    fd = df.detach()
  File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/opt/conda/lib/python3.8/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/opt/conda/lib/python3.8/multiprocessing/connection.py", line 509, in Client
    deliver_challenge(c, authkey)
  File "/opt/conda/lib/python3.8/multiprocessing/connection.py", line 740, in deliver_challenge
    response = connection.recv_bytes(256)        # reject large message
  File "/opt/conda/lib/python3.8/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/opt/conda/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/opt/conda/lib/python3.8/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

ps. If I comment out the code about transforms.RandCropByPosNegLabeld, there is no error too, so I guess something happen inside transforms.RandCropByPosNegLabeld.

Hope someone could help me, thanks a lot!!!

[wyli]

if it's caused by the inconsistent shape of image1 and image2, I think you can try adding a monai.transforms.ResampleToMatchd("image2", "image1"), after loadimage. (converting this to a discussion for now)

[jessie-chen99]

thx! I convert image1 and image2 to same shape offline, this error is solved.