how to best implement random crop sample on data of different sizes

[jxchen01]

I am trying to implement a CycleGAN-like dataloader. I can think of several different ways to do this in MONAI. Currently I am trying the following:

# generate some synthetic data
A_list = []
B_list = []
for img_idx in range(10):
        # image for domain A
        domainA_ = np.random.randint(0, 2, size=(64, 256, 128)).astype(np.uint8)

        # different size image for domain B
        domainB_ = np.random.randint(0, 255, size=(36, 70, 70)).astype(np.uint16)

       # save the images
        A_name = os.path.join(tempdir, "imgA_" + str(img_idx) + ".tiff")
        B_name = os.path.join(tempdir, "imgB_" + str(img_idx) + ".tiff")
        OmeTiffWriter.save(domainA_, A_name, dim_order="ZYX")
        OmeTiffWriter.save(domainB_, B_name, dim_order="ZYX")
        A_list.append(A_name)
        B_list.append(B_name)
        
train_files = [
        {"imageA": image_A, "imageB": image_B}
        for image_A, image_B in zip(A_list, B_list)
]

Then, I can do the random cropping transform

RandSpatialCropSamplesd(
    keys=["imageA", "imageB"],
    random_size=False,
    num_samples=4,
    roi_size=(32, 64, 64),
)

However, because images from domain B is much smaller than images from domain A, RandSpatialCropSamplesd can only do the random cropping within the common intersection region between an image from domain A and the other from domain B. In this case, the cropping will always happen on a small corner of the image from domain A (because image from domain B is much smaller).

Is there a way to do the random crop sampling independently for keys=["imageA", "imageB"] ?

Many thanks!!
Jianxu

[Nic-Ma]

Hi @jxchen01 ,

Thanks for your interest here.
Can you crop the keys in 2 separate RandSpatialCropSamplesd transforms?
Or I maybe misunderstood your question?

Thanks

[jxchen01]

Dear @Nic-Ma , Here is a very simple example which you can use to reproduce the above issue.

https://github.com/MMV-Lab/demo_box/blob/main/monai_different_samplings.py

[Nic-Ma]

Hi @jxchen01 ,

Thanks for sharing the test program, I see the root cause: the first RandSpatialCropSamplesd transform will create a list of {patch, other keys}, then all the following transforms will execute on each item of the list. So if you set multiple RandSpatialCropSamplesd transforms, it becomes nested lists.
Going back to your original issue, can you resize the images from domain A and domain B first, then do the cropping?

Thanks.

[jxchen01]

thanks for the update. My original question is what is the most reasonable way to implement a cycle-gan like dataloader. Suppose we have a bunch of images from domain A and another bunch of images from domain B, I think the root question is how to do two independent sampler on images from domain A and images from domain B. Even if we assume all images in domain A and all images in domain B are of the same size, we still cannot make the sampling on domain A and domain B independent from each other.

Any suggestion will be appreciated.

[Nic-Ma]

Hi @jxchen01 ,

Can you use non-samples transform in your case? For example, just use RandSpatialCrop instead of RandSpatialCropSamplesd, if so, you can just use 2 separate RandSpatialCrop transforms.

Thanks.

[jxchen01]

Dear @Nic-Ma , after getting rid of the sampling and using plain random spatial crop, it works. I think this is good for now. Thanks for your help!

In the long run, i would still be very interested in developing an efficient dataloader for the following use case:

I want to learn an unpaired transformation in 3D microscopy images between domain A and domain B, while those images might be very large (e.g., 10x larger than the patch size used for training). In addition, the number of images and the size of each image in domain A and domain B could be different.

(this is not a rare application, especially in high throughput bioimaging problems, and becoming more and more common as the data volume gets larger and larger).