Are both the images and masks/segs transformed identically in the 2d_segmentation tutorials?

[GamblerEd]

I am currently experimenting with the torch codes on 2d-segmentation (from the Project-MONAI/tutorials repository).

There is one general part in these codes that confuse me a bit though.
For example in unet_training_array.py:

# define transforms for image and segmentation
    train_imtrans = Compose(
        [
            LoadImage(image_only=True),
            AddChannel(),
            ScaleIntensity(),
            RandSpatialCrop((96, 96), random_size=False),
            RandRotate90(prob=0.5, spatial_axes=(0, 1)),
            EnsureType(),
        ]
    )
    train_segtrans = Compose(
        [
            LoadImage(image_only=True),
            AddChannel(),
            ScaleIntensity(),
            RandSpatialCrop((96, 96), random_size=False),
            RandRotate90(prob=0.5, spatial_axes=(0, 1)),
            RandWarp_gvtulder(prob=1.0, sigma = 25, points=3, mode='constant',),
            EnsureType(),
        ]
    )

So according to the code, the sequences of various transform-operations are executed on the training images independently from the sequences of transforms which are executed on the segmentation masks (of those images).
Does that actually mean that the images will be transformed/augmented/deformed etc. totally differently from the segmentation masks, so the resulting images will not necessarily be transformed exactly like the corresponding masks at all, as if they are unrelated to each other?

But I mean doesn't it logically make sense if every training image and its corresponding segmentation masks are both transformed identically to be used in the UNet, or is that actually indeed what the program does at

train_ds = ArrayDataset(images[:20], train_imtrans, segs[:20], train_segtrans)

?

Has this maybe also something to do with the notorious self.R-random state used by these transform-classes btw?
Sry but it's still not entirely clear to me what self.R does exactly, is it supposed to "synchronize" all random transformations
listed in the Compose-chain with one identical seed value?
Does it synchronize the transform-chain of the training images with that of the segmentation images, too?

Someone please help me out on this issue, I'd appreciate any response!

Thanks in advance!

[yiheng-wang-nv]

Hi @GamblerEd ,

is that actually indeed what the program does at train_ds = ArrayDataset(images[:20], train_imtrans, segs[:20], train_segtrans)

Yes, for all Randomizable transforms, same seed will be used (see here)

is it supposed to "synchronize" all random transformations

Yes, let me take the transform RandShiftIntensity for instance.
When trying to load data from ArrayDataset, the __getitem__ function will assign the same seed for all transforms (as shown in the link posted above). The line transform.set_random_state(seed=self._seed) is used to set self.R = np.random.RandomState(_seed).
If we check the source code of RandShiftIntensity, we will see its random place is to randomly select an offset value and it is determined via:

self._offset = self.R.uniform(low=self.offsets[0], high=self.offsets[1])

What we need to check here is to see if self.R.uniform(...) will return the same results for different instances that have the same R. You can run the following code for multiple times to confirm it:

import numpy as np

seed = 1
r1 = np.random.RandomState(seed)
r2 = np.random.RandomState(seed)

print(r1.uniform(0, 1))
print(r2.uniform(0, 1))

You can also run the following code to confirm if two RandShiftIntensity instances will return same values if they have the same seed:

from monai.transforms import RandShiftIntensity
import torch

seed = 1
offset = 2

shift_image = RandShiftIntensity(offsets=offset, prob=1.0)
shift_mask = RandShiftIntensity(offsets=offset, prob=1.0)

shift_image.set_random_state(seed=seed)
shift_mask.set_random_state(seed=seed)

print(shift_image.R, shift_mask.R)

example = torch.rand([3, 3])

print("image shift result: ")
print(shift_image(example))
print("mask shift result: ")
print(shift_mask(example))