Dictionary-based transformation for same transforms on images and labels

[elahe-badali]

Hello @Nic-Ma, I hope you well.
I want to use dictionary based transformation to get same transform on both images and labels. My dataset images are 3d cardiac MRI images and labels are binary masks.
But it seems that is not working.
My code :

train_transforms = Compose(
    [   AddChanneld(keys=["image", "label"]),
        RandFlipd(keys=["image", "label"]),
        RandRotate90d(keys=["image", "label"],prob=0.10 ),
        ToTensord(keys=["image", "label"]),
    ]
)

train_dataset = Dataset( data=train_dataset, transform=train_transforms )

sample_num = 110
img = train_dataset[sample_num]['image'].squeeze(0)
lbl = train_dataset[sample_num]['label'].squeeze(0)
fig , (ax1 , ax2 , ax3) = plt.subplots(1 , 3 , figsize = (8 , 8))
ax1.imshow(img[1 , : , :])
ax2.imshow(img[2 , : , :])
ax3.imshow(lbl)

When i run it, i get this results ( first and second slice of MRI and label showed):

and when run show code again i get this:

I have 2 questions:
First, It seems that "Flip" is worked just on labels. Why the same transformation is not applied two both images and labels?
Second, Why there are two different results for the same code?

[rijobro]

Hi, every time you access an element in the dataset, the transforms will be randomised. Hence, in this code snippet:

img = train_dataset[sample_num]['image'].squeeze(0)
lbl = train_dataset[sample_num]['label'].squeeze(0)

there is a different random state between the img and lbl, and therefore different transforms are applied to them. If you instead do:

data = train_dataset[sample_num] 
img = data['image'].squeeze(0)
lbl = data['label'].squeeze(0)

this will hopefully sort your problem.

[elahe-badali]

Thank you for sharing jupyter file. I run it and it worked. I think i find where the problem comes from. I use "Compose" to create transform.
Could i ask to you add below lines to your jupyter and run it?

This lines as transform:

from monai.transforms import RandRotated , Compose, AddChanneld, RandRotate90d, ToTensord

transform = Compose(
    [
        AddChanneld(keys=["img" , "lbl"]),
        RandRotate90d(keys=["img", "lbl"], prob=0.7),
        ToTensord(keys=["img", "lbl"]),
    ]

)

And this line to show (i just add "squeeze" to remove added channel):

d = ds[0]
img = d["img"].squeeze(0)
lbl = d["lbl"].squeeze(0)
print(img.shape, lbl.shape)
imshow(img[0], lbl[0])

And run last cell multiple times to see results. In some case i get :

If my transformation definition has problems, Could you please code correct one with "Compose"?

[rijobro]

Hi, I don't know how you intend to add a channel to your image given that it already has the channel dimension attached. If the image is (25, 100, 100), where number of channels = 25, then you can't use that in conjunction with AddChanneld.

I took my previously attached notebook and wrapped the RandRotated in a Compose operator:

Compose([
    RandRotated(["img", "lbl"], prob=1.0, range_x=np.pi, keep_size=True)
])

and the results were the same.

I'm not sure what you're doing in terms of squeezing and indexing -- lbl.squeeze(0) will remove the channel dimension, as will lbl[0], so I think there might be some errors in your additions to the notebook. Try and start again with my example and add things in piece-by-piece until you can convince yourself that if you access an element of the dataset in one go, the label and image will have the same random instance of the transforms applied to them.

[elahe-badali]

For example take a look at https://pytorch.org/docs/stable/generated/torch.nn.Conv3d.html. Conv3d's input has the shape (N,C,D,H,W). In my case 25 is not showing channels. 25 is "D" in the input shape and shows depth of 3d image. "C" is channel and in my dataset because of 25 gray images i want set the channel to 1. For this, i used "AddChanneld" to set input shape to (1,25,64,64) and later in loader added batch dimension to it.

[rijobro]

Ok, I think you're saying that your image is like this:
C=1, D=25, H=64, W=64
and your label is like this:
C=1, H=64, W=64.
The problem is that if you apply a rotation around the first spatial dimension, and in your image the first spatial dimension, (D=25) is different from the first spatial dimension of your label (H=64), then a rotation around the first spatial axis will give you different results.

Why don't you try moving the depth component to the end of the image. In this way, you'll have image:
C=1, H=64, W=64, D=25
and label:
C=1, H=64, W=64, D=1

[elahe-badali]

I think it's too late but thank you @rijobro, your last comment fix the problem.