Random tranforms are applied to each channel independently?

[lidialuq]

I'm trying to train a model to segment the BraTs dataset, and I'm using this tutorial as a starting point.

My understanding is that augmentation transforms (such as RandFlipd) should be applied randomly BUT equally for all channels in the image (in this case MRI sequences) and the labels. It seems like what happens instead is that the transform is applied randomly to each channel (both of image and label), so that they don't necessary match after the transform. Each sequence is flipped/not flipped independently of each other, and the same happens with the labels.

This can be reproduced by running the tutorial changing val_ds for train_ds in 'Check data shape and visualize' (random transforms are only applied to the train dataset, so the validation dataset, which the tutorial plots, is fine).

Any tips on how to avoid this are much appreciated!

[Nic-Ma]

Hi @lidialuq ,

Thanks for your interest here.
I think we applied map_spatial_axes in RandFlip to flip all the channels together:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/transforms/spatial/array.py#L362
Could you please help share some test result images that the random operation on different channel is different?

Thanks in advance.

[lidialuq]

Hi @Nic-Ma , thanks for the prompt response.
Here's an image showing the problem. The first image shows an axial plane of one of the patients in the BraTs dataset without doing any random transformation (like RandFlipd). The second images shows the same planes, the only difference is that I add RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=0). Cahnnels 0, 1 and 3 are flipped, but channel 2 isn't. This doesn't just happen with RandFlip, I have the same problem with RandAffine for example. Here's my code in case it helps, though it's mostly the same as the MONAI tutorial I linked to.

Here's the output of print_config():

MONAI version: 0.7.0
Numpy version: 1.21.2
Pytorch version: 1.10.0
MONAI flags: HAS_EXT = False, USE_COMPILED = False
MONAI rev id: bfa054b9c3064628a21f4c35bbe3132964e91f43

Optional dependencies:
Pytorch Ignite version: 0.4.5
Nibabel version: 3.2.1
scikit-image version: 0.18.3
Pillow version: 8.4.0
Tensorboard version: 2.7.0
gdown version: 4.2.0
TorchVision version: 0.11.1
tqdm version: 4.62.3
lmdb version: 1.2.1
psutil version: 5.8.0
pandas version: 1.3.4
einops version: 0.3.2
transformers version: 4.12.5

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

[rijobro]

Hi @lidialuq, every time you call train_ds[2], you are recomputing the random transforms. And since you have set_determinism, you will see the same results every time you call this script.

If you do something like the following that should hopefully solve your problem.

data = train_ds[2]
# now visualise but only use `data` don't re-call `train_ds[2]`
...

[lidialuq]

That makes sense (and works). I've spent way too long trying to figure this out, it's super helpful to be able to get help here. Thanks!