GradCAM visualisation with pytorch-lightning

[ik362]

Hi there,
I am using pytorch-lightning to create a custom model and I would like to use GradCAM to visualise the output of my self-attention layer.

So far I have not found any documantation/examples on how MONAI can be implemented within a pytorch-lightning module so I wanted to ask: how MONAI visualisations could be implemented in a pytorch-lightning module?

Thanks for any help/suggestions!

[wyli]

There is a thread might be useful: #1598

[ik362]

Hi @Nic-Ma and @rijobro,

I created a minimal example as a colab notebook: https://colab.research.google.com/drive/1DdqFkdFscZosFgXlbHxgGsuZMHDCTIag?usp=sharing

Thanks for having a look, much appreciated!

[rijobro]

Ignoring the GradCAM for a second, does your current code give you results that you expect?

The code is telling me that you're using 2D images with spatial dimensions of (229, 193), and that there are 193 channels. Is this correct? I just want to check that you aren't trying to create 3D images with spatial dimensions of (193, 229, 193).

[ik362]

Hi @rijobro,

Thanks for taking a look (you will have to forgive me I am a relative newbie), you are correct: my aim is to use an entire 3D T1w scan rather than a stack of 2D images. I changed the dummy data so that the data now has size[1, 193, 229, 193], i.e. 1 channel and spatial dimensions (193, 229, 193). Is this correct?

Thanks!

[ik362]

Hi @rijobro,
Just wanted to give this discussion a bump to see if there might be a fix for the problem above?
Thanks!

[rijobro]

Hi, sorry you're still having the same problem. In answer to your previous question, the input to the GradCAM expects the input image to have the shape: B, C, H, W, [D]. So your dummy data should probably have the shape [1, 1, 193, 229, 193], such that it has B=1, C=1, H=193, W=229, D=193.