Inference time and prediction issue.

[lenlobo]

I am working with 3d segmentation using Monai and Nifti files as inputs and using a model which has 6 input channels, I get an error in the inference testing of the model.

class Net(pytorch_lightning.LightningModule):
    def __init__(self):
        super().__init__()
        self._model = UNet(
            dimensions=3,
            in_channels=6,
            out_channels=2,
            channels=(16, 32, 64, 128, 256),
            strides=(2, 2, 2, 2),
            num_res_units=2,
            norm=Norm.BATCH,
        )

So after using the tutorial code from "https://github.com/Project-MONAI/tutorials/blob/72728b678947bc21443d02d3a19d29f01c9eed01/3d_segmentation/spleen_segmentation_3d.ipynb", as a reference.

I have two methods for running the inference.
1st method:
create 6 image variables of the inputs like,
img1 = path/to/img/nifti
img2 = path/to/img/nifti
.
.
img6 = path/to/img/nifti

and then during the inference, I use torch.cat to concat all of them together.
eg. sliding_window_inference(torch.cat(img1,img2,..img6)

Now this, gives me the correct output and it is processed withing 1-3mins, which is ideal. But I cannot use post tranforms on it, I made a old discussion on this link "#5335". That issue is fixed. But now while following the new method, of, creaing an image variable with all 6 paths together as per the above link

Datalist like: 
[
{'image': [input_1, input_2, ....]
'label': label},

My number of predictions are equal, the start values of black pixels are predicted correctly, but the actual prediction that I require, the values are wrong and the time taken for the whole process is about 30mins. Which is not at all ideal for me.

To check the pred values are correct or not, I found the max value of prediction in my old method to be something 30.xx, but the new method the max value is around 4.xx

Any reason, why this method takes more time, but also predicts wrongly.

[Joeycho]

Hi, could you share the error message which you get? Or is this related with computing time? (30 mins > 1-3 mins)?

If it is related with computing time, could you make a run with only one image channel and how long it will take?

6 channels, it seems quite demanding work for me, of course, it depends on how big your images are.

[lenlobo]

Hey @Joeycho ,
It's my work code that's the reason I can't really give you much details, unfortunately. I've been given the data the model is already trained on, so I don't need to normalise it.
That's why, the issue being, the model works properly in one method, but the other method (which would be our preferred method) doesn't.

[Joeycho]

Do you want to apply the exact same post-processing or any transforms to all modalities?

Then why not..

From : Datalist like:
[
{'image': [input_1, input_2, ....]
'label': label},
]
To:
[
{'image': torch.cat(img1,img2,..img6)
'label': label},
]

I didn't realize that list for images can work for monai transforms. It works for you, right? just slow..

[lenlobo]

Hey @Joeycho ,

Seems like an approach I didn't try, will give it a look. Yeah, it works slow and for some reason wrong predictions as well, which is weird. Thanks, I'll give the concatenation of images earlier a try.

[lenlobo]

Hey @Joeycho,

I hope you had a good time the past few weeks! Sorry for the late reply.
Anyhow, I was giving the above solution a try but then I figured it wouldn't be possible to do so as img1,img2.. the 6 images are tensors and not paths to the images which are read later by the loaders. So this approach won't work, or am I missing something here?

[Joeycho]

Hi, @lenlobo,

Quite busy to finalize my thesis, but otherwise, yes, I had good time. Thanks for asking.
img1, img2, img3... I meant they are already tensors.
Try it as the below, maybe there was an error by putting tensor at the dimension position.

The below code means, concatenate img1(1, 215,215,215)img2(1, 215,215,215) img3(1, 215,215,215) in 0 dimension. then,
the output will be (3, 215,215,215).

torch.cat((img1, img2, img3), 0)