Run a 2D model on 3D inputs

[Eddowesselink]

Dear developers,

First of all, let me thank you for all the hard work on building MONAI. We have already succesfully used Monai (and NIftynet in the past) for 3D segmentation tasks for research purposes.

At this moment, we would like to train a 2D model (x,y,1) on 3D inputs (pix dim = 0.39x0.39x5, FOV = 512,512,50). In Niftynet we could (DenseVnet architecture) set the zth dimension to 1 (x,y,1), but it didn't work in Monai. Is there also an option in Monai to train a 2D model on 3D inputs using an UNET architecture?

Kind regards,
Eddo Wesselink

[Nic-Ma]

Hi @Eddowesselink ,

Thanks for your interest here.
I think maybe you can refer to this tutorial:
https://github.com/Project-MONAI/tutorials/blob/master/modules/2d_inference_3d_volume.ipynb

Thanks.

[wyli]

the sliding window component is recently merged into the core codebase

MONAI/monai/inferers/inferer.py

Lines 231 to 234 in 4b66746

	class SliceInferer(SlidingWindowInferer):
	"""
	SliceInferer extends SlidingWindowInferer to provide slice-by-slice (2D) inference
	when provided a 3D volume.

[umgpy]

Hi @Nic-Ma ,

I have been closely following the 2D train on 3D data discussion here and have even tried the new SliceInferer. My question is, is there a way to use slice inference for all three planes when the dimensions are not equal?,

My data shape is [256, 512, 512], instead of training and inferring on one axis like #808. I would like to train and infer on all planes.

Since my model would require a consistent spatial size, say [256,256]. How may I infer using SliceInferer.

[wyli]

in general, set the spatial_size to (H, W, 1) for the random cropping transforms https://docs.monai.io/en/stable/transforms.html#crop-and-pad can generate one or multiple (H, W, 1)-shape samples from each volume.

[Eddowesselink]

Hi @wyli, and @Nic-Ma

Forgive me for the questions before, they're solved. Hopefully I may ask you one last question with regard to SliceInferer. I am able to train a 2D UNET over 3D subvolumes after using the Squeezedimd transform with hxwx1 spatial window size :

  train_transforms.append(SqueezeDimd(
          keys=['image', 'label'], dim=-1))

However, I don't get satisfying results using the SliceInferer (DICE = 0.16). The loss is decreasing as expected, somewhat similar to my 3D model; with 3D volumes I got average DICE 0.93 for validation.

Could you help me out what I possibly did wrong with the SliceInferer? Happy to share the entire code if necessary.

Spatial window size for training = (480,480,1)

Code for validation:

        def validation(epoch_iterator_val):
            net.eval()
            dice_vals = list()
            with torch.no_grad():
                for batch in epoch_iterator_val:
                    val_inputs, val_labels = (batch["image"].cuda(), batch["label"].cuda())
                    axial_inferer = SliceInferer(roi_size=(480,480), sw_batch_size=5, cval=-1)
                    val_output = axial_inferer(val_inputs, net)
                    
                    val_output = [post_pred(i) for i in decollate_batch(val_output)]
                    val_labels_convert = [post_label(i) for i in decollate_batch(val_labels)]
                  
                    dice_metric(y_pred=val_output, y=val_labels_convert)
                    dice_metric_batch(y_pred=val_output, y=val_labels_convert)
                    
                    dice = dice_metric.aggregate().item()
                    dice_vals.append(dice)
        
                    metric_batch = dice_metric_batch.aggregate()
                    metric_label1 = metric_batch[1].item()
                    metric_values_label1.append(metric_label1)
                    metric_label2 = metric_batch[2].item()
                    metric_values_label2.append(metric_label2)
                    metric_label3 = metric_batch[3].item()
                    metric_values_label3.append(metric_label3)
                    metric_label4 = metric_batch[4].item()
                    metric_values_label4.append(metric_label4)
                    metric_label5 = metric_batch[5].item()
                    metric_values_label5.append(metric_label5)
                    metric_label6 = metric_batch[6].item()
                    metric_values_label6.append(metric_label6)
                    
                    epoch_iterator_val.set_description(
                        "Validate (%d / %d Steps) (dice=%2.5f)" % (global_step, 10.0, dice)
                    )
                dice_metric.reset()
                dice_metric_batch.reset()
            mean_dice_val = np.mean(dice_vals)

return mean_dice_val

Kind regards,
Eddo

[Eddowesselink]

Hi @wyli and @Nic-Ma ,

Thank you very much implementing this sliding inferer into the core codebase. This works very well.

Kind regards,
Eddo