What neural network should I choose for voxel segmentation

[relyativist]

Hi,
I have few volume feature maps extracted from brain images and labels (green contours on image) in link. I wan't to classify voxels that are in labels and out of labels and create probabilistic map. What NN model should I use for this?

[Nic-Ma]

Hi @ericspod ,

Could you please help share some comments about this question? I think you have many best practices.

Thanks in advance.

[ericspod]

I'm not sure there's anything specific to this application that would favour any network architecture over UNet. Our implementation can handle 3D as well as 2D as demonstrated in our various notebooks on the subject. The UNetR implementation would be a good one to try if UNet isn't giving you good enough results.

[relyativist]

@ericspod @Nic-Ma I followed brats lesion segmentation, and tried to create segmentation model for 4 channel tensors and segmentation tensors:

following the example, I used SegResNet and Unet, with no luck: It seems that network is not learning, because of high train losses:

epoch 1/10
1/37, train_loss: 0.9997, step time: 218.6272
2/37, train_loss: 0.9991, step time: 35.3239
3/37, train_loss: 0.9994, step time: 27.2364
4/37, train_loss: 0.9986, step time: 35.2655
5/37, train_loss: 0.9991, step time: 24.8010
6/37, train_loss: 0.9996, step time: 36.2708

I found that modified unet can do the work, is there architecture implementation if this in Monai?

[ericspod]

We don't have an implementation of the modified UNet yet. Our default UNet implementation uses residual blocks like the modified UNet example, it could be modified to use SE blocks instead by inheriting and overriding methods to use the SE blocks MONAI already defines.

[relyativist]

@ericspod, @Nic-Ma Thanks for recommendation. I gave an attempt to use just dense layer to classify only features maps without T1 images. But with no luck, Will try with Unet.

[relyativist]

Is there any tutorials how input dataset dict should be created? I can't get the concept of how actually channels of image should be stacked. From unet segmentation tutorial notebook:

# load 4 Nifti images and stack them together
LoadImaged(keys=["image", "label"])

Where actually all the "magic" happens? For my example I wan't to concatenate T1 and feature maps extracted manualy, how it should be done?

[ericspod]

In this example only one nifti file is actually being loaded in fact. The list of image/label filename pairs is stored in the dataset.json file which comes with each task tarball for the Decathlon challenge, the DecathlonDataset object loads these files and constructs the appropriate internal list. What's happening with the transform sequence is ConvertToMultiChannelBasedOnBratsClassesd is converting the single channel label into a multichannel label for input to the network. @yiheng-wang-nv @wyli correct me if I'm wrong describing this process.

[Nic-Ma]

Hi @relyativist ,

Here is the stack logic when reading a list of images:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/data/image_reader.py#L128

Thanks.