3D feature extractor

[joaolcguerreiro]

Hi!

I just found out that MONAI has pre-trained 3d models. I was wondering if any of them can be used for feature extraction in order to compute a perceptual loss...

Is this possible with any of the provided networks? If it is possible do we need to normalize the input with some dataset statistics?

Thank you :)

[KumoLiu]

Hi @joaolcguerreiro, IMO, CNN, RNN, and Transformer can all be considered feature extractors for extracting data features.
Thanks!

[joaolcguerreiro]

I am looking for pre-trained models that handle 3D data. MONAI nets don't seem to support this right now. Do you know any alternative?

[Wangrui-berry]

model = monai.networks.nets.resnet10(spatial_dims=3, n_input_channels=1, n_classes=1)
What's the meaning? Can it solve your problem?

[Wangrui-berry]

https://docs.monai.io/en/stable/_modules/monai/networks/nets/resnet.html
conv_type: Callable = Conv[Conv.CONV, spatial_dims]
conv_type: type[nn.Conv1d | nn.Conv2d | nn.Conv3d] = Conv[Conv.CONV, spatial_dims]
It seems could solve your problem. I am also trying to use 3d network to process 3d data using Monai.

[KumoLiu]

Hi @joaolcguerreiro, since most pre-trained weights are based on the ImageNet which is a 2d dataset.

MONAI/monai/networks/blocks/fcn.py

Line 112 in a89a0af

pretrained: If True, returns a model pre-trained on ImageNet

You can try to use the pre-trained weights introduced in https://monai.io/model-zoo.html.

Thanks!