Fine-tuning models with nnUNet v2 using multimodal data

[SergioRodLla]

Hello,

Thank you very much for the repository and the provided pretrained models. I have seen that you added some scripts for nnUNet v2 to run fine-tuning as well as the STUNetTrainer.
I have a dataset with CT and PET modalities and I was wondering if it is possible to use nnUNet v2 to run fine-tuning using a pretrained model that you provide (e.g. huge_ep4k.model) to do this combining both modalities (concatenating them as input as you showed for the AutoPET challenge).

Best regards,
Sergio

[Ziyan-Huang]

Hi Sergio,

Fine-tuning on multi-modal datasets like AutoPET (CT and PET concatenated) is supported. We copy the first convolutional layers (pretrained on a single CT modality) from the pretrained model to support multi-modal inputs.

Best regards,
Ziyan