Guidance Needed: Combining Multiple NIfTI Files for inference in Brats_MRI_Segmentation Model

[hoedha12]

Hi,

First of all, thank you for this wonderful project! I'm a neuroradiologist and I'm trying to learn more about using this project for my research needs. I want to use the Brats MRI segmentation model to perform inference/segmentation on my own set of data, which consists of skull-stripped and coregistered T1, T2, FLAIR, and T1ce images (just like the Brats training and validation data).

My question is this: in 3D-Slicer, there's only one input for "Source volume", yet I have four NIfTI files. If I place these four files in the dataset folders, they are treated as different samples. What is the best approach for this? Should I combine these four NIfTI files, and if so, how do I do that? Additionally, are there any other pre-processing steps required for external data to be used in the model inference? I don't know much about coding, but I'm willing to learn. However, please bear with me, and try to explain in a step-by-step manner or refer me to easy-to-understand references.

Thank you in advance.

[diazandr3s]

Hi @hoedha12,

Thanks for opening this discussion.

Are you using MONAI Label for this? In any case, I won't suggest merging all modalities into a single file. We have discussed this before with other users:

• RuntimeError when "brats_mri_segmentation_v0.2.1" from monaibundle is used. MONAILabel#1051 (comment)
• Wrong Orientation/Origin for multichannel volumes (4D) using Slicer - Issue visualizing BRATS images on 3DSlicer MONAILabel#22 (comment)

Instead, I'd suggest placing one of the modalities in a folder (i.e. FLAIR), and the other three in a different one. Something like this:

/tmp/BRATS/

        ├──---- FLAIR

        │       ├── BRATS-001.nrrd
      
        │       ├── BRATS-002.nrrd
      
        │       ├── BRATS-003.nrrd
      
        │       └── BRATS-004.nrrd
      
        ├── BRATS-001-T1C.nrrd
      
        ├── BRATS-001-T1.nrrd
      
        ├── BRATS-001-T2.nrrd
      
        ├── BRATS-002-T1C.nrrd
      
        ├── BRATS-002-T1.nrrd
      
        ├── BRATS-002-T2.nrrd
      
        ├── BRATS-003-T1C.nrrd
      
        ├── BRATS-003-T1.nrrd
      
        └── BRATS-003-T2.nrrd

Then you can start the MONAI Label server pointing to the one-modality folder and modify the LoadImage transform to load the other 3 when you want to train or do inference.

Hope this makes sense. Let me know. Happy to help further.

[hoedha12]

Yes, I'm using MonaiLabel with 3D Slicer. I plan to use the brats_mri_segmentation bundle with its pre-trained model to make inferences (segmentation) on my images. So, I suppose I just need to modify the inference.json file? Could you please tell me what I should modify in this file?

In addition, if I only load the FLAIR image, will the model still consider information from the other three images? And will the model segment all possible areas (whole tumor, tumor core, enhancing tumor)? I'm drawing from my daily practice where I need to review all images (T1, T2, FLAIR, and T1ce) in order to confidently determine which area is enhancing, necrotic, or affected by edema. I apologize if I was not clear or if I misunderstood your explanation.

Thank you once again.

[diazandr3s]

Hi @hoedha12,

Yes, I'm using MonaiLabel with 3D Slicer. I plan to use the [brats_mri_segmentation bundle](https://github.com/Project-MONAI/model-zoo/tree/dev/models/brats_mri_segmentation) with its pre-trained model to make inferences (segmentation) on my images.

Superb!

So, I suppose I just need to modify the inference.json file? Could you please tell me what I should modify in this file?

If you want to run inference only, yes.
Before, you need to create a new transform for this. It should be something like this:

class LoadOtherModalitiesd(MapTransform):
    """
    Merge all modalities into one

    "0": "FLAIR",
    "1": "T1",
    "2": "T1C",
    "3": "T2"

    """

    def __init__(self, keys: KeysCollection,  target_spacing, allow_missing_keys: bool = False):
        super().__init__(keys, allow_missing_keys)
        self.target_spacing = target_spacing

    def __call__(self, data):
        d: Dict = dict(data)
        input_path = '/PATH-TO-THE-OTHER-MODALITIES/'
        for key in self.key_iterator(d):
            img = copy.deepcopy(d[key])
            name = img.meta['filename_or_obj'].split('/')[-1].split('.')[0]
            all_mods = np.zeros(
                (4, img.array.shape[-3], img.array.shape[-2], img.array.shape[-1]), dtype=np.float32
            )
            all_mods[0, ...] = img.array[0, ...]
            logging.info(f'Size of the images in transform: {all_mods.shape}')
            loader = LoadImage(reader='ITKReader')
            chan_first = EnsureChannelFirst()
            spacer = Spacing(pixdim=self.target_spacing, mode="bilinear")
            res = Resize(spatial_size=(img.array.shape[-3], img.array.shape[-2], img.array.shape[-1]))
            for idx, mod in enumerate(['-T1.nrrd', '-T1C.nrrd', '-T2.nrrd']):
                aux = loader(input_path + name + mod)
                logging.info(f'Modality: {mod} is being read')
                aux = chan_first(aux[0])
                spaced_img = spacer(aux)
                logging.info(f'Modality: {mod} is being spaced')
                # This solves the issue of images having slightly different resolution 
                resized_img = res(spaced_img)
                all_mods[idx+1, ...] = resized_img[0].array
            d['spatial_size'] = all_mods[0, ...]
            d[key].array = all_mods
        return d

Then you add the new transform here to load the other modalities.

If you think it helps, happy to join a video call and further discuss this.

[Beken22]

Hello,
I am a novice in medical image processing at these levels, and I am also trying to learn. I think that my case is similar to hoedha12's.

I have 250 MR brain images of diffuse gliomas with T1, T1c, FLAIR, T2 and DWI sequences. I need to obtain the mask image corresponding to the tumor, for each of them. The images are in DICOM format, and I have converted them to nifty.

From your comments I understand that I can apply inference with the brats mri segmentation model for obtaining the segmented images of my images.

If so, I would like to understand what you have indicated needs to be done to accomplish this. That is, the structure of directories with the images, what is the transformation you mention, where to place the LoadOtherModalitiesd class, etc., and everything I need to apply inference of brats mri segmentation model on my images.

Please, could you help me to move forward and achieve my goal?

Thank you in advance.

[diazandr3s]

Hi @Beken22,

The first thing before running inference is to make sure the sequences (T1, T1c, FLAIR, T2 and DWI) are aligned, fused OR co-registered - Multiple terms for the same thing.

For this step, you may find this useful: https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FLIRT

The second step is to organize the aligned sequences using this structure: #6796 (comment)

Then you can add the customised transform (LoadOtherModalitiesd) in the infer.py file to run inference on the aligned sequences.

Once you manage to align the sequences, let me know and I can further explain the next steps.

Hope this helps,

[Beken22]

Hi, thank you very much.
I will follow those steps, and hope to get back to you as soon as possible.