Brats-20 dataset labels visualization on 3D images

[faizan1234567]

Hi, I am working on the Brats-20 dataset. I want to create a gif animation of labels on images and prediction results on images. I am trying to do that. However, I am confused about the plots I get from the visualization.

I use the Keras function 'to_categorical' to convert labels to one hot representation from (240, 240, 155) shape as in the following code.

`def categorical(image, mask, is_categorical=False):
if not is_categorical:
label = to_categorical(mask, num_classes = 4).astype(np.uint8)

image = cv2.normalize(image[:, :, :, 0], None, alpha=0, beta=255,
                        norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F).astype(np.uint8)
                        
labeled_image = np.zeros_like(label[:, :, :, 1:])
labeled_image[:, :, :, 0] = image * (label[:, :, :, 0])
labeled_image[:, :, :, 1] = image * (label[:, :, :, 0])
labeled_image[:, :, :, 2] = image * (label[:, :, :, 0])

# color labels
labeled_image += label[:, :, :, 1:] * 255

return labeled_image`

Here the image has the shape of (240, 240, 155, 4) corresponding to a patient scan, and the mask has the shape of (240, 240, 155).
I want to create a nice visualization like that:

And a gif file to show labels or prediction results on an image. But the _seg.nii.gz (label file) contains the number 4 in the tensor. Which gives me an error in to_categorical. Because I am using num_classes =4 in the above function as you can see. Then I normalized the labels which converted labels for int to float values, and it was wrong.

Then I wrote something like limiting pixel labels from 1 to 3 to remove the error. Which successfully removed the error, however, I don't know if it is correct to label colors for the visualization. OR has it been mapped accurately or not? I need guidance, please.

I wrote the following code to map labels from 1 to 3 (I don't know if it's correct or not), but it removed the error in the to_categorical function. The code:

`def normalize(mask: np.ndarray):

# data_min = np.min(data)
# return (data - data_min) / (np.max(data) - data_m
mask[mask == 1] = 1
mask[mask == 2] = 2
# mask[mask == 3] = 0
mask[mask == 4] = 3
return mask`

Help will be appreciated as I am stuck here.

[KumoLiu]

Hi @faizan1234567, if you have the number 4 in the tensor, then maybe your num_classes in the one-hot function should be 5. You can use AsDiscrete in MONAI. Just a simple example for your reference.

MONAI/monai/transforms/post/array.py

Line 131 in 78d4f42

class AsDiscrete(Transform):

import torch
from monai.transforms import AsDiscrete

data = torch.tensor([[[0, 1], [1, 1], [3, 4]]])
data_onehot = AsDiscrete(to_onehot=5)(data)

Hope it can help you, thanks!

[faizan1234567]

I want to create a visualization of tumors on the scan. The scan has the shape:(4, 240, 240, 155), and the shape of mask and prediction is: (3, 240, 240, 155), how can I use this to plot abnormalities on the scan, is there any way or example? Could you please help? Also, I want to create a gif from this. Like labels is being plotted on the input or the case

[KumoLiu]

Hi @faizan1234567, you can get some visualization examples from here.
https://github.com/Project-MONAI/tutorials/blob/main/modules/transform_visualization.ipynb
Thanks!

[faizan1234567]

I want to visualize something like this.