Calculating DCS and IoU of different channels of a segmentation at once

[Kiarashdnsh]

Hello,

I'm a novice in AI and image segmentation.
I have a dataset of [256, 256, 256]-sized images and labels. The input channel is 1 and the output channel is 3 (label IDs are 0, 1, and 2, respectively). I've segmented the images using UNet within the MONAI framework. However, I want to check the segmentation DSC and IoU score of each defined channel separately.

Previousely, I'd used this code in for this task:

if len(devices) > 1:
    model = torch.nn.DataParallel(model, device_ids=devices)

post_label = Compose([AsDiscrete(to_onehot=3)])
model.eval()
with torch.no_grad():
    for val_data in val_loader:
        val_inputs, val_labels = val_data["image"].to(device), val_data["label"].to(device)
        # define sliding window size and batch size for windows inference
        roi_size = (128, 128, 128)
        sw_batch_size = 4
        val_outputs = sliding_window_inference(val_inputs, roi_size, sw_batch_size, model)
        val_outputs = [post_pred(i) for i in decollate_batch(val_outputs)]
        val_labels = [post_label(i) for i in decollate_batch(val_labels)]
        # compute metric for current iteration
        dice_metric(y_pred=val_outputs, y=val_labels)
        for val_output_dice in val_outputs:
            saver(val_output_dice)
            print("evaluation dice metric:", dice_metric.aggregate().item())
            #dice_metric.reset()
        IoU_metric(y_pred=val_outputs, y=val_labels)
        for val_output_IoU in val_outputs:
            #saver(val_output_IoU)
            print("evaluation IoU metric:", IoU_metric.aggregate().item())
            #IoU_metric.reset()
    # reset the status
    dice_metric.reset()
    IoU_metric.reset()

which returns one value for DSC and one value for IoU for the whole segmented image. However, as I mentioned above, I want four values for each segmentation. One DSC and one IoU score for segmented channel no.2 and one DSC and one IoU score for segmented channel no.3 for each segmented image (I don't want the background channel no.0). How do I do that?

Thanks in advance,
Kiarash

[wyli]

have you tried changing the include_background and reduction parameters of the metrics?

MONAI/monai/metrics/meandice.py

Lines 40 to 44 in 77b1759

	include_background: whether to include Dice computation on the first channel of
	the predicted output. Defaults to ``True``.
	reduction: define mode of reduction to the metrics, will only apply reduction on `not-nan` values,
	available reduction modes: {``"none"``, ``"mean"``, ``"sum"``, ``"mean_batch"``, ``"sum_batch"``,
	``"mean_channel"``, ``"sum_channel"``}, default to ``"mean"``. if "none", will not do reduction.

[Kiarashdnsh]

Hello Wyli,
thanks for your reply.
I've tried that. It evaluates the first segmentation with a mean value and for the rest, renders an error:

2023-10-23 11:17:00,611 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30455_AV45_S_E/30455_AV45_S_E_seg.nii
evaluation dice metric: 0.7969357967376709
evaluation IoU metric: 0.6898679733276367
2023-10-23 11:17:03,554 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30458_AV451_S_E/30458_AV451_S_E_seg.nii

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

[<ipython-input-25-d08ce1cd708b>](https://localhost:8080/#) in <cell line: 6>()
     18         for val_output_dice in val_outputs:
     19             saver(val_output_dice)
---> 20             print("evaluation dice metric:", dice_metric.aggregate().item())
     21             #dice_metric.reset()
     22         IoU_metric(y_pred=val_outputs, y=val_labels)

RuntimeError: a Tensor with 2 elements cannot be converted to Scalar

Seems something is wrong with the aggregate().item() part. I've used mean_channel as the reduction mode.

[wyli]

This runtime error seems to be triggered by the .item() call, please remove that call..

[Kiarashdnsh]

I removed the .item() call. This is the output of 5 segmented images:

2023-10-23 17:09:01,729 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30455_AV45_S_E/30455_AV45_S_E_seg.nii
evaluation dice metric: tensor([0.6994], device='cuda:0')
evaluation IoU metric: tensor([0.5428], device='cuda:0')
2023-10-23 17:09:04,613 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30458_AV451_S_E/30458_AV451_S_E_seg.nii
evaluation dice metric: tensor([0.6994, 0.7015], device='cuda:0')
evaluation IoU metric: tensor([0.5428, 0.5455], device='cuda:0')
2023-10-23 17:09:07,444 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30458_AV452_S_E/30458_AV452_S_E_seg.nii
evaluation dice metric: tensor([0.6994, 0.7015, 0.6948], device='cuda:0')
evaluation IoU metric: tensor([0.5428, 0.5455, 0.5388], device='cuda:0')
2023-10-23 17:09:10,317 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30462_AV451_S_E/30462_AV451_S_E_seg.nii
evaluation dice metric: tensor([0.6994, 0.7015, 0.6948, 0.6987], device='cuda:0')
evaluation IoU metric: tensor([0.5428, 0.5455, 0.5388, 0.5410], device='cuda:0')
2023-10-23 17:09:13,665 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30462_AV452_S_E/30462_AV452_S_E_seg.nii
evaluation dice metric: tensor([0.6994, 0.7015, 0.6948, 0.6987, 0.7006], device='cuda:0')
evaluation IoU metric: tensor([0.5428, 0.5455, 0.5388, 0.5410, 0.5433], device='cuda:0')

One value for each metric for each segmented image. However, the values for the previous images are also printed for each later image.

[Kiarashdnsh]

I found this configuration that is working:

for val_output_dice in val_outputs:
            saver(val_output_dice)
            print("evaluation dice metric:", dice_metric(y_pred=val_outputs, y=val_labels))
            #dice_metric.reset()
        IoU_metric(y_pred=val_outputs, y=val_labels)
        for val_output_IoU in val_outputs:
            #saver(val_output_IoU)
            print("evaluation IoU metric:", IoU_metric(y_pred=val_outputs, y=val_labels))
            #IoU_metric.reset()

The output is:

2023-10-23 17:48:52,650 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30455_AV45_S_E/30455_AV45_S_E_seg.nii
evaluation dice metric: metatensor([[0.9920, 0.6254, 0.7734]], device='cuda:0')
evaluation IoU metric: metatensor([[0.9841, 0.4550, 0.6306]], device='cuda:0')
2023-10-23 17:48:55,606 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30458_AV451_S_E/30458_AV451_S_E_seg.nii
evaluation dice metric: metatensor([[0.9917, 0.6262, 0.7768]], device='cuda:0')
evaluation IoU metric: metatensor([[0.9835, 0.4558, 0.6351]], device='cuda:0')
2023-10-23 17:48:58,497 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30458_AV452_S_E/30458_AV452_S_E_seg.nii
evaluation dice metric: metatensor([[0.9913, 0.6103, 0.7793]], device='cuda:0')
evaluation IoU metric: metatensor([[0.9827, 0.4391, 0.6384]], device='cuda:0')
2023-10-23 17:49:01,459 INFO image_writer.py:197 - writing: /content/drive/MyDrive/UNet/Validation/30462_AV451_S_E/30462_AV451_S_E_seg.nii
evaluation dice metric: metatensor([[0.9910, 0.6314, 0.7660]], device='cuda:0')
evaluation IoU metric: metatensor([[0.9821, 0.4614, 0.6207]], device='cuda:0')

which is what I was aiming for.
Thanks for your time and help @wyli. Much appreciate it,