proper Activation Function and Loss Function

[Harris91-lee]

Hello there,
I'm using SegresNet model in MONAI framwork for multi-label 3D segmentation task and the results were great so far. However, i wonder about the activation function. Since i have multi-label segementation should i change the activation function to softmax? Originally it's set to sigmoid.
post_trans originally:
post_trans = Compose([EnsureType(), Activations(sigmoid=True), AsDiscrete(argmax=True, to_onehot=NUM_OF_LABELS)])
is that correct?
Modify:
post_trans = Compose([EnsureType(), Activations(softmax=True), AsDiscrete(argmax=True, to_onehot=NUM_OF_LABELS)])

---

loss function originally:
loss_function = DiceLoss(smooth_nr=0, smooth_dr=1e-5, squared_pred=True, to_onehot_y=False, sigmoid=True)
Modify:
　loss_function = DiceLoss(smooth_nr=0, smooth_dr=1e-5, squared_pred=True, to_onehot_y=True, softmax=True)

---

i want to say here that even in multi-label segementation using sigmoid didnt effect the results, is that normal? or we have to use softmax for multi-label segmentation task?

[KumoLiu]

Hi @Harris91-lee,
Multilabel assumed that classes are not mutually exclusive and each class have its own channel, so use Sigmoid=True is correct.

[KumoLiu]

Hi @sh3rlock14 ,

1. For brain tumor segmentation , you can see that classes are not mutually exclusive, you should use Sigmoid instead of Softmax.
2. Before the loss function, you don't need to use Activations.

[sh3rlock14]

1. For [brain tumor segmentation ](https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/brats_segmentation_3d.ipynb), you can see that classes are not mutually exclusive, you should use `Sigmoid` instead of `Softmax`.

I was working on this tutorial but honestly I can't figure it out how in this case the classes are not mutually exclusive.
I have a clear understanding of 'multi-label' definition in the general case of classification, e.g. in Scene Understanding, but for this kind of Segmentation I'am confused: in each image (let's consider a single volume slice) shouldn't a pixel have only one class? I read online that one should pick sigmoid in the case of overlapping regions (as in this case), but none of these comments were proper explanation of the reasoning behind this choice.

I'am still sticking to the fact that we should produce a valid probability distribution over the labels, instead of computing the sigmoid per each one... could you please help me to see why that's not the right way to think about that?

2. Before the loss function, you don't need to use `Activations`.

Yeah, sorry: I didn't mean the loss, but actually the metric computation as done in the validation step: in the various tutorials I saw, I noticed that the Activations are only used directly on the single input (once the batch has been decollated) rather on the entire batch: do you think letting us pass the dimension onto which compute the activation could be more easy?

Thanks in advance for your help!

[KumoLiu]

They are not mutually exclusive since each pixel may not point to only one class. You can see ConvertToMultiChannelBasedOnBratsClassesd for how they get the label.

[sh3rlock14]

Thanks for your feedback:
I actually wrote my version of ConvertToMultiChannelBasedOnBratsClassesd since I noticed some discrepancies in the one implemented in the tutorial, and the one in the docs, in particular:

Tutorial:

• label 1 is the peritumoral edema
• label 2 is the GD-enhancing tumor
• label 3 is the necrotic and non-enhancing tumor core

Library:

• label 1 is the necrotic and non-enhancing tumor core
• label 2 is the peritumoral edema
• label 4 is the GD-enhancing tumor

while after downloading the Task01_BrainTumour dataset, I found out that the labels were:

• label 0 is the background
• label 1 is the peritumoral edema
• label 2 is the non-enhancing tumour
• label 3 is the enhancing tumour

So, my version is:

class ToMultiChannelBratsClassesd(MapTransform):
    """
    Convert labels to multi channels based on brats classes
    label 1 is the peritumoral edema
    label 2 is the GD-enhancing tumor
    label 3 is the necrotic and non-enhancing tumor core

    The classes are: BG (background), TC (Tumore core), WT (Whole tumor), ET (Enhancing tumor)
    """
    def __call__(self, data):
        d = dict(data)
        for key in self.keys:
            # merge label 2 and 3 to consrtuct TC
            # merge label 1, 2 and 3 to construct WT
            # label 2 is ET
            result = [ (d[key]==0), (d[key]==2) | (d[key]==3), (d[key]==1) | (d[key]==2) | (d[key]==3), d[key]==2 ]
            d[key] = stack(result, axis=0).float()
        return d

and I do not perform any kind of target inversion while computing the loss: given an input of size BCHWD, the model outputs a tensor BNHWD, where N=4 (one channel for each class).

Then I compute:

criterion = DiceLoss(include_background=True,
                         to_onehot_y=False,
                         sigmoid=False,
                         softmax=True,
                         ...
                         )

dice_loss = criterion(output, target)

So, if I understand correctly your prompt, one should use sigmoid=True since the classes (TC, WT, ET) defined here are actually a merging of multiple labels?

For my own purposes I think I'll try both configurations, but do you think that going with the softmax is conceptually wrong?

[KumoLiu]

Yes, if softmax is used, then the sum of the probabilities of each label will become 1, which is not correct.