How to use different weights in focalloss for binary classification tasks

[Minxiangliu]

I am working on a Binary Classification model. The weight parameter in focalloss can be adjusted according to requirements, but the Docs description that if the sequence must be of a classification number, how do I modify it to apply it to Binary Classification?

from monai.losses import FocalLoss
FocalLoss(weight=[....])

[Nic-Ma]

Hi @yiheng-wang-nv ,

Could you please help share some comments here?

Thanks in advance.

[yiheng-wang-nv]

Hi @Minxiangliu , for a binary classification task, the length of weight should be 2, like weight=(0.5, 1).
Correct me if I misunderstand your question, thanks.

[Minxiangliu]

I have tried writing like this, but he got the following error:

File "/root/miniconda3/envs/lightning/lib/python3.7/site-packages/monai/losses/focal_loss.py", line 162, in forward
    "the length of the weight sequence should be the same as the number of classes. "
ValueError: the length of the weight sequence should be the same as the number of classes. If `include_background=False`, the number should not include class 0.

My model output is 1, I guess it may be where the focalloss is wrong,
If the weight in focalloss is currently only designed for multi-classification tasks, I may need to change the loss function.

[yiheng-wang-nv]

Hi @Minxiangliu , if your output has the shape B, 1, your weight should be a single value or a sequence with length 1, and this value will be applied to the class that you predicted to.

FYI, I wrote a simple function based on the formula, and it can help to validate:

from monai.losses import FocalLoss
import torch

alpha = 2
gamma = 3
loss = FocalLoss(weight=alpha, gamma=gamma, include_background=True)

logits = torch.randn([1, 1])
label = torch.tensor([[0]])

print("focal loss from monai: ", loss(logits, label))


def calculate_focal_loss(logits, label, alpha=2, gamma=2):
    prob = logits.sigmoid()
    y = label.item()
    if y == 1:
        pt = prob
    else:
        pt = 1 - prob
    return - alpha * torch.pow(1-pt, gamma) * pt.log()

print("focal loss from formula: ", calculate_focal_loss(logits, label=label, alpha=alpha, gamma=gamma))

[Minxiangliu]

Hi @yiheng-wang-nv ,
I tried to add a sequence of length 1 and apply it to the Label, but there is an error, can you please help me ? Thanks!

from monai.losses import FocalLoss
import torch

loss = FocalLoss()

lable = torch.tensor([
    [1.],[0.],[0.],
    [0.],[1.],[0.],
    [0.],[0.],[1.]
])
logits  = torch.tensor([
    [0.1],[0.3],[0.8],
    [0.2],[0.8],[0.01],
    [0.02],[0.7],[0.4]
])

print(lable.shape)
print(logits.shape)

print(loss(logits ,lable))
loss.weight = (lable+1).view(1,-1)
print(loss(logits, lable))

output:

torch.Size([9, 1])
torch.Size([9, 1])
tensor(0.2436)
Traceback (most recent call last):
  File "xxx.py", line 22, in <module>
    print(loss(logits, lable))
  File "D:\Anaconda3\envs\monai\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "D:\Anaconda3\envs\monai\lib\site-packages\monai\losses\focal_loss.py", line 172, in forward
    at = at.expand((t.size(0), -1, t.size(2)))  # 1,N,1 => B,N,H*W
RuntimeError: expand(torch.FloatTensor{[1, 1, 1, 9]}, size=[9, -1, 1]): the number of sizes provided (3) must be greater or equal to the number of dimensions in the tensor (4)

[Minxiangliu]

Hi, @yiheng-wang-nv
Is there any update here please?

[yiheng-wang-nv]

Hi, your shape of input is [9, 1] thus your weight should be a single value, or a sequence of length 1.