Is 'ignore_index=0' necessary for binary classification?

[dakomura]

I would like to use the IoU metric for the binary segmentation task (background or target).
The prediction matrix has only one channel for the target class (not 2 channels for background and target class).

The current code is as follows.

self.train_metric = torchmetrics.IoU(
    num_classes=2,
    dist_sync_on_step=True)

Do I need to add 'ignore_index=0' when I'm only interested in the target class?

[SkafteNicki]

Hi @dakomura,
Yes, you will need to set ignore_index=0 in your usecase

[dakomura]

Thank you!!

[dakomura]

Hi @SkafteNicki

I added ignore_index=0 but the behavior was strange.

So I checked the value of confmat in the code (_iou_from_confmat) and found that the value was like this.

tensor([[1.9591e+05, 7.1420e+06],
        [1.8340e+03, 3.3070e+04]], device='cuda:0')

I think the IoU for the target class can be calculated as follows:

confmat[1,1] / (confmat[0,1] + confmat[1,0] + confmat[1,1]) = 0.0046

When I added ignore_index=0, confmat became

tensor([[0, 0],
        [1.8340e+03, 3.3070e+04]], device='cuda:0')

, where the essential information (confmat[0,1]) was lost.

When I didn't add ignore_index=0, score was

[0.0267, 0.0046]

, where the 2nd component is the IoU for the target class.

So I think the following code should be removed in my case.

    # Remove the ignored class index from the scores.
    if ignore_index is not None and 0 <= ignore_index < num_classes:
        confmat[ignore_index] = 0.0

Am I right?