Why is "micro" not in "allowed average" in AUROC?

[JoeNatan30]

Documentation at AUROC shows that "micro" can be used on average https://torchmetrics.readthedocs.io/en/stable/references/modules.html#f1

average:
- 'micro' computes metric globally. Only works for multilabel problems
- 'macro' computes metric for each class and uniformly averages them
- 'weighted' computes metric for each class and does a weighted-average,
where each class is weighted by their support (accounts for class imbalance)
- None computes and returns the metric per class

So, I initialized AUROC in my code in this way

self.val_auroc = torchmetrics.AUROC(num_classes=NUM_CLASSES, average="micro")

but I got this error message

ValueError: Argument average expected to be one of the following: ('None', 'macro', 'weighted') but got micro

When I check AUROC code, I notice that "micro" is not in "allowed_average" https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/classification/auroc.py

allowed_average = (None, "macro", "weighted", "micro")
if self.average not in allowed_average:
    raise ValueError(
        f"Argument `average` expected to be one of the following: {allowed_average} but got {average}"
    )

https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/functional/classification/auroc.py

 # calculate average
            if average == AverageMethod.NONE:
                return tensor(auc_scores)
            if average == AverageMethod.MACRO:
                return torch.mean(torch.stack(auc_scores))
            if average == AverageMethod.WEIGHTED:
                if mode == DataType.MULTILABEL:
                    support = torch.sum(target, dim=0)
                else:
                    support = torch.bincount(target.flatten(), minlength=num_classes)
                return torch.sum(torch.stack(auc_scores) * support / support.sum())

            allowed_average = (AverageMethod.NONE.value, AverageMethod.MACRO.value, AverageMethod.WEIGHTED.value)
            raise ValueError(
                f"Argument `average` expected to be one of the following:" f" {allowed_average} but got {average}"
            )

I am not sure if it is a documentation error or developed functionality missing from the AUROC code.

I use conda torchmetrics version (0.6.2)

[SkafteNicki]

I do not get the error you are mentioning when running locally.
Also, the code on master clearly allows micro average:
https://github.com/PyTorchLightning/metrics/blob/94dec52004b45523779f28f450673a07a5493ad0/torchmetrics/classification/auroc.py#L136-L140
Could you please installing from master:

pip install git+https://github.com/PytorchLightning/metrics.git@master

[JoeNatan30]

I followed your suggestion and got this error.

File "/home/joe/project/PeruvianSignLanguage/3.Classification/ChaLearn-2021-LAP/models/module.py", line 190, in validation_epoch_end
val_auroc = self.val_auroc.compute()
File "/home/joe/miniconda3/envs/sign-language/lib/python3.9/site-packages/torchmetrics/classification/auroc.py", line 185, in compute
return _auroc_compute(
File "/home/joe/miniconda3/envs/sign-language/lib/python3.9/site-packages/torchmetrics/functional/classification/auroc.py", line 173, in _auroc_compute
raise ValueError(
ValueError: Argument average expected to be one of the following: ('None', 'macro', 'weighted') but got micro

I consider it is caused by how I implement my code. (I am doing multiclass classification)
This error occurs at the beginning when pytorch lightning does the val verification before starting training in the first epoch. As the error message shown, it occurs when do val_auroc = self.val_auroc.compute() in validation_epoch_end

class Module(pl.LightningModule):

    def __init__(self, model, **kwargs):
        super().__init__()
        ...
        self.train_auroc = torchmetrics.AUROC(num_classes=NUM_CLASSES, average="micro")
        ...
        self.val_auroc = torchmetrics.AUROC(num_classes=NUM_CLASSES, average="micro")
        ...

    def training_step(self, batch, batch_idx):
        x, y = batch
        z = self.forward(x)
        ...
        self.train_auroc.update(z, y)
        ...

    def validation_step(self, batch, batch_idx):
        x, y = batch
        z = self.forward(x)
        ...
        self.val_auroc.update(z, y)
        ...

    def training_epoch_end(self, training_step_outputs):
        ...
        train_auroc = self.train_auroc.compute()
        ...

    def validation_epoch_end(self, validation_step_outputs):
        ...
        val_auroc = self.val_auroc.compute()
        ...
        self.val_auroc.reset()

Sorry for the misunderstanding about "allowed_average", I have not been very clear. I wanted to show the place where "raise ValueError" occurred which was in the last reference, but it sounded more like I was referring to the section of code you showed me.

[Borda]

@JoeNatan30 — this was a bug in torchmetrics 0.6.x. Fully resolved now.

Current usage (v1.9.0):

from torchmetrics.classification import MulticlassAUROC

auroc = MulticlassAUROC(
    num_classes=NUM_CLASSES,
    average="macro",     # "macro", "weighted", or "none"
    thresholds=None,
)
score = auroc(preds, target)

Important: average="micro" is intentionally not supported for multiclass AUROC — it doesn't have a standard definition for the one-vs-rest case. The scikit-learn docs note this same subtlety.

If you need micro-averaged AUROC, convert to multilabel:

from torchmetrics.classification import MultilabelAUROC
import torch.nn.functional as F

target_onehot = F.one_hot(target, num_classes=NUM_CLASSES)
auroc = MultilabelAUROC(num_labels=NUM_CLASSES, average="micro")
score = auroc(preds, target_onehot)

Docs: AUROC