Compute AUPRC?

[2533245542]

Currently I could only use torchmetrics.classification.PrecisionRecallCurve. Is there an easy way to compute AUPRC from it?

[SkafteNicki]

You can use the torchmetrics.classification.PrecisionRecallCurve to get the precision recall curve and afterwards use the auc implementation (https://torchmetrics.readthedocs.io/en/latest/references/functional.html#auc-func) to get the AUPRC value.

[angerhang]

@SkafteNicki I don't think this method works anymore. Do you know how we can obtain the AUPRC from the precision recall curve? I could use the AUC function from sklearn but I don't think if that will has the same set of lighting features.

[Borda]

@angerhang, why do you assume that it does not work?

[el-hult]

Look for torchmetrics.utilities.auc for the auc computation from the precision and recall lists.

[el-hult]

Consider using Average Precision instead. they are almost the same, (not exactly, but almost).
unless your application really needs AUPRC, the AP is already implemented and simple to use.

[Borda]

Building on @SkafteNicki's and @el-hult's suggestions — as of TorchMetrics v1.9.0, you don't need to manually compute AUPRC anymore. The AveragePrecision metric is mathematically equivalent to AUPRC (area under the precision-recall curve) and is a first-class citizen:

from torchmetrics.classification import BinaryAveragePrecision

# Binary case
metric = BinaryAveragePrecision(thresholds=None)  # exact computation
preds = torch.tensor([0.1, 0.4, 0.35, 0.8])
target = torch.tensor([0, 0, 1, 1])
metric(preds, target)  # tensor(0.8333)

For multiclass:

from torchmetrics.classification import MulticlassAveragePrecision

metric = MulticlassAveragePrecision(num_classes=5, average="macro", thresholds=None)

For multilabel:

from torchmetrics.classification import MultilabelAveragePrecision

metric = MultilabelAveragePrecision(num_labels=3, average="macro", thresholds=None)

Performance tip: if memory is a concern (large datasets), pass thresholds=200 to use binned computation — constant memory at the cost of slight approximation.

The old PrecisionRecallCurve + auc() approach still works but is unnecessary for most cases. Use AveragePrecision directly.

Docs: AveragePrecision

[el-hult]

N.B. they are not exactly the same. at least not in the various software implementations online. the interpolate differently between the points on the PR curve. The AP uses a step function. The AUPRC (often) uses a trapezoidal rule (linear interpolation). If there are enough points on the PR curve, the difference between the methods is effectively zero. But in cases when e.g. the test set is small, this can matter. Below is a small numerical study of the bias made in sklearn, with their AUPRC and AP impementations. The result depends on your exact model and data set, of course, but in this case, the difference was effectively zero at N=200.

For most torchmetrics users, the AP should be enough.