Can‘t use customize metrics

[Cua1103]

I was trying to use custom metrics Accuracy from torchmetrics:

from torchmetrics import Accuracy
from anomalib.metrics import AnomalibMetric, create_anomalib_metric

AnomalibAccuracy = create_anomalib_metric(Accuracy)
accuracy = AnomalibAccuracy(fields=["pred_label", "gt_label"])

However:
Traceback (most recent call last):
File "/home/admin001/anomalib/test002.py", line 7, in
accuracy = AnomalibAccuracy(fields=["pred_label", "gt_label"])
TypeError: Accuracy.new() missing 1 required positional argument: 'task'

So I tried another option in documentation:

from torchmetrics import Accuracy
from anomalib.metrics import AnomalibMetric, create_anomalib_metric

class AnomalibAccuracy(AnomalibMetric, Accuracy):
    pass
accuracy = AnomalibAccuracy(fields=["pred_label", "gt_label"])

But the error remained. So I added "task":
accuracy = AnomalibAccuracy(fields=["pred_label", "gt_label"],task='binary')

An error occurred：
Traceback (most recent call last):
File "/home/admin001/anomalib/test002.py", line 7, in
accuracy = AnomalibAccuracy(fields=["pred_label", "gt_label"],task='binary')
File "/opt/anaconda3/envs/anomalib/lib/python3.10/site-packages/torchmetrics/classification/accuracy.py", line 515, in new
return BinaryAccuracy(threshold, **kwargs)
File "/opt/anaconda3/envs/anomalib/lib/python3.10/site-packages/torchmetrics/classification/stat_scores.py", line 173, in init
super(AbstractStatScores, self).init(**kwargs)
File "/opt/anaconda3/envs/anomalib/lib/python3.10/site-packages/torchmetrics/metric.py", line 154, in init
raise ValueError(f"Unexpected keyword arguments: {', '.join(kwargs)}")
ValueError: Unexpected keyword arguments: fields

I am a beginner and I wonder what did I missed?

anomalib 2.1.0.dev0
torch 2.7.0
torchmetrics 1.7.1
CUDA 12.1

[rajeshgangireddy]

Hi @Cua1103,
This is because the Accuracy class from torchmetrics uses new() instead of init() to create an object.

I suggest using a more specific torchmetrics accuracy class. It looks like you want to use binary classification metric with AnomalibMetric

from torchmetrics.classification.accuracy import BinaryAccuracy
from anomalib.metrics import AnomalibMetric

class AnomalibBinaryAccuracy(AnomalibMetric, BinaryAccuracy):
    pass

binary_accuracy = AnomalibBinaryAccuracy(
    fields=["pred_label", "gt_label"],
)

Can also be used with create_anomalib_metric

from torchmetrics.classification.accuracy import BinaryAccuracy
from anomalib.metrics import create_anomalib_metric

binary_accuracy = create_anomalib_metric(BinaryAccuracy)
binary_accuracy = binary_accuracy(
    fields=["pred_label", "gt_label"],
)

Let me know if this solves your question.

[Cua1103]

Hi @Cua1103, This is because the Accuracy class from torchmetrics uses new() instead of init() to create an object.

I suggest using a more specific torchmetrics accuracy class. It looks like you want to use binary classification metric with AnomalibMetric

from torchmetrics.classification.accuracy import BinaryAccuracy
from anomalib.metrics import AnomalibMetric

class AnomalibBinaryAccuracy(AnomalibMetric, BinaryAccuracy):
    pass

binary_accuracy = AnomalibBinaryAccuracy(
    fields=["pred_label", "gt_label"],
)

Can also be used with create_anomalib_metric

from torchmetrics.classification.accuracy import BinaryAccuracy
from anomalib.metrics import create_anomalib_metric

binary_accuracy = create_anomalib_metric(BinaryAccuracy)
binary_accuracy = binary_accuracy(
    fields=["pred_label", "gt_label"],
)

Let me know if this solves your question.

Thank you. That works.