Unexpected results when using ConfusionMatrixMetric

[hblanco2009]

Hello guys, I'm having problems to properly compute some metrics using the Monai's ConfusionMatrixMetric class.
I guess I'm doing something wrong or there are some parameters missing. Attached you can see the example and the code. Particularly, the "accuracy" is correctly computed while the "f1 score" is not. I'm always getting the same metric value.

Any ideas on what's wrong?

Thanks in advance
H.

[KumoLiu]

Hi @hblanco2009, if you only want to calculate the second channel, you should set include_background to False. By default, it's True.

metric = ConfusionMatrixMetric(metric_name="f1 score", include_background=False)

Hope it can help you, thx!

[yiheng-wang-nv]

Hi @hblanco2009 ,
The default setting for the class is that it will consider all classes, in your case the number is 2. If only count class 1, you need to set include_background=False.

[hblanco2009]

Thank you so much. it works perfectly now!
Best regards
H.

[giemmecci]

Hi, @KumoLiu @yiheng-wang-nv ,

Since, in some scenarios, we are interested in having the confusion matrix metrics for multiple classes (i.e., not only the positive class), what would be a way to simulate the behavior of the classification report from scikit learn in MONAI?

For example, how can we calculate the macro average of each metric of the confusion matrix for the two classes in the example provided by the original poster? The aim would be to obtain the same information shown in the last cell of one of your tutorials.

Thanks!

[KumoLiu]

Hi @giemmecci, thanks for your interest here.
I think you can set reduction=mean_channel to get the average metric for all classes.

MONAI/monai/metrics/confusion_matrix.py

Lines 54 to 56 in 0037679

	reduction: define mode of reduction to the metrics, will only apply reduction on `not-nan` values,
	available reduction modes: {``"none"``, ``"mean"``, ``"sum"``, ``"mean_batch"``, ``"sum_batch"``,
	``"mean_channel"``, ``"sum_channel"``}, default to ``"mean"``. if "none", will not do reduction.

Hope it can help you, thanks!