Question about different average in Accuracy.

[busFred]

I'm having a little hard time understanding the difference between different ways of averaging accuracy score. I got a imbalanced 4 categories classification problem and i'm attempting to compute the balanced_accuracy_score. I'm wondering which option should i choose

[Borda]

The choice of averaging method for accuracy metrics depends on your specific goals and the class distribution in your dataset. For an imbalanced 4-category classification problem, balanced accuracy is often preferred because it equally weights the accuracy of each class, preventing dominant classes from skewing the overall metric. This is usually done by computing the recall (true positive rate) for each class independently and then averaging these scores (often called “macro” averaging).
If you want a single accuracy value that treats all classes fairly despite imbalance, use balanced accuracy (macro averaging of recalls), not the plain overall accuracy which can be biased towards majority classes.
In TorchMetrics, check if their accuracy metric has a average='macro' or specifically a balanced accuracy option. If unsure, you can calculate balanced accuracy manually by computing recall per class and averaging.