Add AUC ROC and AUC PRC metrics

[gsateesh5]

Hi All,

To compare the results of continual learning algorithms on imbalanced problems ROC-AUC and PRC-AUC are better metrics.

So, Is it possible to add ROC-AUC and PRC-AUC metrics to the avalanche library?

[AndreaCossu]

Hi @gsateesh5 ! The best way to do this is to create a new metric subclassing the PluginMetric base class and then pass its instance to the EvaluationPlugin instance in order to have it computed during training/evaluation.

In order to build the new metric you can look at the different Accuracy metrics (e.g. here ).
You can access the output of your model through strategy.mb_output and the targets via strategy.mb_y. You can even access the model through strategy.model.

The strategy object is available within every callback of the PluginMetric (e.g. after_train_minibatch, after_eval_exp...). You just need to subclass the callbacks you need and add your logic there.

Does this help?