-
Notifications
You must be signed in to change notification settings - Fork 22
[BUG]: Error with NRMSE calculation with model=1 #11
Description
Description of the bug
For the calculation of NRMSE with model=1, RMSE is normalized by the mean of the predicted values, not the mean of the true values. See https://agrimetsoft.com/calculators/Normalized%20Root%20Mean%20Square%20Error
normalized_root_mean_square_error(self, y_true=None, y_pred=None, model=1):
if model == 1:
result = rmse / np.mean(y_pred, axis=0)
Steps To Reproduce
from numpy import array
from permetrics.regression import RegressionMetric
tru = array([5.0, 30.0, 50.0, 60.0, 45.0])
pre = array([3.0, 44.0, 57.0, 57.0, 35.5])
evaluator = RegressionMetric(tru, pre)
print(evaluator.RMSE()) # returns 8.346
print(evaluator.NRMSE(model=1)) # returns 0.212
print(evaluator.RMSE()/pre.mean()) # returns 0.212 since the mean of the predicted (modeled) values is 39.3
If RMSE is normalized by the mean of the true values (38.0), the answer is 0.220
print(evaluator.RMSE()/tru.mean()) # returns 0.220
Additional Information
The following websites state RMSE should be normalized by the mean (or std. deviation or range etc.) of the true (observed values):
https://agrimetsoft.com/calculators/Root%20Mean%20Square%20Error
https://search.r-project.org/CRAN/refmans/hydroGOF/html/nrmse.html
https://lifesight.io/glossary/normalized-root-mean-square-error/
https://www.marinedatascience.co/blog/2019/01/07/normalizing-the-rmse/