Questions regarding leaderboard results

[slies]

I have downloaded the results folder from this repository and noticed that the average MASE value does not match the value reported on the leaderboard. Could you please explain how the mean value of the metrics is calculated?

[cuthalionn]

Hi @slies, the values shown in the leaderboard are calculated after standardizing by seasonal_naive results across each 97 configurations, and then taking a geometric mean. The leaderboard source code does the calculation on the exact same results file that reside in this repo.

[gwyanCN]

Hello，I tried using the geometric mean indicator to calculate the result of Moirai2, but the result still didn't match the one on the list. Could there be any omissions? Here is the process I used for calculation:

[cuthalionn]

Hi @gwyanCN, are you standardizing results by seasonal naive?

Also, for any future queries I would suggest refering directly to leaderboard source code --> https://huggingface.co/spaces/Salesforce/GIFT-Eval/tree/main

This is where the calculation logic happens on the fly.