Questions about AUC truncation and per-task op stride on the leaderboard

[xyn24]

Hi! Thanks for open-sourcing and maintaining gsm-infinite—the repo and leaderboard are very helpful for reproduction. I have two questions while aligning the evaluation:

1. The paper says AUC is a Riemann sum starting from op=2, truncated when accuracy drops below 5%. I couldn’t find the “<5% truncation” logic in the repo, and symbolic seems to start from op=1. Is this intended? If truncation exists, could you point to the relevant code?

2. The leaderboard uses different op strides across tasks. Is there a recommended convention for choosing stride to reproduce/align results?

Thanks!