Question about the performance of GRPO

[hh666-star]

Hello,
I run verl-agent/examples/grpo_trainer/run_webshop.sh in the webshop to test the performance of the GRPO baseline and found that the final results are much higher than the results reported in the paper, even higher than SPEAR. Could you please explain why this is happening and why this situation occurs?

[yuleiqin]

Hello, I run verl-agent/examples/grpo_trainer/run_webshop.sh in the webshop to test the performance of the GRPO baseline and found that the final results are much higher than the results reported in the paper, even higher than SPEAR. Could you please explain why this is happening and why this situation occurs?

Yes indeed, this is possible. We find that the webshop benchmark is of high variance (either higher or lower than the reported or cited from GiGPO paper). We recommend that you can try more seed (e.g., 0, 42, ...) and use the highest few results for average. Otherwise, the performance can very greatly and it is difficult to analyze whether it is due to the algorithm or simply the randomness. @hh666-star