Is the newly implemented GRPO supposed to be slower than PPO?

[yxchng]

it seems to be much slower on my side with G=8? though with less memory (which is expected)

[qgallouedec]

Yes, generation is the main bottleneck here, as for each prompt, you need to generate several completions (8 by default). We're currently pushing hard to make the generation way faster, see #2600