grpo_demo.ipynb: Excessive output

[cool-RR]

This isn't an outright bug, just something that makes it difficult to work with grpo_demo.ipynb.

The cell with grpo_trainer.train produces a huge amount of text output. It becomes 95% of the notebook. This makes it difficult to scroll around the notebook or even tell when it's finished. I understand that you would like to show samples of outputs, but if it's important, I'd suggest showing a much smaller selection of outputs to keep the notebook workable.

If you agree with this I'll be happy to help.

[wang2yn84]

Hi cool-RR, thank you for the inputs! Are you talking about the training section? We can comment out the messages. Do you have any suggestions of how to show a much smaller selection of outputs?

[cool-RR]

I'm not sure how to print less outputs, but I suggest as a first step to completely stop printing outputs by simply removing this block from the definition of check_numbers in the notebook:

print("START ============================")
print(f"Question: {question[0]}")
print(f"Answer: {answer[0]}")
print(f"Response: {responses[0]}")
print(f"Extracted: {extracted_responses[0]}")
print("END ==============================")

If that works for you, I can do a PR that removes it here and in the 3 other scripts in which it's used. If you do want to print some output, it should not be in the reward function, which I suggest should remain functional without side effects.

[wang2yn84]

Thank you! Another contributor will merge a more polished notebook today or tomorrow. Can you hold on a bit and let's work on top of the newer version?

[cool-RR]

Sure, thank you.