The addition of non-math-nor-code RL tasks to the training of R1 (but maybe not R1-Zero)

[sunjin-k]

The R1 report states "rule-based rewards [...] in math, code, and logical reasoning domains" were used.

I assume the logical reasoning tasks refer to things like [1][2].
(In [2], it was shown that RLing on one particular type of logic puzzle was enough to improve performance in AIME/AMCto generalize to math performance).

Plus RL on formatting correctness tasks[3] might also have been done.

Perhaps the collection of such non-math&code RL tasks should be added to to-do list?

Also might be worth noting: the report mentions the usage of logical reasoning tasks only in the section on R1 and not R1-zero.

[1] https://github.com/open-thought/reasoning-gym
[2] Xie, Tian, et al. "Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning."
[3] like the IFeval task in Tulu3 https://huggingface.co/datasets/allenai/RLVR-IFeval?row=0

[ocramz]

Maybe relevant: #379