GRPO training examples for Qwen2.5

[sglucas]

Hi, thank you very much for your great work. I noticed that current examples mainly focus on Qwen3 series, may I ask whether the project plan to relase some examples for Qwen2.5 models, since I found the reward will always be 0 if directly change the model from Qwen3 to Qwen2.5 in current examples.

Best,

[Zhuohao-Li]

can you share your launching scripts?

[sglucas]

Thank you for your response. I follow the command to start my experiment:

python3 train.py \
--hf-checkpoint ./Qwen2.5-1.5B-Instruct --load ./Qwen2.5-1.5B-Instruct --ref-load ./Qwen2.5-1.5B-Instruct --save ./checkpoints \
--save-interval 20 --prompt-data /opt/dapo-math-17k.jsonl \
--input-key prompt --label-key label --apply-chat-template --rollout-shuffle --balance-data --rm-type deepscaler \
--num-rollout 100 --rollout-batch-size 8 --n-samples-per-prompt 8 --rollout-max-response-len 4096 --rollout-temperature 0.8 \
--global-batch-size 64 --eval-interval 20 --eval-prompt-data aime /opt/aime-2024/aime-2024.jsonl --n-samples-per-eval-prompt 8 \
--eval-max-response-len 16384 --eval-top-p 0.7 --optimizer adam --lr 5e-5 --lr-decay-style constant --weight-decay 0.1 \
--adam-beta1 0.9 --adam-beta2 0.98 --use-kl-loss --advantage-estimator grpo --kl-loss-coef 0.00 --kl-loss-type low_var_kl \
--kl-coef 0.00 --entropy-coef 0.00 --eps-clip 0.2 --eps-clip-high 0.28 --use-wandb --wandb-project slime --wandb-group qwen3-4B-fsdp-4gpu\
 --rollout-num-gpus-per-engine 1 \
--sglang-mem-fraction-static 0.75 --sglang-decode-log-interval 1000 --sglang-chunked-prefill-size 4096 --sglang-attention-backend flashinfer \
--train-backend fsdp --update-weight-buffer-size 536870912 --gradient-checkpointing --attn-implementation sdpa \
--use-dynamic-batch-size --max-tokens-per-gpu 9216 \
--actor-num-nodes 1 --actor-num-gpus-per-node 4 --colocate --use-fault-tolerance --dump-details /opt/shared_data/run_20251220_152850/dump_details

[jiahe7ay]

@sglucas The issue stems from the rm-type. The deepscaler reward model specifically looks for or ###Response tags, assigning a zero score if they are missing. Switching the rm-type to math should resolve this. See /slime/rollout/rm_hub/deepscaler.py for the underlying logic