center_rewards_coefficient always null

[yardenhoch]

Hi,

When training a reward model in the past, I used the center_rewards_coefficient parameter in my config.
However, now when I add it to my config file, I see in the wandb run config that it’s always set to null.

Example snippet from my config:
center_rewards_coefficient: 0.01

Expected behavior:
center_rewards_coefficient should be respected and appear with the correct value in the wandb config.

Actual behavior:
It is always logged as null in wandb, and I’m not sure if it’s being used at all during training.

Axolotl version: 0.12.2
Command used: accelerate launch -m axolotl.cli.train /tmp/config.yaml

[NanoCode012]

Hello, could you share where you found that parameter center_rewards_coefficient?

Edit: Seems to be a Reward Modeling config https://huggingface.co/docs/trl/v0.10.1/en/reward_trainer#centering-rewards , we didn't pass this config through before, that was why wandb shows it as None.

[NanoCode012]

Would you be interested in making a PR to add this to Axolotl? This would involve:

• Adding it to our Pydantic schema src/axolotl/utils/schemas/config.py
• Pass it to the trainer training_arguments_kwargs within src/axolotl/core/builders/causal.py
• Adding it to the list in set base args src/axolotl/core/builders/base.py
• Testing that it runs

[yardenhoch]

@NanoCode012 I've created the PR as requested

PR: Center rewards coefficient #3124

This implements all the requirements you mentioned:
✅ Added to Pydantic schema (src/axolotl/utils/schemas/config.py)
✅ Passed to training_arguments_kwargs (src/axolotl/core/builders/causal.py)
✅ Added to base args list (src/axolotl/core/builders/base.py)
✅ Comprehensive testing included

The PR includes documentation, example configs, and unit tests. Ready for review!