GRPO training

[TanZz9]

Hi authors,
I’m having an issue when running GRPO training.

[rank0]: output_reward_func = reward_func( [rank0]: TypeError: accuracy_reward() missing 1 required positional argument: 'assistant'

Is there a quick fix? Any guidance would be greatly appreciated. Thanks for your help!

[2U1]

Hi,

This error occurs because the accuracy_reward function requires an assistant argument, but it's not being passed from your dataset.

def accuracy_reward(completions, assistant, **kwargs):
# 'assistant' is required for comparing completions against ground truth

Possible causes:

1. Your dataset doesn't include the assistant field - The GRPODataset in this repo returns a dict with prompt, assistant, images, etc. Make sure your dataset also returns the assistant field containing the ground truth response.
2. The assistant column is being removed - If you're using a custom trainer or modified _signature_columns, ensure "assistant" is included:
   self._signature_columns = ["prompt", "assistant", "images", ...]

Quick fix:

Check your dataset's getitem method and make sure it returns:

  return {
      "prompt": your_prompt,
      "assistant": ground_truth_answer,  # ← This is required
      "images": your_images,
      ...
  }

If you don't need answer verification and just want format-based rewards, you can modify reward_funcs.py to only use format_reward which doesn't require assistant.

Hope this helps!