GRPO for Qwen3-VL

[woxihuanjiangguo]

Hi authors,

I am using GRPO to fine-tune Qwen/Qwen3-VL-2B-Instruct and encoutering the following error:

[rank0]:   File "src/trainer/grpo_trainer.py", line 53, in _generate_and_score_completions
[rank0]:     prompts = [x["prompt"] for x in inputs]
[rank0]:   File "src/trainer/grpo_trainer.py", line 53, in <listcomp>
[rank0]:     prompts = [x["prompt"] for x in inputs]
[rank0]:                ~^^^^^^^^^^
[rank0]: TypeError: string indices must be integers, not 'str'

It looks like the autoprocessor of Qwen3-VL is turning the inputs into a BatchFeature object with the following keys:
b.data.keys() Out[8]: dict_keys(['input_ids', 'attention_mask', 'pixel_values', 'image_grid_thw'])

Are there any quick fixes I can do? Thanks for the help!

[2U1]

I've made a safe guard for the input.

    def _generate_and_score_completions(
        self, inputs: list[dict[str, torch.Tensor | Any]]
    ) -> dict[str, torch.Tensor | Any]:
        device = self.accelerator.device
        mode = "train" if self.model.training else "eval"

        if isinstance(inputs, dict):
            bsz = len(inputs["prompt"])
            inputs = [
                {k: (v[i] if v is not None else None) for k, v in inputs.items()}
                for i in range(bsz)
            ]

        prompts = [x["prompt"] for x in inputs]

This would fix the problem I think.

[JayceonHo]

I've made a safe guard for the input.

    def _generate_and_score_completions(
        self, inputs: list[dict[str, torch.Tensor | Any]]
    ) -> dict[str, torch.Tensor | Any]:
        device = self.accelerator.device
        mode = "train" if self.model.training else "eval"

        if isinstance(inputs, dict):
            bsz = len(inputs["prompt"])
            inputs = [
                {k: (v[i] if v is not None else None) for k, v in inputs.items()}
                for i in range(bsz)
            ]

        prompts = [x["prompt"] for x in inputs]

This would fix the problem I think.

Hi, i have the same problem, and it still does not work. I print the inputs:
"{'input_ids': tensor([[151644, 8948, 198, 2610, 525, 264, 10950, 17847, 13,
151645, 198, 151644, 872, 198, 151652, 151655, 151655..
151655, 151655, 151655, 151655, 151653, 28715, 389, 279, 2168,
11, 892, 52484, 1714, 374, 1483, 311, 8253, 279,
18043, 9210, 11, 323, 3170, 374, 432, 279, 1429,
8311, 5754, 30, 151645, 198, 151644, 77091, 198]]), 'attention_mask': tensor([[1, 1, 1, 1... 1]]), 'pixel_values': tensor([[-0.0667, -0.0667, -0.0667, ..., -0.0275, -0.0196, -0.0196],
[-0.0588, -0.0667, -0.0745, ..., -0.1059, -0.1059, -0.1059],
[-0.0353, -0.0353, -0.0353, ..., -0.0510, -0.0353, -0.0353],
...,
[-0.8667, -0.8745, -0.8667, ..., -0.9608, -0.9529, -0.9294],
[-0.9216, -0.9294, -0.9451, ..., -0.7098, -0.7412, -0.7255],
[-0.9686, -0.9765, -0.9686, ..., -0.7490, -0.7176, -0.7020]]), 'image_grid_thw': tensor([[ 1, 28, 28],
[ 1, 28, 28]])}"

and i checked isinstance(inputs, dict) is False

[2U1]

@woxihuanjiangguo @JayceonHo I've tested with some data and now it would work. Sorry for the issue.
I was trying with old versions of libraries, and datas.