[Bug] Sequence Packing of Qwen3-VL

[lostkevin]

Describe the bug

Hi devs! Recently we attempt to train Megatron-Bridge-based Qwen3-VL with slime and find large logprobs diff between train and rollout backends. After deeply investigating the modeling, we find that:

1. get_rope_index called in model.py cannot correctly process packed input_ids.
2. _apply_rotary_pos_emb_thd only support RoPE of traditional mapping without offsets when apply_rope_fusion is enabled. --no-rope-fusion must be set to get the correct behavior.

Here is a draft patch fix (for dense and MoE, not strictly tested) on 5c7ebe7 .

[nanjiangwill]

hey @lostkevin, thanks for this patch! i am trying to train qwen3-vl-moe with this patch, the log prob diff is smaller but it is still very high(with megatron, the logprob diff between rollout and train is like 0.73 but with fsdp it is 0.03)

[lostkevin]

hey @lostkevin, thanks for this patch! i am trying to train qwen3-vl-moe with this patch, the log prob diff is smaller but it is still very high(with megatron, the logprob diff between rollout and train is like 0.73 but with fsdp it is 0.03)

We've reached 0.82 on Geo-3K with Qwen3-VL-4B on an internal slime impl (also based on Megatron-Bridge) after ~1K steps with the above patch, the logprob diff is ~0.01 during training process.

[nanjiangwill]

hey @lostkevin, thanks for this patch! i am trying to train qwen3-vl-moe with this patch, the log prob diff is smaller but it is still very high(with megatron, the logprob diff between rollout and train is like 0.73 but with fsdp it is 0.03)

We've reached 0.82 on Geo-3K with Qwen3-VL-4B on an internal slime impl (also based on Megatron-Bridge) after ~1K steps with the above patch, the logprob diff is ~0.01 during training process.

interesting, because previously i ran megatron with dense vl model and without this patch the logprob diff is 0.01 for qwen3-vl-4b.

the issue right now is that the qwen3-vl-moe does not work

[lostkevin]

We observed logprobs diff at ~0.4 w/o the above patch on 4B model with custom slime. Besides, we also tested the official slime commit of Qwen3-VL 2B, the logprobs is around 0.15. The MoE experiment may be done in the near future.

[nanjiangwill]

i got the following stats with (relatively) latest slime docker and megatron(megatron-bridge):

for qwen3-vl-2b-instruct: logprobs diff is 0.044
for qwen3-vl-4b-instruct: logprobs diff is 0.012
for qwen3-vl-8b-instruct: logprobs diff is 0.115
for qwen3-vl-30b-a3b-instruct: logprobs diff is 1.228(without this patch) and 0.85(with this patch)

[lostkevin]

@yaoyu-33 Recently we check the convergence of Qwen3-VL-30B-A3B with GRPO on GEO3K and finally reach eval reward 0.8 at ~200 steps after solving some issues:

1. The RoPE of MoE will lose precision on position_ids, as inv_freq is bf16

Megatron-Bridge/src/megatron/bridge/models/qwen_vl/modelling_qwen3_vl/rope.py

Line 46 in d94cb20

seq = position_ids.to(device=self.inv_freq.device, dtype=self.inv_freq.dtype)

2. The ExpertMLPDownProjMapping and ExpertMLPGateUpProjMapping are problematic:

• .contiguous() is not added in hf_megatron, which will encounter PyTorch issue, see dist.scatter is incompatible with transpose/permute operation pytorch/pytorch#107352
• If the order in hf_to_megatron is transpose->super(), the order in megatron_to_hf should be super()->transpose
• ExpertMLPGateUpProjMapping should use logic in GatedMLPMapping, but currently ColumnParallelMapping

3. maybe_modify_converted_hf_weight is missing in Qwen3VLMoEBridge