It costs a long time to test on the DocVQA

[01upup10]

When I use Qwen3-VL-8B Instruction to evaluate on the DocVQA_val, I waited for a long time but didn't get the final test results after running the dataset. The program seems to be stuck. My command is

accelerate launch --num_processes 4 -m lmms_eval \
  --model qwen3_vl \
  --model_args=pretrained=Qwen/Qwen3-VL-8B-Instruct,max_pixels=12845056,attn_implementation=flash_attention_2,interleave_visuals=False \
  --tasks docvqa_val \
  --batch_size  3 \
  --limit 1500 \
  --output_path ./logs/Qwen3-VL-8B-Instruct_docvqa_limit1500_eval

[Fu-Fu-Fu-Fu]

不是数据集的问题，我也是几乎同样的命令，测试qwen3vl系列的模型，奇慢无比，450个样本要几十个小时，这是很离谱的事情。用vllm会好很多，但是vllm测试出来的qwen3vl模型的结果比官方报告总是低很多

[Fu-Fu-Fu-Fu]

发现上周更新了qwen3vl的chat文件，使用这个速度变得好一点了，但是还是很慢

[Maerceci-27]

Check if the PyTorch version is 2.9. There seem to be some issues with conv3d (which Qwen3VL uses for visual embedding) in PyTorch 2.9. Try downgrading to 2.8 or an earlier version.

[Fu-Fu-Fu-Fu]

Check if the PyTorch version is 2.9. There seem to be some issues with conv3d (which Qwen3VL uses for visual embedding) in PyTorch 2.9. Try downgrading to 2.8 or an earlier version.检查一下 PyTorch 版本是否为 2.9。PyTorch 2.9 中 conv3d（Qwen3VL 用于视觉嵌入）似乎存在一些问题。尝试降级到 2.8 或更早的版本。

Hello, my Torch version is 2.8, and I'm still experiencing slow testing. Also, for qwen3vl, vllm requires at least version 0.11, which corresponds to a minimum Torch version of 2.8.