Skip to content

Qwen2.5-VL Expect all tensors on the same devices #952

@yyy195

Description

@yyy195

Dear author,
when I evaluate Qwen2.5-VL-3B on only one GPU and set num_processes=1, the error occurred!
"Error during evaluation: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select). Please set --verbosity=DEBUG to get more information."

I've searched issues for this , but all are about multiple GPUS, so could you help me solve this. ‼️I've tried "device_map=auto",but it not works
My .sh file is as follows

export HF_HOME="~/.cache/huggingface"
export CUDA_VISIBLE_DEVICES=0

accelerate launch --num_processes=1 --main_process_port=12344 -m lmms_eval
--model qwen2_5_vl
--model_args=pretrained=./datasets/Qwen2.5-VL-3B-Instruct,max_pixels=12845056,attn_implementation=flash_attention_2,interleave_visuals=False,device_map=auto
--tasks mathvision_test
--device cuda:0
--output_path ./logs/
--batch_size 4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions