cannot use the inference code

[VagRant2333]

System Info

RuntimeError: shape mismatch: value tensor of shape [50, 2048] cannot be broadcast to indexing result of shape [8, 2048]

Thanks a lot for helping

Reproduction

RuntimeError: shape mismatch: value tensor of shape [50, 2048] cannot be broadcast to indexing result of shape [8, 2048]

Expected behavior

/

[DelinQu]

Hello VagRant2333,

I'm confident that this issue is related to the chunksize setting. Could you provide more details to help us investigate further?

[VagRant2333]

Thanks a lot for replying and helping!
my code is just modified from README, I am trying to complete a simple inference code so I curated some random data:

processor = AutoProcessor.from_pretrained("IPEC-COMMUNITY/eo1-qwen2_5_vl-initial", trust_remote_code=True, cache_dir="***")
model = AutoModel.from_pretrained(
  "IPEC-COMMUNITY/eo1-qwen2_5_vl-initial",
  cache_dir="***",
  trust_remote_code=True, 
  dtype=torch.bfloat16,
  # local_dir_use_symlinks=False,
  # resume_download=True,
  token="***"
).eval().cuda()


import numpy as np
from PIL import Image
image0 = (np.random.rand(224, 224, 3) * 255).astype(np.uint8)
image1 = Image.fromarray(image0.copy())

# prepare the model input
batch = {
        "observation.images.image": [image0],
        "observation.images.wrist_image": [image1],
        "observation.state": [np.random.randn(8).tolist()],
        "task": ["put the object in the box."],
        # "repo_id": [args.repo_id],
}
ov_output = processor.select_action(
  model,
  batch,
)

print(ov_output)

and it gets shape mismatch Error.

---

also, I have several questions on https://huggingface.co/IPEC-COMMUNITY/eo1-qwen2_5_vl, i
s it the finetuned version of EO1? can I use it to inference and test it on LIBERO? As the experiments/2_libero/train.sh will train qwen2.5VL on LIBERO's data again, I think it will differ from the performace of your version of EO1.

It's quite confusing to me, appreciate very much for solving my questions!

[DelinQu]

Hi VagRant2333, this version "IPEC-COMMUNITY/eo1-qwen2_5_vl-initial" is a quick start for my recent lerobot pr huggingface/lerobot#1971. It's designed for lerobot fast loading. In your experiments, you can use this repo's scripts directly in libero training.

[VagRant2333]

oh I see, so I just cannot use EO1 official model to inference right now, isn't it?

and while I am trying to train qwen2.5vl on LIBERO's dataset using this repo's train code, I got warning:

You are using a model of type qwen2_5_vl to instantiate a model of type eo1. This is not supported for all configurations of models and can yield errors.

did you meet similar situations and how did you fix it? and after training qwen2.5vl on LIBERO, is it the same as EO-1 model?

thank you for answering!

[DelinQu]

It’s a. Hugging face transformer model_type checking, you can easily find it in the hf repo. Bout do not cause any affects. Feel free to load qwen vl.

…

On Nov 14, 2025, at 12:47, 邱深凌(Qiu Shenling) ***@***.***> wrote: VagRant2333 left a comment (SHAILAB-IPEC/EO1#1) <#1 (comment)> oh I see, so I just cannot use EO1 official model to inference right now, isn't it? and while I am trying to train qwen2.5vl on LIBERO's dataset using this repo's train code, I got warning: You are using a model of type qwen2_5_vl to instantiate a model of type eo1. This is not supported for all configurations of models and can yield errors. did you meet similar situations and how did you fix it? and after training qwen2.5vl on LIBERO, is it the same as EO-1 model? thank you for answering! — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOOJI5F7C74GJUM52FTL2CD34VNGDAVCNFSM6AAAAACL4LXCCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMZQHAYTSMZWHA>. You are receiving this because you commented.

[VagRant2333]

Thanks for answering!
I want to know when I use embodied-onevision/EO1/experiments/2_libero/train.sh to train qwen2.5VL-3B(not EO1 ckpt) with LIBERO's data, will the backbone(qwen) be freezed?
and is it trained like EO1, which means language and action is generated together?(I doubt it as Libero's data lack language)

[DelinQu]

Thanks for answering! I want to know when I use embodied-onevision/EO1/experiments/2_libero/train.sh to train qwen2.5VL-3B(not EO1 ckpt) with LIBERO's data, will the backbone(qwen) be freezed? and is it trained like EO1, which means language and action is generated together?(I doubt it as Libero's data lack language)

In VLA benchmarks like LIBERO and Simpler, we perform full-parameter fine-tuning and don’t use any language. These simplified benchmarks only test a model’s fitting ability and are not related to language understanding.

In contrast, latest benchmarks such as BEHAVIOR-1K expose the limitations of VLA models.

[dwipddalal]

Hi @DelinQu,
So, in the finetuning for the SimplerEnv environment, only two datasets, i.e., Fractal and Bridge datasets, were used? This would imply that there’s no language-specific loss involved, only action-specific loss. Is my understanding accurate?

[ustcwhy]

Hi @DelinQu, I also have similar questions. Is IPEC-COMMUNITY/eo1-qwen25_vl-bridge a fine-tuned model of EO-1 or just Qwen2.5-VL-3B without VLA training? I evaluate its performance on SimplerEnv tasks, the results seem to be much lower...