DREAM-1K性能无法复现

[WeihuangLin]

感谢你的工作！

请问是否能提供推理和测试脚本关于Qwen2VL 7B（baseline）在Dream-1K性能是如何评测的？无法复现文中提供的性能。

[Kerin637]

Thank you for your interest and for raising this question!
For Qwen2-VL-7B on DREAM-1K, we used the prompt: "Describe the video in detail." The evaluation strictly followed the official Tarsier2 pipeline (https://github.com/bytedance/tarsier), using GPT-3.5-turbo-0125 for AutoDQ scoring. Each video was uniformly sampled into 64 frames, with a per-frame pixel limit of 100,352. Slight variations in frame count or resolution may lead to minor performance differences, but the impact is generally negligible. A comparable result is reported for Qwen2-VL-7B in the Tarsier2 Technical Report (https://arxiv.org/abs/2501.07888, Table 1).