Search results

[Data] Discrepancy in RefCOCO dataset sizes compared to original splits

Status: Open.

#1066 In EvolvingLMMs-Lab/lmms-eval;

· PeterWangyi opened on Feb 5, 2026

Issues about MMVP Dataset

Status: Open.

#1018 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Jan 22, 2026

When I tested Qwen-2.5-vl-7B on the Videomme dataset, ValueErrorr: The decoder prompt (length 163270) is longer

Status: Open.

#988 In EvolvingLMMs-Lab/lmms-eval;

· LyuQi127 opened on Jan 12, 2026

It costs a long time to test on the DocVQA

Status: Open.

#933 In EvolvingLMMs-Lab/lmms-eval;

· LyuQi127 opened on Dec 13, 2025

The test results for Qwen3VL show significant discrepancies compared to the official report.

Status: Open.

#932 In EvolvingLMMs-Lab/lmms-eval;

· Fu-Fu-Fu-Fu opened on Dec 13, 2025

MVBench dataset average result

Status: Open.

#925 In EvolvingLMMs-Lab/lmms-eval;

· ylllllll opened on Dec 8, 2025

用vllm和不用vllm结果差的很多

Status: Open.

#921 In EvolvingLMMs-Lab/lmms-eval;

· Fu-Fu-Fu-Fu opened on Dec 3, 2025

The LLM Judge function takes too long, resulting in low GPU utilization.

Status: Open.

#908 In EvolvingLMMs-Lab/lmms-eval;

· Fu-Fu-Fu-Fu opened on Nov 20, 2025

Error while runnig lava-onevision-qwen2-7b example

Status: Open.

#860 In EvolvingLMMs-Lab/lmms-eval;

· gabrielsantosrv opened on Oct 17, 2025

Padding Side Option Missing in Qwen2_5_VL

Status: Open.

#830 In EvolvingLMMs-Lab/lmms-eval;

· ThakurRajAnand opened on Sep 18, 2025

Progress bar overflowed

Status: Open.

#800 In EvolvingLMMs-Lab/lmms-eval;

· futrime opened on Aug 21, 2025

Abnormal GPU Utilization When Evaluating Qwen 2.5 VL 72B

Status: Open.

#738 In EvolvingLMMs-Lab/lmms-eval;

· LiamLian0727 opened on Jul 6, 2025