Data download incomplete

[mrpeerat]

Hi,

I ran the data download script from https://github.com/TIGER-AI-Lab/VLM2Vec/blob/main/experiments/public/data/download_data.sh

There is no error, but when I checked the data, only half of them were downloaded.

So, when I ran ls -1 data/vlm2vec_train/MMEB-train/images/DocVQA/Train/ | wc -l, The data is available only 13.9k samples, while the data should have around 30k++.

I already installed Git LFS.

Is there anything I need to do to get the full data?

Thank you!

[XMHZZ2018]

@mrpeerat

Hi, I just downloaded it on my side and the file size (39K) looks correct to me.
Let's take DocVQA as an example. If you run the download script, it will download the data from here:
https://huggingface.co/datasets/TIGER-Lab/MMEB-train/blob/main/images_zip/DocVQA.zip

(vlm2vec) ziyan@darth:/data/ziyan/vlm2vec_eval/DocVQA/Train$ ls | wc -l 39463