ImageFolder BadZipFile: Bad offset for central directory

[hmartiro]

Describe the bug

I'm getting the following exception:

lib/python3.10/zipfile.py:1353 in _RealGetContents        │
│                                                                                                  │
│   1350 │   │   # self.start_dir:  Position of start of central directory                         │
│   1351 │   │   self.start_dir = offset_cd + concat                                               │
│   1352 │   │   if self.start_dir < 0:                                                            │
│ ❱ 1353 │   │   │   raise BadZipFile("Bad offset for central directory")                          │
│   1354 │   │   fp.seek(self.start_dir, 0)                                                        │
│   1355 │   │   data = fp.read(size_cd)                                                           │
│   1356 │   │   fp = io.BytesIO(data)                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
BadZipFile: Bad offset for central directory                                                                            
Extracting data files:  35%|█████████████████▊                                 | 38572/110812 [00:10<00:20, 3576.26it/s]

Steps to reproduce the bug

load_dataset(
    args.dataset_name,
    args.dataset_config_name,
    cache_dir=args.cache_dir,
),

Expected behavior

loads the dataset

Environment info

datasets==2.8.0
Python 3.10.8
Linux 129-146-3-202 5.15.0-52-generic #58~20.04.1-Ubuntu SMP Thu Oct 13 13:09:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[lhoestq]

Hi ! Could you share the full stack trace ? Which dataset did you try to load ?

it may be related to #5640

[mariosasko]

The BadZipFile error means the ZIP file is corrupted, so I'm closing this issue as it's not directly related to datasets.

[justindujardin]

For others that find this issue following a BadZipFile error, I had the same problem because I had a file in a folder dataset my-image.target and the datasets library was incorrectly determining that the (PNG) file was a zip archive. When it tried to extract the file, this error occurred.

Updating to datasets==2.12.0 fixed the problem for me.