训练时报错[2024/06/25 14:00:06] root ERROR: DataLoader reader thread raised an exception!

[fx-xiaojia]

[2024/06/25 14:00:06] root ERROR: DataLoader reader thread raised an exception!
Traceback (most recent call last):
File "/opt/PaddleOCR-2.1.1/tools/train.py", line 125, in
Exception in thread Thread-1:
Traceback (most recent call last):
File "/opt/python3/lib/python3.9/threading.py", line 980, in _bootstrap_inner
main(config, device, logger, vdl_writer)
File "/opt/PaddleOCR-2.1.1/tools/train.py", line 100, in main
self.run()
File "/opt/python3/lib/python3.9/threading.py", line 917, in run
program.train(config, train_dataloader, valid_dataloader, device, model,
File "/opt/PaddleOCR-2.1.1/tools/program.py", line 204, in train
self._target(*self._args, **self._kwargs)
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 411, in _thread_loop
for idx, batch in enumerate(train_dataloader):
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 585, in next
batch = self._get_data()
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 525, in get_data
data = self.reader.read_next_var_list()
SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
[Hint: Expected killed != true, but received killed:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:166)

batch.reraise()

File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/worker.py", line 168, in reraise
raise self.exc_type(msg)
TypeError: DataLoader worker(0) caught TypeError with message:
Traceback (most recent call last):
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/worker.py", line 320, in _worker_loop
batch = fetcher.fetch(indices)
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/fetcher.py", line 109, in fetch
data = self.collate_fn(data)
File "/opt/PaddleOCR-2.1.1/venv/lib/python3.9/site-packages/paddle/fluid/dataloader/collate.py", line 79, in default_collate_fn
raise TypeError("batch data con only contains: tensor, numpy.ndarray, "
TypeError: batch data con only contains: tensor, numpy.ndarray, dict, list, number, but got <class 'NoneType'>

[SWHL]

一般是训练集中某个图像损坏导致的。建议先排查数据集问题。