Meet assertion error assert loss <= 1 when running tools/train.py

[junyizhao04]

打算测试的时候先用一下demo的ICDAR数据集和测试代码，结果直接报错。。。
Traceback (most recent call last):
File "tools/train.py", line 191, in
main(config, device, logger, vdl_writer)
File "tools/train.py", line 164, in main
program.train(config, train_dataloader, valid_dataloader, device, model,
File "/home/楼主电脑/PaddleOCR/tools/program.py", line 268, in train
loss = loss_class(preds, batch)
File "/home/楼主电脑/.local/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/home/楼主电脑/.local/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/home/楼主电脑/PaddleOCR/ppocr/losses/det_db_loss.py", line 65, in forward
loss_binary_maps = self.dice_loss(binary_maps, label_shrink_map,
File "/home/楼主电脑/.local/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/home/楼主电脑/.local/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/home/楼主电脑/PaddleOCR/ppocr/losses/det_basic_loss.py", line 128, in forward
assert loss <= 1
AssertionError
这是什么情况？如何处理？

[WenmuZhou]

Does it appear every time？

[junyizhao04]

yes - I tried different conda environments and the problem persists.

[4dv3ntur3]

did you solve it? I had same problem and trying docker environment worked.

[Pawel1B]

Anyone has solution that can be done strictly on windows without docker involved?

[Pawel1B]

with cuDNN v8.2.1?
also check if this version works
python -m pip install paddlepaddle-gpu==2.5.1.post112 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html

[xReniar]

Yes I am using cuDNN v8.2.1, I will try the new installation

Sorry to ask but is the command for linux or for windows?

[Pawel1B]

if u have pip just write without python -m (it is probably windows)

[xReniar]

Still does not work

[Pawel1B]

try pulling whole thing from github, put your coding file there and if that does not work idk how to help
there if some package is giving you tuble fix it by pytting other version and so on.