Eval Text Recognition Model

[tuan-vmt]

I get an error when running to eval Text Recognition model step. Can you help me? Thank you.

[tuan-vmt]

Is it due to version difference: Driver API Version: 12.3, Runtime API Version: 11.6?

[tuan-vmt]

When I run: python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_resnet_stn_bilstm_att.yml -o Global.checkpoints=pretrain_models/rec_resnet_stn_bilstm_att/best_accuracy also get the same error as above .Please help me :(((:+1: File "tools/eval.py", line 148, in
main()
File "tools/eval.py", line 138, in main
metric = program.eval(model, valid_dataloader, post_process_class,
File "/point/tuanvm/PaddleOCR/tools/program.py", line 524, in eval
preds = model(images, data=batch[1:])
File "/point/anaconda3/envs/paddle-tuanvm/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/point/tuanvm/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 100, in forward
x = self.head(x, targets=data)
File "/point/anaconda3/envs/paddle-tuanvm/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/point/tuanvm/PaddleOCR/ppocr/modeling/heads/rec_aster_head.py", line 63, in forward
rec_pred, rec_pred_scores = self.decoder.beam_search(
File "/point/tuanvm/PaddleOCR/ppocr/modeling/heads/rec_aster_head.py", line 223, in beam_search
mask = paddle.nonzero(mask)
File "/point/anaconda3/envs/paddle-tuanvm/lib/python3.8/site-packages/paddle/tensor/search.py", line 463, in nonzero
outs = _C_ops.nonzero(x)
OSError: (External) CUDA error(700), an illegal memory access was encountered.
[Hint: 'cudaErrorIllegalAddress'. The device encountered a load or store instruction on an invalid memory address. This leaves the process in an inconsistentstate and any further CUDA work will return the same error. To continue using CUDA, the process must be terminated and relaunched. ] (at ../paddle/phi/backends/gpu/cuda/cuda_info.cc:265)

eval model:: 0%| | 2/2077 [00:01<29:03, 1.19it/s]

[tink2123]

Please provide the paddle version used for training. It is recommended to use paddlepaddle==2.5.2

[tuan-vmt]

Please provide the paddle version used for training. It is recommended to use paddlepaddle==2.5.2

Thank you for your reply. I tried installing paddlepaddle-gpu version 2.5.2.post116 but this error still occurs. Is it due to the Ubuntu 22.04 operating system?

[tuan-vmt]

I can still run it with CPU but not GPU

[tink2123]

CUDA11.6 may cause problems during training. Is it convenient to install cuda11.7 version of paddle?

python -m pip install paddlepaddle-gpu==2.6.0.post117 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

[tink2123]

It is highly recommended to use Docker installation, which can solve most environmental problems.

nvidia-docker pull registry.baidubce.com/paddlepaddle/paddle:2.6.0-gpu-cuda11.7-cudnn8.4-trt8.4

nvidia-docker run --name paddle -it -v $PWD:/paddle registry.baidubce.com/paddlepaddle/paddle:2.6.0-gpu-cuda11.7-cudnn8.4-trt8.4 /bin/bash