No Prediction when use_gpu=True

[sminkwitz]

Hello when i try to run paddleocr on an image with use_gpu=True there is no result. If i run it without gpu use i get the right result.

I am on a Jetson Orin Nano with Jetpack 5.1.2.

Here the Logs with use_gpu=False:

[2023/08/29 14:43:38] ppocr DEBUG: Namespace(alpha=1.0, benchmark=False, beta=1.0, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/multi/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_box_type='quad', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/multi/.paddleocr/whl/det/en/en_PP-OCRv3_det_infer', det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, fourier_degree=5, gpu_id=0, gpu_mem=500, help='==SUPPRESS==', image_dir=None, image_orientation=False, ir_optim=True, kie_algorithm='LayoutXLM', label_list=['0', '180'], lang='en', layout=True, layout_dict_path=None, layout_model_dir=None, layout_nms_threshold=0.5, layout_score_threshold=0.5, max_batch_size=10, max_text_length=25, merge_no_span_structure=True, min_subgraph_size=15, mode='structure', ocr=True, ocr_order_method=None, ocr_version='PP-OCRv4', output='./output', page_num=0, precision='fp32', process_id=0, re_model_dir=None, rec=True, rec_algorithm='SVTR_LCNet', rec_batch_num=6, rec_char_dict_path='/home/multi/installVSCode/paddle/lib/python3.8/site-packages/paddleocr/ppocr/utils/en_dict.txt', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_model_dir='/home/multi/.paddleocr/whl/rec/en/en_PP-OCRv4_rec_infer', recovery=False, return_word_box=False, save_crop_res=False, save_log_path='./log_output/', scales=[8, 16, 32], ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ser_model_dir=None, show_log=True, sr_batch_num=1, sr_image_shape='3, 32, 128', sr_model_dir=None, structure_version='PP-StructureV2', table=True, table_algorithm='TableAttn', table_char_dict_path=None, table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=False, use_mp=False, use_npu=False, use_onnx=False, use_pdf2docx_api=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, use_visual_backbone=True, use_xpu=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
[2023/08/29 14:43:40] ppocr DEBUG: dt_boxes num : 2, elapse : 0.09264564514160156
[2023/08/29 14:43:40] ppocr DEBUG: cls num : 2, elapse : 0.03076481819152832
[2023/08/29 14:43:40] ppocr DEBUG: rec_res num : 2, elapse : 0.4024841785430908
[[[[[34.0, 33.0], [194.0, 42.0], [190.0, 113.0], [30.0, 103.0]], ('PURE', 0.9276974201202393)], [[[35.0, 119.0], [191.0, 123.0], [190.0, 193.0], [34.0, 189.0]], ('TEXT', 0.9927603006362915)]]]

Here the Logs with use_gpu=True:

2023/08/29 14:44:30] ppocr DEBUG: Namespace(alpha=1.0, benchmark=False, beta=1.0, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/multi/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_box_type='quad', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/multi/.paddleocr/whl/det/en/en_PP-OCRv3_det_infer', det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, fourier_degree=5, gpu_id=0, gpu_mem=500, help='==SUPPRESS==', image_dir=None, image_orientation=False, ir_optim=True, kie_algorithm='LayoutXLM', label_list=['0', '180'], lang='en', layout=True, layout_dict_path=None, layout_model_dir=None, layout_nms_threshold=0.5, layout_score_threshold=0.5, max_batch_size=10, max_text_length=25, merge_no_span_structure=True, min_subgraph_size=15, mode='structure', ocr=True, ocr_order_method=None, ocr_version='PP-OCRv4', output='./output', page_num=0, precision='fp32', process_id=0, re_model_dir=None, rec=True, rec_algorithm='SVTR_LCNet', rec_batch_num=6, rec_char_dict_path='/home/multi/installVSCode/paddle/lib/python3.8/site-packages/paddleocr/ppocr/utils/en_dict.txt', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_model_dir='/home/multi/.paddleocr/whl/rec/en/en_PP-OCRv4_rec_infer', recovery=False, return_word_box=False, save_crop_res=False, save_log_path='./log_output/', scales=[8, 16, 32], ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ser_model_dir=None, show_log=True, sr_batch_num=1, sr_image_shape='3, 32, 128', sr_model_dir=None, structure_version='PP-StructureV2', table=True, table_algorithm='TableAttn', table_char_dict_path=None, table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=True, use_mp=False, use_npu=False, use_onnx=False, use_pdf2docx_api=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, use_visual_backbone=True, use_xpu=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
[2023/08/29 14:44:39] ppocr DEBUG: dt_boxes num : 0, elapse : 4.481969833374023
[2023/08/29 14:44:39] ppocr DEBUG: cls num : 0, elapse : 0
[2023/08/29 14:44:39] ppocr DEBUG: rec_res num : 0, elapse : 2.86102294921875e-06
[[]]

[alanxinn]

It may be that the cuda and cudnn versions of the gpu are wrong，i think

[sminkwitz]

thank you for your awnser. I already thought the same... . So maybe i could slove it when i convert the models to onnx and run the models via onnxruntime? Is there any los of performance when i try this? Or do you have any other work around?

[alanxinn]

Can you send the version of cuda, dudnn,gpu and paddle of gpu to have a look?
I encountered garbled characters before, because the cuda and cudnn versions are wrong.

[sminkwitz]

Cuda= 11.4.315 and cuDNN= 8.6.0.166 so what can i do? :D

[alanxinn]

I am garbled, graphics card 1060 cuda12.0 cudnn8.8, gpu reasoning is garbled, cpu reasoning is normal, try a few more versions
Win10 Professional Edition 22H2 paddlepaddle-gpu==2.3.2 cuda==10.1 cudnn==7.6.5 paddleocr==2.6.0 Successful reasoning, no garbled characters

your graphics card name？

[sminkwitz]

Hey i am on a Nvidia Jetson Orin Nano. I use paddlepaddle-gpu=2.4.1 and paddleocr=2.7.0.2

[alanxinn]

i can't help you sry, i never use this card

[sminkwitz]

okay thank you. maybe someone else can help?

[hungtrieu07]

You can try diferent version on page https://www.paddlepaddle.org.cn/en/install/quick?docurl=/documentation/docs/en/install/pip/linux-pip_en.html

[yaibaiu]

I think GPU is Pascal architecture. You can try install follow link https://www.paddlepaddle.org.cn/documentation/docs/en/install/Tables_en.html

[xReniar]

@yaibaiu i tried to install linux-cuda11.2-cudnn8-mkl-gcc8.2-avx-pascal by this command:

pip3 install paddlepaddle_gpu-2.6.0.post112-cp38-cp38-linux_x86_64.whl

but it says:
PreconditionNotMetError: Cannot load cudnn shared library. Cannot invoke method cudnnGetVersion. [Hint: cudnn_dso_handle should not be null.] (at ../paddle/phi/backends/dynload/cudnn.cc:64) [operator < fill_constant > error]

I'm using a conda environment should i install something else?