相同模型在不同版本cuda下预测结果不一样

[gaowei0518]

请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem
在不同服务器使用不同cuda版本编译的paddlegpu运行文字检测时结果相差极大，模型使用的均为同一通用模型，字典也没变化

• 系统环境/System Environment：
  • linux/ubuntu20.04
  • cuda11.3/cudnn 8.20(NVIDIA RTX A6000) , cuda10.1/cudnn7.65(NVIDIA GeForce RTX 2080 Ti)
  • python3.8
• 版本号/Version：
  • Paddle：paddlepaddle-gpu 2.3.2.post112/2.3.2.post101
  • PaddleOCR：2.4.0.4
  • 问题相关组件/Related components：ch_ppocr_server_v2.0_rec_infer
• 运行指令/Command Code：

>>> from paddleocr import PaddleOCR
>>> import os,cv2
>>> one_file = "test.jpg"
>>> img =  cv2.imread(one_file)
>>> print(img.shape)
(55, 208, 3)
>>> print(img[0:10,0,0])
[159 149 183 213 191 162 157 161 161 165]
>>> ocr = PaddleOCR(rec_model_dir=os.environ["HOME"] + "/workarea/git/PaddleOCR/ch_ppocr_server_v2.0_rec_infer")
>>> results = ocr.ocr(img, det=False)
>>> print(results)
[('商', 0.22717525)]                  ( 不同预测结果 with cuda10.1 [('DP-485RR', 0.9526073)]  )

• 完整报错/Complete Error Message：

我们提供了AceIssueSolver来帮助你解答问题，你是否想要它来解答(请填写yes/no)?/We provide AceIssueSolver to solve issues, do you want it? (Please write yes/no): no

请尽量不要包含图片在问题中/Please try to not include the image in the issue.

[gaowei0518]

预测用图片 ：

模型加载时超参数 ：

• cuda11.3

ocr = PaddleOCR(rec_model_dir=os.environ["HOME"] + "/workarea/git/PaddleOCR/ch_ppocr_server_v2.0_rec_infer")
WARNING 2023-10-18 15:57:14,815 paddleocr.py:267] version PP-OCRv2 not support cls models, auto switch to version PP-OCR
[2023/10/18 15:57:14] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/datou/.paddleocr/2.4/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/datou/.paddleocr/2.4/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, is_visualize=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/datou/workarea/git/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/datou/workarea/git/PaddleOCR/ch_ppocr_server_v2.0_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=True, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, use_xpu=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
results = ocr.ocr(img, det=False)

预测结果均为单一中文字 [('商', 0.22717525)]

• cuda 10.1

ocr = PaddleOCR(rec_model_dir=os.environ["HOME"] + "/workarea/git/PaddleOCR/ch_ppocr_server_v2.0_rec_infer")
WARNING 2023-10-18 15:57:25,655 paddleocr.py:267] version PP-OCRv2 not support cls models, auto switch to version PP-OCR
[2023/10/18 15:57:25] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR
Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/admin/.paddleocr/2.4/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/admin/.paddleocr/2.4/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, is_visualize=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/admin/workarea/git/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/admin/workarea/git/PaddleOCR/ch_ppocr_server_v2.0_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=True, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, use_xpu=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)
results = ocr.ocr(img, det=False)

预测结果为准确车牌号 ：[('DP-485RR', 0.9526073)]

经过仔细比对各种参数没有任何变化，由于在不同服务器所以路径有区别但使用的模型均是同模型

个人不觉得cuda/cudnn版本不会带来如此大的差别，
由于使用不同服务器，所以有没有可能哪个额外的系统环境变量/python库会大幅影响预测结果？有没有大佬能给出点可能性？

[SWHL]

猜测更多的是Paddle框架导致的