paddleocr的v3识别模型从pdmodel格式转化onnx，然后转换为engine格式报错‘浮点数例外’ #12979

XJPeng12 · 2024-01-02T08:56:19Z

XJPeng12
Jan 2, 2024

您好：我在尝试将paddleocr推理模型转换为onnx格式，然后使用tensorrt转换为engine格式的时候，识别模型报错‘浮点数例外’，检测模型和方向分类模型可以正常转换并使用，具体问题如下所示。希望可以帮我看一下是哪里的问题，非常感谢！

系统环境/System Environment：
在centos7.6的系统中GPU为2080ti，驱动为NVIDIA-SMI 470.57.02 ,Driver Version: 470.57.02,CUDA Version: 11.4。
cuda=11.4,cudnn==8.2.2,tensorrt==8.2.0.6，python=3.8.16
paddle2onnx=1.1.0，
paddleocr =2.7.0.3
paddlepaddle-gpu=2.6.0
onnx=1.15.0
onnx-simplifier=0.4.35
onnxruntime=1.9.0
运行指令/Command Code：
# 执行报错
/usr/src/tensorrt/bin/trtexec --onnx=./inference/det_onnx/model.onnx --saveEngine=./inference/det_onnx/model.engine

使用以下命令将pdmodel格式转换为onnx

paddle2onnx --model_dir ./inference/ch_PP-OCRv3_rec_infer
--model_filename inference.pdmodel
--params_filename inference.pdiparams
--save_file ./inference/rec_onnx/model.onnx
--opset_version 10
--input_shape_dict="{'x':[-1,3,-1,-1]}"
--enable_onnx_checker True

完整报错/Complete Error Message：

(paddleocr) [root@localhost v3-cuda11.1]# /usr/src/tensorrt/bin/trtexec --onnx=./inference/rec_onnx/model.onnx --saveEngine=./inference/rec_onnx/model.engine

&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=./inference/rec_onnx/model.onnx --saveEngine=./inference/rec_onnx/model.engine
[01/02/2024-16:48:59] [I] === Model Options ===
[01/02/2024-16:48:59] [I] Format: ONNX
[01/02/2024-16:48:59] [I] Model: ./inference/rec_onnx/model.onnx
[01/02/2024-16:48:59] [I] Output:
[01/02/2024-16:48:59] [I] === Build Options ===
[01/02/2024-16:48:59] [I] Max batch: explicit
[01/02/2024-16:48:59] [I] Workspace: 16 MiB
[01/02/2024-16:48:59] [I] minTiming: 1
[01/02/2024-16:48:59] [I] avgTiming: 8
[01/02/2024-16:48:59] [I] Precision: FP32
[01/02/2024-16:48:59] [I] Calibration:
[01/02/2024-16:48:59] [I] Refit: Disabled
[01/02/2024-16:48:59] [I] Safe mode: Disabled
[01/02/2024-16:48:59] [I] Save engine: ./inference/rec_onnx/model.engine
[01/02/2024-16:48:59] [I] Load engine:
[01/02/2024-16:48:59] [I] Builder Cache: Enabled
[01/02/2024-16:48:59] [I] NVTX verbosity: 0
[01/02/2024-16:48:59] [I] Tactic sources: Using default tactic sources
[01/02/2024-16:48:59] [I] Input(s)s format: fp32:CHW
[01/02/2024-16:48:59] [I] Output(s)s format: fp32:CHW
[01/02/2024-16:48:59] [I] Input build shapes: model
[01/02/2024-16:48:59] [I] Input calibration shapes: model
[01/02/2024-16:48:59] [I] === System Options ===
[01/02/2024-16:48:59] [I] Device: 0
[01/02/2024-16:48:59] [I] DLACore:
[01/02/2024-16:48:59] [I] Plugins:
[01/02/2024-16:48:59] [I] === Inference Options ===
[01/02/2024-16:48:59] [I] Batch: Explicit
[01/02/2024-16:48:59] [I] Input inference shapes: model
[01/02/2024-16:48:59] [I] Iterations: 10
[01/02/2024-16:48:59] [I] Duration: 3s (+ 200ms warm up)
[01/02/2024-16:48:59] [I] Sleep time: 0ms
[01/02/2024-16:48:59] [I] Streams: 1
[01/02/2024-16:48:59] [I] ExposeDMA: Disabled
[01/02/2024-16:48:59] [I] Data transfers: Enabled
[01/02/2024-16:48:59] [I] Spin-wait: Disabled
[01/02/2024-16:48:59] [I] Multithreading: Disabled
[01/02/2024-16:48:59] [I] CUDA Graph: Disabled
[01/02/2024-16:48:59] [I] Separate profiling: Disabled
[01/02/2024-16:48:59] [I] Skip inference: Disabled
[01/02/2024-16:48:59] [I] Inputs:
[01/02/2024-16:48:59] [I] === Reporting Options ===
[01/02/2024-16:48:59] [I] Verbose: Disabled
[01/02/2024-16:48:59] [I] Averages: 10 inferences
[01/02/2024-16:48:59] [I] Percentile: 99
[01/02/2024-16:48:59] [I] Dump refittable layers:Disabled
[01/02/2024-16:48:59] [I] Dump output: Disabled
[01/02/2024-16:48:59] [I] Profile: Disabled
[01/02/2024-16:48:59] [I] Export timing to JSON file:
[01/02/2024-16:48:59] [I] Export output to JSON file:
[01/02/2024-16:48:59] [I] Export profile to JSON file:
[01/02/2024-16:48:59] [I]
[01/02/2024-16:49:00] [I] === Device Information ===
[01/02/2024-16:49:00] [I] Selected Device: NVIDIA GeForce RTX 2080 SUPER
[01/02/2024-16:49:00] [I] Compute Capability: 7.5
[01/02/2024-16:49:00] [I] SMs: 48
[01/02/2024-16:49:00] [I] Compute Clock Rate: 1.815 GHz
[01/02/2024-16:49:00] [I] Device Global Memory: 7982 MiB
[01/02/2024-16:49:00] [I] Shared Memory per SM: 64 KiB
[01/02/2024-16:49:00] [I] Memory Bus Width: 256 bits (ECC disabled)
[01/02/2024-16:49:00] [I] Memory Clock Rate: 7.751 GHz
[01/02/2024-16:49:00] [I]

Input filename: ./inference/rec_onnx/model.onnx
ONNX IR version: 0.0.8
Opset version: 10
Producer name:
Producer version:
Domain:
Model version: 0
Doc string:

[01/02/2024-16:49:01] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/02/2024-16:49:01] [W] Dynamic dimensions required for input: x, but no shapes were provided. Automatically overriding shape to: 1x3x48x1
[01/02/2024-16:49:01] [W] [TRT] TensorRT was linked against cuDNN 8.1.0 but loaded cuDNN 8.0.5
[01/02/2024-16:49:12] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
浮点数例外

总结：
1、检测模型和方向分类模型可以转换为onnx格式，然后转化为engine格式，并进行推理
2 、尝试过v4，v2版本的识别模型都可以转换为onnx，同样转换为engine格式的时候报错，并且与v3版本的报错不一致。
3、尝试过降低降低cuda版本为nvidia=11.4，cuda=11.1，cudnn=8.0.5.39，tensorrt=7.2.3.4，同样报错‘浮点数例外’

XJPeng12 · 2024-01-03T01:09:51Z

XJPeng12
Jan 3, 2024
Author

详细的部分报错日志：
[01/03/2024-09:20:05] [V] [TRT]
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(512,1536,1,1536) -> Float(512,1536,1,1536) ***************
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: p2o.Conv.30 (FusedConvActConvolution)
[01/03/2024-09:20:05] [V] [TRT] FusedConvActConvolution has no valid tactics for this config, skipping
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: p2o.Conv.30 (CaskConvolution)
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: 861694390046228376 time 0.08912
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: 5258189349241541167 time 0.050408
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: 5821621277990374316 time 0.088916
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: 5863767799113001648 time 0.030348
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -9147980667639709536 time 0.088436
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -8892196987859366827 time 0.08884
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -8850904373104590857 time 0.049988
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -8010679767156598961 time 0.029596
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -7751035352149795660 time 0.088892
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -5115676123557684531 time 0.088608
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -493597327599791285 time 0.048896
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] Tactic: -423878181466897819 time 0.030508
[01/03/2024-09:20:05] [V] [TRT] Fastest Tactic: -8010679767156598961 Time: 0.029596
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: p2o.Conv.30 (CudaConvolution)
[01/03/2024-09:20:05] [V] [TRT] CudaConvolution has no valid tactics for this config, skipping
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: p2o.Conv.30 (CudaDepthwiseConvolution)
[01/03/2024-09:20:05] [V] [TRT] CudaDepthwiseConvolution has no valid tactics for this config, skipping
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: p2o.Conv.30 (CublasConvolution)
[01/03/2024-09:20:05] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping
[01/03/2024-09:20:05] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -8010679767156598961
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT]
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_ldg4_relu_exp_small_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x128_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x64_sliced1x2_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_medium_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] p2o.Conv.30 (scudnn) Set Tactic Name: volta_scudnn_128x32_sliced1x4_ldg4_relu_exp_interior_nhwc_tn_v1
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(1,3,3,1536) -> Float(1,1,3,1536) ***************
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: shuffle_after_batch_norm_53.tmp_3 (Shuffle)
[01/03/2024-09:20:05] [V] [TRT] Tactic: 0 time 0.003764
[01/03/2024-09:20:05] [V] [TRT] Tactic: 1 time 0.008384
[01/03/2024-09:20:05] [V] [TRT] Fastest Tactic: 0 Time: 0.003764
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(512,1536,1,1536) -> Float(512,512,1,1536) ***************
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: shuffle_after_batch_norm_53.tmp_3 (Shuffle)
[01/03/2024-09:20:05] [V] [TRT] Tactic: 0 time 0.004112
[01/03/2024-09:20:05] [V] [TRT] Tactic: 1 time 0.00794
[01/03/2024-09:20:05] [V] [TRT] Fastest Tactic: 0 Time: 0.004112
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(1,3,3:32,48) -> Float(1,1,3:32,48) ***************
[01/03/2024-09:20:05] [V] [TRT] --------------- Timing Runner: shuffle_after_batch_norm_53.tmp_3 (Shuffle)
[01/03/2024-09:20:05] [V] [TRT] Tactic: 0 time 0.003792
[01/03/2024-09:20:05] [V] [TRT] Tactic: 1 time 0.013212
[01/03/2024-09:20:05] [V] [TRT] Fastest Tactic: 0 Time: 0.003792
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(1,1,3,1536) -> Float(1,1,3,1536) ***************
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(512,512,1,1536) -> Float(512,512,1,1536) ***************
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(1,1,3:32,48) -> Float(1,1,3:32,48) ***************
[01/03/2024-09:20:05] [V] [TRT] *************** Autotuning format combination: Float(1,1,3,1536) -> Float(1,0,0,0) ***************
浮点数例外

0 replies

james9203 · 2024-01-05T06:08:28Z

james9203
Jan 5, 2024

解决了吗？我也遇到了

0 replies

LeoRainly · 2024-02-06T08:07:52Z

LeoRainly
Feb 6, 2024

Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
我想知道paddle2onnx如何导出int32的onnx

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

paddleocr的v3识别模型从pdmodel格式转化onnx，然后转换为engine格式报错‘浮点数例外’ #12979

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

paddleocr的v3识别模型从pdmodel格式转化onnx，然后转换为engine格式报错‘浮点数例外’ #12979

Uh oh!

XJPeng12 Jan 2, 2024

使用以下命令将pdmodel格式转换为onnx

Input filename: ./inference/rec_onnx/model.onnx ONNX IR version: 0.0.8 Opset version: 10 Producer name: Producer version: Domain: Model version: 0 Doc string:

Replies: 3 comments

Uh oh!

XJPeng12 Jan 3, 2024 Author

Uh oh!

james9203 Jan 5, 2024

Uh oh!

LeoRainly Feb 6, 2024

XJPeng12
Jan 2, 2024

Input filename: ./inference/rec_onnx/model.onnx
ONNX IR version: 0.0.8
Opset version: 10
Producer name:
Producer version:
Domain:
Model version: 0
Doc string:

XJPeng12
Jan 3, 2024
Author

james9203
Jan 5, 2024

LeoRainly
Feb 6, 2024