Skip to content

Commit 0747fdc

Browse files
SWHLSWHL
authored
docs: add TensorRT docs
1 parent f418f8f commit 0747fdc

File tree

1 file changed

+59
-9
lines changed

1 file changed

+59
-9
lines changed

‎docs/install_usage/rapidocr/how_to_use_infer_engine.md‎

Lines changed: 59 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -267,14 +267,14 @@ comments: true
267267
3. 查看输出日志。下面日志中打印出了 **Using engine_name: paddle**，则证明使用的推理引擎是 PaddlePaddle。
268268

269269
```bash linenums="1" hl_lines="3 6"
270-
[INFO] 2025-03-22 15:20:45,528 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer/inference.pdmodel
271-
[INFO] 2025-03-22 15:20:45,529 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer/inference.pdiparams
270+
[INFO] 2025-03-22 15:20:45,528 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer/inference.pdmodel
271+
[INFO] 2025-03-22 15:20:45,529 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer/inference.pdiparams
272272
[INFO] 2025-03-22 15:20:45,746 base.py:30: Using engine_name: paddle
273-
[INFO] 2025-03-22 15:20:45,746 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_ppocr_mobile_v2_cls_infer/inference.pdmodel
274-
[INFO] 2025-03-22 15:20:45,746 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_ppocr_mobile_v2_cls_infer/inference.pdiparams
273+
[INFO] 2025-03-22 15:20:45,746 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_ppocr_mobile_v2_cls_infer/inference.pdmodel
274+
[INFO] 2025-03-22 15:20:45,746 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_ppocr_mobile_v2_cls_infer/inference.pdiparams
275275
[INFO] 2025-03-22 15:20:45,903 base.py:30: Using engine_name: paddle
276-
[INFO] 2025-03-22 15:20:45,904 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer/inference.pdmodel
277-
[INFO] 2025-03-22 15:20:45,904 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer/inference.pdiparams
276+
[INFO] 2025-03-22 15:20:45,904 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer/inference.pdmodel
277+
[INFO] 2025-03-22 15:20:45,904 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer/inference.pdiparams
278278
```
279279

280280
### 使用 PyTorch
@@ -363,9 +363,59 @@ comments: true
363363

364364
```bash linenums="1" hl_lines="1 3 5"
365365
[INFO] 2025-03-22 15:39:13,241 base.py:30: Using engine_name: torch
366-
[INFO] 2025-03-22 15:39:13,956 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer.pth
366+
[INFO] 2025-03-22 15:39:13,956 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_infer.pth
367367
[INFO] 2025-03-22 15:39:14,136 base.py:30: Using engine_name: torch
368-
[INFO] 2025-03-22 15:39:14,136 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_ptocr_mobile_v2.0_cls_infer.pth
368+
[INFO] 2025-03-22 15:39:14,136 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_ptocr_mobile_v2.0_cls_infer.pth
369369
[INFO] 2025-03-22 15:39:14,168 base.py:30: Using engine_name: torch
370-
[INFO] 2025-03-22 15:39:14,168 utils.py:35: File already exists in /Users/jiahuawang/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer.pth
370+
[INFO] 2025-03-22 15:39:14,168 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_infer.pth
371+
```
372+
373+
### 使用 TensorRT
374+
375+
1. 安装 TensorRT 运行环境。
376+
377+
我在这里仅给出参考配置环境：
378+
379+
- Docker镜像：[@LocNgoXuan23](https://github.com/LocNgoXuan23) 在 [Discord](https://discord.com/channels/1143707958690189373/1143707958690189376/1468529402118672512) 中给出的镜像：[7.0-gc-triton-devel](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/deepstream?version=7.0-gc-triton-devel)
380+
- 设备配置：8 CPU / 256 GB
381+
- NVIDIA环境：(详细参见：[link](https://gist.github.com/SWHL/0efe902ee469d49fc63d50e297d7fd98) )
382+
- cuda: 12.2
383+
- tensorrt: 8.6.1
384+
- cuda-python: 12.2.0
385+
386+
大家可以根据实际情况，选择安装需要的版本。
387+
388+
2. 指定 TensorRT 作为推理引擎。
389+
390+
运行下面代码，程序会自动现在对应的 ONNX 模型，并转换为 `.engine` 格式。该转换仅在首次运行时执行。因此，首次运行速度会较慢，后面再次运行就快了。
391+
392+
```python linenums="1" hl_lines="3-9"
393+
from rapidocr import EngineType, RapidOCR
394+
395+
engine = RapidOCR(
396+
params={
397+
"Det.engine_type": EngineType.TENSORRT,
398+
"Cls.engine_type": EngineType.TENSORRT,
399+
"Rec.engine_type": EngineType.TENSORRT,
400+
"EngineConfig.tensorrt.use_fp16": False,
401+
"EngineConfig.tensorrt.device_id": 0, # 指定GPU id
402+
}
403+
)
404+
405+
img_url = "https://www.modelscope.cn/models/RapidAI/RapidOCR/resolve/master/resources/test_files/ch_en_num.jpg"
406+
result = engine(img_url)
407+
print(result)
408+
409+
result.vis('vis_result.jpg')
410+
```
411+
412+
3. 查看输出日志。下面日志中打印出了 **Using engine_name: tensorrt**，则证明使用的推理引擎是 PyTorch。
413+
414+
```bash linenums="1" hl_lines="1 3 5"
415+
[INFO] 2025-03-22 15:39:13,241 base.py:30: Using engine_name: tensorrt
416+
[INFO] 2025-03-22 15:39:13,956 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_det_mobile_sm80_fp32.engine
417+
[INFO] 2025-03-22 15:39:14,136 base.py:30: Using engine_name: tensorrt
418+
[INFO] 2025-03-22 15:39:14,136 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_ptocr_mobile_v2.0_cls_sm80_fp32.engine
419+
[INFO] 2025-03-22 15:39:14,168 base.py:30: Using engine_name: tensorrt
420+
[INFO] 2025-03-22 15:39:14,168 utils.py:35: File already exists in /Users/SWHL/projects/RapidOCR/python/rapidocr/models/ch_PP-OCRv4_rec_mobile_sm80_fp32.engine
371421
```

0 commit comments

Comments
 (0)