PaddleX/latest/pipeline_deploy/high_performance_inference #2698
giscus[bot]
bot
Replies: 13 comments 25 replies
-
|
有没有NPU的高性能支持，比如昇腾910b的，目前测试并发高的情况下，响应速度急剧下降，目前看是多进程对NPU资源的竞争性使用导致的。 |
Beta Was this translation helpful? Give feedback.
-
|
遇到了同样的问题，请问解决了吗 |
Beta Was this translation helpful? Give feedback.
-
|
不用了，换回GPU了，目前看910b适合部署大模型，小模型多并发性能不行 |
Beta Was this translation helpful? Give feedback.
-
|
您好，可否提供一下在npu上跑的模型和并发配置，我们想复现一下，看看能不能优化这块 |
Beta Was this translation helpful? Give feedback.
-
|
NPU的教程可以参考这里：https://paddlepaddle.github.io/PaddleX/latest/practical_tutorials/high_performance_npu_tutorial.html |
Beta Was this translation helpful? Give feedback.
-
|
请问为什么pipeline中指定yaml文件的时候会报如下错误？ Traceback (most recent call last):
File "D:\TestProject\pythonProject3\PaddleX\test.py", line 3, in <module>
pipeline = create_pipeline(
^^^^^^^^^^^^^^^^
File "D:\TestProject\pythonProject3\PaddleX\paddlex\inference\pipelines\__init__.py", line 119, in create_pipeline
return create_pipeline_from_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\TestProject\pythonProject3\PaddleX\paddlex\inference\pipelines\__init__.py", line 70, in create_pipeline_from_config
pipeline_name = config["Global"]["pipeline_name"]
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'pipeline_name' |
Beta Was this translation helpful? Give feedback.
-
|
It looks like you’re trying to use the config module inside the pipeline configuration. A good approach would be to copy one of the existing pipeline configs from .venv/lib/python3.xx/site-packages/paddlex/configs/pipelines and then modify the module_name accordingly. You might also find this tutorial helpful: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot! i will try it. |
Beta Was this translation helpful? Give feedback.
-
|
搞得太复杂了，产品经理脑子有坑吗，这都啥呀 原来的套件都不维护了，搞这种大一统的东西，搞得不伦不类。我就想在windows项目中集成使用dll，好家伙，要搞docker了现在，搞成客户端和服务端通信模式了，佩服 我几百台电脑部署都给人家客户先装个docker吗？ |
Beta Was this translation helpful? Give feedback.
-
|
你好，抱歉产品的不完善给你带来了不便，也感谢你指出这些问题！作为PaddleX一个普通的研发，下面是我对这些问题的一些看法：
诚然，PaddleX目前不太完善，但我们希望可以做得更好。罗马不是一日建成的，一个产品常常需要不断地根据用户反馈增加新的功能，这可能是一个相对长期的过程。也希望你能给我们一些耐心，多给我们一些反馈（比如这次提到的），我相信我们能不断打磨这个产品，做到更好。 |
Beta Was this translation helpful? Give feedback.
-
|
高性能推理插件支持在windows11系统上安装使用吗？ |
Beta Was this translation helpful? Give feedback.
-
|
对于PaddleX 3.0.0rc1版本，可以使用WSL或者Docker，暂时没有原生Windows支持 |
Beta Was this translation helpful? Give feedback.
-
|
您好，想了解一下后续有类似FastDeploy 之类的方案更进吗，因为我用再工业领域，部署的时候真的希望开箱即用，类似的这样的部署非常不便，而且对于硬件很极限,常部署在intel核显上。连4060都舍不得上。 |
Beta Was this translation helpful? Give feedback.
-
|
请问你指的是C++本地推理嘛？ |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
目前C++本地推理能力还没有建设，感谢你的建议，我们会讨论这些需求，并且可能在之后的版本推出相关解决方案～ |
Beta Was this translation helpful? Give feedback.
-
|
高性能能推理支持的模型列表在哪里？ |
Beta Was this translation helpful? Give feedback.
-
|
paddlex升级到 3.0.0-rc1 版本后，理论上PaddleX支持的所有使用静态图格式的模型都支持高性能推理，但根据实际情况，有可能开了高性能推理之后无法加速。可以参考FAQ中的说明：
|
Beta Was this translation helpful? Give feedback.
-
|
目前 PaddleX 官方仅提供 CUDA 11.8 + cuDNN 8.9 的预编译包。CUDA 12 已经在支持中。 预计什么时候发布？ |
Beta Was this translation helpful? Give feedback.
-
|
目前develop分支已经支持，预计在8月中旬的3.2版本中会体现，如果着急使用的话也可以先尝试安装develop分支的paddlex |
Beta Was this translation helpful? Give feedback.
-
|
使用高性能推理出现下面ERROR，推理结果能正常输出，paddlex3.1+paddlepaddle3.0.0 |
Beta Was this translation helpful? Give feedback.
-
|
这是底层框架的一些bug导致的，如果这个问题不影响正常推理流程的话，可以忽略～我们在文档的FAQ部分也做了一些解释：https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/high_performance_inference.html#3 |
Beta Was this translation helpful? Give feedback.
-
|
你好，我只是想把paddleOCR 部署到上位机软件里，用来做零件上文字识别，产品生产周期很短，，我们需要在极短的时间内完成OCR模型的本地推理，考虑使用PaddleOCRV5, 转成ONNX模型，最后再转成tensorRT模型， 使用tensorRT对OCR模型推理加速，不使用docker和服务的形式, 只是转tensorRT后封装一个调用tensorRT OCR模型的库（python语言开发），以上本地使用python调用tensorRT ocr模型能做到吗，这个PaddleOCR V5 是不是可以转tensorRT(在不依赖你们的高性能推理部署框架的情况下)？ |
Beta Was this translation helpful? Give feedback.
-
|
以上需求可以通过PaddleX高性能推理插件实现，这种方式可以不使用docker，也无需服务化。 |
Beta Was this translation helpful? Give feedback.
-
|
windows10 可以不使用Docker部署吗？ |
Beta Was this translation helpful? Give feedback.
-
|
对于Windows系统，如果不使用Docker，也可以考虑在WSL中执行，不过目前确实不支持原生Windows |
Beta Was this translation helpful? Give feedback.
-
|
高性能推理+高稳定性服务化推理 可以结合起来部署吗？ 用哪个容器镜像？ |
Beta Was this translation helpful? Give feedback.
-
|
可以，用高稳定性服务化部署的镜像，设置 |
Beta Was this translation helpful? Give feedback.
-
|
用高稳定性服务化部署，设置PADDLEX_HPS_USE_HPIP环境变量开关为1 后，出现如下错误。 ==========
|
Beta Was this translation helpful? Give feedback.
-
|
请问使用的是最新版本的镜像吗？ |
Beta Was this translation helpful? Give feedback.
-
|
是的，就是高稳定服务部署文档提到的镜像。 docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.1-gpu |
Beta Was this translation helpful? Give feedback.
-
|
好像是我弄错了。 |
Beta Was this translation helpful? Give feedback.
-
|
我很好奇，连最简单的通用图像分类网络，都无法加载高性能插件，我都怀疑是不是我的配置有问题，还是这个高性能插件就只支持个 例的模型，我测试几个产线，都是报The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance |
Beta Was this translation helpful? Give feedback.
-
|
在服务化部署的讨论区回复你了～ |
Beta Was this translation helpful? Give feedback.
-
|
自从出了这个3.0之后的什么管线功能，我就不再用paddlepaddle了 ------------------ Original ------------------From: 牛马 ***@***.***>Date: Thu,Aug 21,2025 6:21 PMTo: PaddlePaddle/PaddleX ***@***.***>Cc: Bin Qian ***@***.***>, Comment ***@***.***>Subject: Re: [PaddlePaddle/PaddleX]PaddleX/latest/pipeline_deploy/high_performance_inference (Discussion #2698)
我很好奇，连最简单的通用图像分类网络，都无法加载高性能插件，我都怀疑是不是我的配置有问题，还是这个高性能插件就只支持个 例的模型，我测试几个产线，都是报The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance
通用图像分类产线加载日志如下:
λ 335394c1debc /home paddlex --serve --pipeline image_classification --device gpu:0 --host 0.0.0.0 --port 8010 --use_hpip
Creating model: ('PP-LCNet_x0_5', None)
Using official model (PP-LCNet_x0_5), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Connecting to https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_5_infer.tar ...
Downloading PP-LCNet_x0_5_infer.tar ...
[==================================================] 100.00%
Extracting PP-LCNet_x0_5_infer.tar
[==================================================] 100.00%
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance.
Using Paddle Inference backend
Paddle predictor option: device_type: gpu, device_id: 0, run_mode: paddle, trt_dynamic_shapes: {'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]}, cpu_threads: 10, delete_pass: [], enable_new_ir: True, enable_cinn: False, trt_cfg_setting: {}, trt_use_dynamic_shapes: True, trt_collect_shape_range_info: True, trt_discard_cached_shape_range_info: False, trt_dynamic_shape_input_data: None, trt_shape_range_info_path: None, trt_allow_rebuild_at_runtime: True, mkldnn_cache_capacity: 10
INFO: Started server process [4305]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8010 (Press CTRL+C to quit)
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>
[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "#2698 (comment)",
"url": "#2698 (comment)",
"name": "View Discussion"
},
"description": "View this Discussion on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]
|
Beta Was this translation helpful? Give feedback.
-
|
1，高性能推理感觉速度差不多呀，我用的是ocr产线，纯cpu配置8核32G内存，安装 Paddle2ONNX 插件 ，有什么好的建议，目前识别一张100字的英文图片，10秒以上了，每次初始化ocr都花3~5秒。2，服务化部署后支持切换ocr识别模型吗，我想根据前端传的类型来具体初始化是英文还是中文模型，目前好像做不到。3，我通过python脚本方式集成，但是每次识别都必须初始化一次OCR，这是什么原因？如果在函数外先初始化一次，再传入图像识别，第一次识别成功，第二次就失败，第三次就成功。所以目前每次传过来图像，我都先初始化一次，再识别。 |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
好的，关于第三点，我测试了好多次，把产线对象只初始化一次，后续不断调用同一个对象的predict，第一次调用成功，第二次就报错，我是paddleocr==3.2.0，paddlepaddle==3.1.0，paddlex==3.2.0看我日志： |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
PaddleX/latest/pipeline_deploy/high_performance_inference
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/high_performance_inference.html
Beta Was this translation helpful? Give feedback.
All reactions