PaddleX/3.0-rc/pipeline_deploy/serving #3786
giscus[bot]
bot
Replies: 8 comments 47 replies
-
|
我在测试‘高稳定性服务化部署’，使用通用OCR SDK，本地docker启动成功，client.py测试GRPCInferenceService通过，Metrics Service访问成功，但是HTTPService请求失败，一直报错400 Bad Request。想问一下HTTPService的接口调用文档有没有，请求参数是什么，怎么才能正确访问HTTPService |
Beta Was this translation helpful? Give feedback.
-
|
好的，非常感谢 |
Beta Was this translation helpful? Give feedback.
-
|
sudo docker run -e PADDLEX_HPS_DEVICE_TYPE=cpu -v /usr/local/paddleocr:/app -w /app --rm --init --network host --shm-size 8g ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-cpu /bin/bash server.sh |
Beta Was this translation helpful? Give feedback.
-
我完全安装这个文档做的请求参数封装 为什么报错呢？{"error":"Unable to parse JSON bytes array: attempt to access JSON non-string as string"} |
Beta Was this translation helpful? Give feedback.
-
curl -X POST http://localhost:8000/v2/models/ocr/infer -H 'Content-Type: application/json' -d '{"inputs":[{"name":"input","shape":[1,1],"datatype":"BYTES","data":[{"file":"https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png","fileType":1,"visualize":false}]}],"outputs":[{"name":"output"}]}' 结果就是报错：{"error":"Unable to parse JSON bytes array: attempt to access JSON non-string as string"} |
Beta Was this translation helpful? Give feedback.
-
|
经过各种试错，终于解决了，data里的不是json数组，要改成字符串数组 |
Beta Was this translation helpful? Give feedback.
-
|
请问能否同时部署多个产线 |
Beta Was this translation helpful? Give feedback.
-
|
因为一条产线本身可能不止一个endpoint，为了避免过于复杂的架构，一次只支持启动一个服务。如果需要部署多条产线，可以启动多个服务，绑定到不同的端口 |
Beta Was this translation helpful? Give feedback.
-
|
执行/bin/bash server.sh报一下错误，请问有人遇到吗？怎么解决的呢？ |
Beta Was this translation helpful? Give feedback.
-
|
是自己的失误，解决了 |
Beta Was this translation helpful? Give feedback.
-
|
很强，希望能提供高性能的其他语言调用示例 |
Beta Was this translation helpful? Give feedback.
-
|
方便的话可以提个feature request issue，我们将讨论这些需求～ |
Beta Was this translation helpful? Give feedback.
-
|
你好，最新的文档中提供了手动构造HTTP请求的方式，这种方式对client的语言没有限制，可以试试～ |
Beta Was this translation helpful? Give feedback.
-
|
容器里 server.sh 不存在呀， 2.3 直接跑不了吗？ 是要把sdk 里的 sh，和yml 配置抄到挂载目录吗？ sudo docker run -e PADDLEX_HPS_DEVICE_TYPE=cpu -v /usr/local/paddleocr:/app -w /app --rm --init --network host --shm-size 8g ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-cpu /bin/bash server.sh 执行后 docker run -it --entrypoint /bin/bash ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-cpu |
Beta Was this translation helpful? Give feedback.
-
|
paddle应该不会自动安装nvcc。如果你是在安装docker之后安装nvidia driver的，请重启docker服务。 |
Beta Was this translation helpful? Give feedback.
-
|
我重启下docker服务问题解决了，如果不使用高性能推理的话，服务可以正常运行起来了，看到服务启动端口 I1216 11:37:21.602333 35 http_server.cc:2815] Started HTTPService at 0.0.0.0:8000 I1216 11:37:21.643494 35 http_server.cc:167] Started Metrics Service at 0.0.0.0:8002 docker run Tue Jul 15 01:49:33-INFO: set shape of Value(define_op_name=pd_op.data, index=0, dtype=tensor<-1x3x-1x-1xf32>, stop_gradient=False), op is: Operation((%0) = "pd_op.data" () {dtype:float32,name:"x",place:Place(undefined:0),shape:[-1,3,-1,-1],stop_gradient:[false]} : () -> tensor<-1x3x-1x-1xf32>) |
Beta Was this translation helpful? Give feedback.
-
|
高性能推理第一次启动的时候可能需要执行优化，可能需要花费较长时间，从第二次开始有缓存就不会这么慢了。如果采用上面的方式启动，同时希望保留缓存，请考虑建立模型下载目录和宿主机目录之间的映射，使缓存能够被持久地存储而不会随着容器销毁。或者，去掉 |
Beta Was this translation helpful? Give feedback.
-
|
哦，那就是还没执行完是吗，的等着他执行完就行了, 模型下载目录和宿主机目录之间的映射， 这个有哪些参数我可以配置呢，文档在哪里呢 |
Beta Was this translation helpful? Give feedback.
-
|
这取决于你把模型存在什么位置，在执行日志中也可以看到模型的下载目录。可以通过docker run的-v/--volume选项设置目录映射。这部分属于docker的用法，我们的文档中就没有体现了，具体可以参考docker官方文档哈～ |
Beta Was this translation helpful? Give feedback.
-
|
怎么指定2，3号gpu? |
Beta Was this translation helpful? Give feedback.
-
|
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-gpu 文档写的是这个啊？ 3.1版本的在哪里？ |
Beta Was this translation helpful? Give feedback.
-
|
请参考最新的文档：https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html#22 |
Beta Was this translation helpful? Give feedback.
-
|
大佬 能不能指导一下 其它语言 grpc 怎么调用的 |
Beta Was this translation helpful? Give feedback.
-
其他语言的grpc调用建议依赖对应语言的Triton client开发，但Triton官方并没有为每个语言都提供client～ |
Beta Was this translation helpful? Give feedback.
-
|
我跑在4张4090上的为什么会显示内存不足，是不是显存池太小才64mb,设置大一点的命令是什么？ |
Beta Was this translation helpful? Give feedback.
-
|
您好我在config_gpu内指定是2，3号gpu At: E0717 05:38:54.064662 7 model_repository_manager.cc:1186] failed to load 'ocr' version 1: Internal: OSError: (External) CUDA error(2), out of memory. At: |
Beta Was this translation helpful? Give feedback.
-
|
可以了4090上只能跑一个实例，设置为2就报错 |
Beta Was this translation helpful? Give feedback.
-
|
paddlex 的服务话部署，现在支持5090系列显卡了吗？ 我这边4060系列的话，部署没有问题，但是换成5090系列的话服务起不来了 I0813 12:57:11.529281 7 server.cc:592] I0813 12:57:11.529379 7 tritonserver.cc:1920] I0813 12:57:11.529399 7 server.cc:252] Waiting for in-flight requests to complete. C++ Traceback (most recent call last):No stack trace in paddle, may be caused by external reasons. Error Message Summary:FatalError: |
Beta Was this translation helpful? Give feedback.
-
|
50系列显卡正在支持中，目前已经发布的版本暂不支持 |
Beta Was this translation helpful? Give feedback.
-
|
预计什么时候可以发布呢， 非常期待 @Bobholamovic |
Beta Was this translation helpful? Give feedback.
-
|
目前已经在做最后的测试 预计最早在下个月发布 尽情期待 |
Beta Was this translation helpful? Give feedback.
-
|
版本gpu3.2部署成功了： 请求 /v2/models/ocr/infer?Content-Type=application/json 之后报错，错误如下： C++ Traceback (most recent call last):No stack trace in paddle, may be caused by external reasons. Error Message Summary:FatalError: E0829 00:52:27.109194 7 python.cc:1944] Stub process is unhealthy and it will be restarted. 在线等 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
PaddleX/3.0-rc/pipeline_deploy/serving
https://paddlepaddle.github.io/PaddleX/3.0-rc/pipeline_deploy/serving.html
Beta Was this translation helpful? Give feedback.
All reactions