Skip to content

RK3576中调用rknn_run,性能随着调用次数增加下降明显 #370

@hz09310633

Description

@hz09310633

模型使用yolo11s-int8量化模型,采用的zero-copy方案,rknn_run有个现象,一秒钟调用3次,单次调用的平均耗时仅为18ms/次,而一秒钟调用25次,单次调用的平均耗时会上升到31ms/次,且进程和进程之间调用也会互相影响,初步怀疑是某些共享资源,这边看到1秒调用3次,NPU占用率33%,1秒调用25次,NPU占用率80%,这是运行多实例后cma的空间信息
root@linaro-alip:~# cat /proc/meminfo | grep -i cma
CmaTotal: 16384 kB
CmaAllocated: 4704 kB
CmaReleased: 11680 kB
CmaFree: 6984 kB

使用strace打印io的日志出现如下信息
[1717971.244852] mpp_rkvdec2 27b00100.rkvdec: session 1201 task 9314820 irq_status 0xf0000002 timeout 0 abort 0
[1717971.245050] mpp_rkvdec2 27b00100.rkvdec: resetting...
[1717971.245109] mpp_rkvdec2 27b00100.rkvdec: reset done
[1718016.495464] mpp_rkvdec2 27b00100.rkvdec: session 1203 task 9320121 irq_status 0xf0000002 timeout 0 abort 0
[1718016.495797] mpp_rkvdec2 27b00100.rkvdec: resetting...
[1718016.495876] mpp_rkvdec2 27b00100.rkvdec: reset done
[1718016.495887] mpp_rkvdec2 27b00100.rkvdec: resend task 9320122
[1718016.495927] mpp_rkvdec2 27b00100.rkvdec: resend task 9320123
[1718016.495949] mpp_rkvdec2 27b00100.rkvdec: resend task 9320124
[1718016.495997] mpp_rkvdec2 27b00100.rkvdec: resend task 9320125
[1718016.496015] mpp_rkvdec2 27b00100.rkvdec: resend task 9320126

可以看到NPU和CMA都没有见底,但是解码已经出现较多的重发,rknn_run的性能在我跑8路实例的时候,会达到最低的100ms/次

由于NPU占用率没有达到200%(2核),初步排除NPU计算资源不足,CmaFree也没见过底,CPU/内存也消耗不高。目前rknn没有提供队列信息,有什么办法确定资源冲突点吗

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions