How to convert output heatmap into keypoint coordinates and confidence scores

[ggggg111]

Hi, I am working on a Libtorch-based C++ application using the HRNet32 from torchscript. My question is, how does MMPose convert the heatmap [K, H W] into the keypoints and confidences?

Thank you.

[Ben-Louis]

Hi, you can refer to get_heatmap_maximum for how to convert heatmaps into coordinates and confidence. In short, the coordinates and values of the maximum points the each heatmap is returned.

[ggggg111]

Thank you for the fast response!

I've been experimenting with what you just said, and gonna check get_heatmap_maximum in more detail. The problem I encounter is how to handle the cases where the heatmap contains data corresponding to more than 1 person. We could get the topk(2) values, but it will be incorrect if, e.g. the second maximum value is right next to the maximum value pixel corresponding to the same person.

[ggggg111]

@Ben-Louis Nevermind, I forgot to use a detector to detect the persons from the input video first. I just gave the input video directly to the pose estimator.