Where to find the missing file "all_to_latent_32_singlestage.npy"?

[nocollar163]

When running the command python -m posepile.ds.bml_movi.main --generate-affine-weights, the process fails due to the missing file all_to_latent_32_singlestage.npy. Where can I obtain this file, or is there an alternative way to resolve this issue? Thanks.

[isarandi]

Hi, here is the file: all_to_latent_32_singlestage.zip

Unfortunately this repo is not really plug-and-play (yet), but I'm working on cleaning up my repos to be more easy to use and to verify that everything is there.

[nocollar163]

Hi, here is the file: all_to_latent_32_singlestage.zip

Unfortunately this repo is not really plug-and-play (yet), but I'm working on cleaning up my repos to be more easy to use and to verify that everything is there.

Thank you for sharing the file "all_to_latent_32_singlestage.npy".

I noticed its shape is (122, 32), which does not match the dimensions of the pose prediction files. The 2D pose files have shape (580, 2) and the 3D pose files have shape (580, 3).

These pose prediction files (e.g., F_CP1_Subject_1.pkl, F_PG1_Subject_1_L.pkl, etc.) were generated using:
• Command: python -m humcentr_cli.estimate_3d_pose_video
• Model: metrabs_eff2l_y4_384px_800k_28ds
• Video data: from the BML_MoVi dataset (e.g., F_CP1_Subject_1.mp4, F_PG1_Subject_1_L.avi, etc.)

Before running "python -m posepile.ds.bml_movi.main --generate-affine-weights", do I need to design a mapping between the 580 and 122 shapes? Or is there another recommended way to resolve this mismatch?

Could you please advise on this? Thanks in advance.

[isarandi]

This preprocessing was done with an earlier iteration of the metrabs model that had 122 joints, then I processed MoVi and others and it grew to 500+. But I wonder what your goal exactly is, maybe I can help better if I know that. Do you want to get a preprocessed MoVi dataset?

[nocollar163]

This preprocessing was done with an earlier iteration of the metrabs model that had 122 joints, then I processed MoVi and others and it grew to 500+. But I wonder what your goal exactly is, maybe I can help better if I know that. Do you want to get a preprocessed MoVi dataset?

Thank you for your response. We are planning to fine-tune your pretrained model metrabs_eff2l_y4_384px_800k_28ds with the skeleton definition as bml_movi_87. Our current plan is as follows:

1. Collect our own video data and prepare annotations in a format similar to BML_MoVi dataset;
2. Process the data using your posepile project;
3. Fine-tune the model using your metrabs repository and the processed data.

At this stage, we would like to ask for guidance on how to process the BML_MoVi data properly. Thank you again for your help.

[nocollar163]

This preprocessing was done with an earlier iteration of the metrabs model that had 122 joints, then I processed MoVi and others and it grew to 500+. But I wonder what your goal exactly is, maybe I can help better if I know that. Do you want to get a preprocessed MoVi dataset?

Hi, I processed some data using the posepile project and generated the file bml_movi.pkl(attached) by following the steps in main.sh, with the following specific conditions:

1. Datasets Used: 8 videos from the BML_MoVi dataset:
   ◦ F_CP1_Subject_1.mp4
   ◦ F_CP1_Subject_2.mp4
   ◦ F_CP2_Subject_1.mp4
   ◦ F_CP2_Subject_2.mp4
   ◦ F_PG1_Subject_1_L.avi
   ◦ F_PG1_Subject_2_L.avi
   ◦ F_PG2_Subject_1_L.avi
   ◦ F_PG2_Subject_2_L.avi
2. Code Modification 1: In PosePile/src/posepile/ds/bml_movi/main.py, I modified the convert_pose(coords, weights) function to support an input shape of 122 keypoints (by taking the first 122 points) instead of the original 580.
3. Code Modification 2: In cameralib/cameralib.py, I updated the camera_to_image(self, points) function to camera_to_image(self, points, validate_distortion=None, dst=None) to ensure compatibility between cameralib and deltacamera.

Could you please advise on how to verify if the generated bml_movi.pkl file is correct? What would be the recommended validation steps or sanity checks?

bml_movi.zip