Perfect results for stage 2 but then poor results for stage 3

[Hannibal42]

Hi, thank you for the great code base and contribution, so far it was a pleasure to work with it ;). Now my problem is that im trying to fit a 3D motion capture dataset using humor. I simply adapted the configs from the fit_amass_joints.cfg as my data is very similar to it. When I look at the results of stage 2 the fit looks perfectly but then stage 3 becomes very unstable and is very poor. Do you might have an idea what the overall problem might be? I already tried to work with different loss weights but so far I did not manage to fix this problem. It looks a little bit as if the fitting destabilizes over time and the falls apart. Here is an example: Left Original, Middle Stage 2, Right Stage 3

[davrempe]

It look like your original data skeleton is a bit different than SMPL -- how are you applying the 3D joint loss in this case? Since the joints between the two skeletons are not aligned it may somehow be destabilizing things in the 3rd stage. You may want to try only applying the 3D loss on a subset of joints that are well aligned (e.g. root, elbows, hands, knees, toe end) and see if that works better.

[Hannibal42]

Thank you for the advise I will try to find a suitable subset and see of that works. Until now I tried to approximate some of the points that I had no clear match for in my dataset.

[Hannibal42]

Fitting only a few, better matching points, did improve the results. However I guess I still need to do some more fine tuning.