Multi-clip reward for DeepMimic

[tfederico]

Hi,

I am trying to implement the multi-clip reward described in the DeepMimic paper.

As far as I understood, to calculate multiple rewards I need to load multiple clips. To do so, I need to create multiple kinematic models in pybullet, one for each mocap.
However, if I instantiate multiple kinematic models, I will probably also need multiple pose interpolator, as it retains the rotations and velocities of the character.

Am I doing it correctly? Or is one kinematic model sufficient and I just need to instantiate multiple pose interpolators?
I am a bit confused about what happens when I try to calculate the poses for multiple clips, as I have the feeling that I might end up updating the same model using more than one clip.

[ManifoldFR]

I think someone using the original code tried working on it (xbpeng/DeepMimic#120). It involved a few modifications to the C++ environment code

[tfederico]

Thanks, I have seen at that. They create several kinematic characters (one for each motion clip) and calculate the max reward. However, in this case it is not clear to me if it necessary to create several kinematic models or pose interpolators to implement the same method as in the original code.

[tfederico]

I have tried to instantiate multiple pose interpolators for a single kinematic model, but the training results do not look promising.
@erwincoumans is it safe to assume that to calculate multiple rewards (one for each mocap) I can use multiple pose interpolators on a single kinematic character? Or do I need multiple kinematic chars?

As far as I understood, the simulated model is used for learning and the kinematic character to represent the mocap.

[tchemaly]

Hello, which script are you modifying in order to implement multi-clip reward? I know the script in the original DeepMimic is SceneImitate.cpp. But what is the alternative for SceneImitate.cpp in the pybullet version?

[tfederico]

I modified

• motion_capture data
• humanoid_stable_pd
• pybullet_deep_mimic_env
• DeepMimic_Optimizer
• mpi_run
• testrl

The last three scripts were modified in order to execute the new scripts that I wrote instead of the original ones.

[erwincoumans]

Thanks! Do you have any results (video, learning curves etc) to share? Is there a github fork with the changes?

[tfederico]

I am working on a local copy of the repo, I will happily share it once I am done! Atm I have tried to train a policy to walk in different directions based on the input clip, I will need to look for other clips or interpolate them so that they have the same duration

[tchemaly]

Could you share how you loaded multiple clips and created several kinematic characters?

[tfederico]

I will try to make a pull request in the upcoming weeks with the code

…

On Fri, 31 Jul 2020 at 13:15, tchemaly ***@***.***> wrote: Could you share how you loaded multiple clips and created several kinematic characters? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2903 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADU47BQISWA2IQGSKU46HADR6KRUJANCNFSM4ONTCQKA> .