[Question] How to tune feet_air_time weight for reward

[ClaudioChiariello]

Question

Hello, I am trying to train a small quadruped robot in IsaacGym. It effectively walks, but doesn't seem to want to lift its foot. I am trying to use the feet_air_time reward, but nothing happens. I would like to ask for some tips, as I am currently adjusting the weights using a trial and error approach. Keep in mind that:

• The robot is the xgomini2, weighs 915g, and has a height of 18cm.
• Since this is a small robot, I am using a torque penalty that ranges from 1e-2 to 1e-1
• Its maximum velocity is about 0.3 m/s. The tracking linear velocity weight for the reward is about 1.25
• The USD does not have a foot-link, but ends with a leg-foot. I don’t think this should be an issue, as the reward should trigger when there is no contact between the leg and the ground, encouraging the robot to lift its leg

I am unsure how large the feet_air_time reward should be and would appreciate any advice on reward tuning. Initially, I thought about increasing the weight from 0.125 to a very large value, but this caused strange behaviors, and I haven't been able to find a good trade-off. Therefore, I would appreciate any guidance on reward tuning. Thank you so much

[Ashutosh781]

I have also been facing similar issues when training Unitree Go2 quadruped. I am able to achieve a pronking-like gait for the quadruped and even do sim-to-real transfers. However, these policies were trained without feet air time reward and lifting the feet only barely not to trip over.

Whenever I train with the feet air time reward the policy doesn't learn to take a step at all in simulation. I have tried a lot of tricks, from reducing the threshold value to increasing the weight, but nothing seems to work. Any advice on tuning the feet air time reward to achieve a proper trot gait would be really appreciated.

I have read many recent research papers that have achieved a good walking policy on Go2 using the legged gym, but it seems like there are significant differences in this code base, and the exact reward parameters do not transfer.

[RandomOakForest]

Thank you for posting this. I will move this to our Discussions for the team to follow up.