Drift when the robot make some steps

[N1ckfm]

Hi, when I tried a LIPM Walking controller, the observed value and control value increase the error for any walking step. The robot look like drift. It happen with HRP4 and with JVRC1.
I was thinking that maybe there is a value that has not been considered in the xml model of the robot that allows a more real behavior, maybe damping? stiffness? because these parameters are usually considered in any XML in the default mujoco robot
I tried different things for example.
noslip_iterations="50" in the xml file
I tried too with a damping value:
<joint limited="true" damping="20" />
I understand that this drift does not occur in choreonoid but I don't know if that reduces the "realism" of a smooth surface.
In the case of the real HRP4 robot, I don't know how much should be tolerable.
I will apprecite your support for improve the simulation.

[rohanpsingh]

Hi @N1ckfm
Sorry for the late response.

I talked with @mehdi-benallegue and he too agrees with what I told you via email.
The KinematicInertial state estimation of the floating base is not perfect due to slipping between feet-floor contacts.
If we were to use extereceptive measurements (say SLAM), it could be improved.
That is, mc-rtc has imperfect knowledge of where the real robot is in the world.

In mc-mujoco what you see is the real robot. But the feet markers are drawn relative to the control robot.
Hence, the apparent minor discrepancy after a few steps.

I think we won't see this effect if we set BodySensor update to true.
Unfortunately, I can't seem to make the robot walk using the BodySensor for now.

[N1ckfm]

Hi, I changed in the controller from KinematicInertial to BodySensor, using the LIPMWalking Controller. I see some improvement. But the robot falls in some steps.
In my code using mc_mujoco I use:

  auto & real = controller.realRobot();
  real.posW(robot.bodySensor("FloatingBase").position());

What would happen if in both cases I used KinematicInertial instead bodySensor?

[rohanpsingh]

1. Why are you changing the controller code? You shouldn’t have to change your code to use the simulator ground-truth measurements. It can be changed by modifying the observer pipeline in your controller config file. You can find more information here: https://jrl-umi3218.github.io/mc_rtc/tutorials/recipes/observers.html
2. In my case too, the robot falls if I use BodySensor updates with LIPMWalking. I discussed this with @mehdi-benallegue . He thinks it’s because some controller parameters are tuned to work with estimated state and not the ground-truth state. I may try to solve this problem with his help soon.
3. What do you mean by “in both cases”?

[N1ckfm]

1. I have a reinforcement learning code so in this code I use mc_mujoco and the controller. And yes I modified the observe pipeline by type: BodySensor in the *yaml file in the controller.
2. My doubt is if that drift will impact the training with Reinforcement Learning because I use the error between the realRobot() and robot() and if it will make an error sim to real transfer.
3. Both cases: In my reinforcement learning code (using bodySensor) and the controller (Observers type KinematicInertial)

[rohanpsingh]

Okay I see. It would be great if we’re able to make the robot walk with BodySensor + LIPMwalking.

What would happen if in both cases I used KinematicInertial instead bodySensor?

I don’t know what your RL controller is doing. If you’re making the robot walk using KI, there will be some accumulated drift in the robot’s position in any case.

My doubt is if that drift will impact the training with Reinforcement Learning because I use the error between the realRobot() and robot() and if it will make an error sim to real transfer.

That depends on what exactly you’re trying to do :) If you’re trying to minimize the error between 3D position of real and control robots, it should affect the training IMO. Maybe you can perform your optimization over smaller distances?

[rohanpsingh]

I think we won't see this effect if we set BodySensor update to true.
Unfortunately, I can't seem to make the robot walk using the BodySensor for now.

I was able to make BodySensor work. But contrary to what I wrote above, I saw the robot still drifts away from the footsteps.
As I realized later, this is not unexpected. The footsteps are still with respect to the controlRobot, which is still not at the same position/orientation as the realRobot.

One way to avoid the drift is to reduce the slip between the feet-floor contacts. You can try playing with the solref + solimp, and/or noslip_iterations + friction parameters in the mujoco XML. I am not looking into this currently but I'm confident it is possible and not so hard.

Hi, I changed in the controller from KinematicInertial to BodySensor, using the LIPMWalking Controller. I see some improvement. But the robot falls in some steps.

I investigated this a bit today. Turns out this is an old issue and we must get to the bottom of this ASAP.

[N1ckfm]

I was trying with differents value of solref + solimp + noslip_iterations + friction + codim ever I see this drift.