Define the problem of base estimation without contact assumption

[Zweisteine96]

After almost one month investigation and another discussion with @Giulero and @GiulioRomualdi last Wednesday, eventually we decide to state our base estimation problem as follows:

Given a set of inertial measurements, we want to track the kinematics of the human base during agile and dynamic motions, such as jumping, running or even back-flipping.

Unlike the commonly seen odometry problem, which put more emphasis on the accuracy of positioning and yaw estimation, our goal is to understand if and how can we relax the contact constraint when solving such problems, therefore we focus more on those agile and dynamics movements instead of periodic walking where the contact assumption usually holds.

cc @DanielePucci

[Zweisteine96]

Many works of (base) state estimation rely on the contact assumption, either used to retrieve the base kinematics via the inverse process of the forward kinematics (e.g., leg odometry) or used as constraints to correct the estimate in a filtering-based scheme (e.g., leg inertial odometry).

The motivation of adding such contact assumption is to reduce the integration errors caused by the sensor noises. However, such assumption can also restrict the robot's motion and adaptability in contact-free situations.

A very nice video which demonstrates what we want to achieve can be found here: https://www.youtube.com/watch?v=3x9dimY7o-o. For example, in the case of jump over wall, the base of the reconstructed human very nicely matches the motion we expect. Also, in the case of jumping jacks, where the base moves basically along the z-axis, we can also see clearly the movements of the base at each timestamp.
They can even do this:

There is definitely no foot-contact here, but still, in the presented recordings, the reconstruction captures the real base kinematics very well. Even though with the following limitations claimed by the author:

• Hands and feet motion not tracked
• Drift in global translation

They said:" In the future, we plan to integrate constraints derived from the world geometry and integrate GPS data."

[Zweisteine96]

Note that in the work presented above, the orientation of the base is assumed to be given, which is from the sensor measurements. There is only the global translation of the base considered in the estimate problem. More details see the Section 3.2 in the paper: https://arxiv.org/pdf/1703.08014.

[Zweisteine96]

In another work proposed by Xinyu Yi, they estimate the global translation of the base in a hybrid way. They also assumed the orientation of the base is already available from the IMU attached on the pelvis.

As shown in the green block at the bottom, there are two branches that estimate the base linear velocity, then fuse the two estimates and integrate it to get the base translation.

The first branch Trans-B1 is similar to the idea of leg odometry. They applied the estimated pose parameters on the kinematic model with optional user-specific leg lengths, and the base velocity is the computed as the coordinate difference of the supporting foot between two consecutive frames. Consequently, they trained a network to estimate the foot-ground contact probability of each foot. The inputs of this network is the estimated leaf joint positions and the inertial measurements (i.e., acc and ori).

However, such approach can't handle the cases where both feet are off the ground simultaneously (e.g., jumping forward, jumping upwards, running and etc.). Therefore they trained another network to estimate the base linear velocity directly from the inputs of estimated joint positions and the IMU measurements. The output of this network is the base linear velocity in the local frame and can be easily transformed to the world frame. To train this network, they used a loss function that measures the differences between the predicted and the ground truth translations in every consecutive 1, 3, 9 and 27 frames. They found that using the consecutive 1 frame alone makes the network only focus on neighboring frames. resulting in unstable estimation and large errors. The supervision on larger frame ranges helps to reduce the accumulative error.

At last, they fused the results obtained from the two branches to get the final global linear velocity.

They empirically set the lower bound and upper bound as 0.5 and 0.9, respectively. A nice video demonstration can be found here: https://www.youtube.com/watch?v=KTqj2a3lKo0.

At the end, the authors claimed that: "... Our supporting-foot-based method relies on the assumption that the support foot has no motion in the world coordinate space, which is not true for motions like skating and sliding." My understanding of this statement is that, since the authors used a fusion approach to estimate the base linear velocity, and one of the branch automatically inherits the limitations that leg odometry has. Specifically, in this case, it is assumed that the non-slipping contact foot should have zero-velocity, but this will not hold in many real-world cases.

[Zweisteine96]

Maybe consider the following three approaches:

Purely learning-based

We just train an End-to-End network for the base kinematics estimation. The inputs can include (but not limited to): 1) inertial measurements; 2) joint kinematics. The outputs are the desired base kinematics quantities. We make no contact assumption or zero-velocity assumption here. Ideally, the network should learn the underlying constraints from the data itself (if the data is good and large enough).

Simultaneous fusion

This is the same idea as shown in this comment. We run two branches at the same time, one is based on a network estimation, while the other relies on a conventional method. In the end we fuse the two estimates.

Non-simultaneous fusion

This is similar to the second idea, but not exactly the same. There are also two branches running here, one is the network-based, the other is a conventional method (e.g., filtering-based or simply leg odometry). When there is at least one foot-ground contact detected, we follow the fusion scheme. For the contact-free cases, we enable only the network-based estimator and mute the conventional method.

[Zweisteine96]

The last two approaches still rely on the accurate detection of contacts. Actually I think as long as we are using the contact information (no matter for what reason) we might suffer in the cases the contact situation changes quickly, for example, rope jumping. Therefore I would start with a pure learning-based fashion to solve this problem. Relative investigations on this direction will be documented here: #9

[Zweisteine96]

I think I missed one category in this comment: #8 (comment).

Hybrid approach

As presented in TLIO-based works, a filtering-based method is adopted. However, they utilized past states as measurements rather than the contact information. Therefore they are also free from the contact assumption. However, since a conventional EKF is applied, there still exists the problem of integrating noise accelerations. Even though the relative displacement is used to correct the filter prediction, the final estimate can still drift after a while. Furthermore, the studies focus more on the global positioning rather than the base estimation for highly dynamic motions.