rotational offset/bias for multi-rgbd-inertial-dataset (MRID)

[xonobo]

I am trying to get results for the MRID dataset given here.

For the indoor sequence, I tried to get results with the RGBDdataset executable by feeding the bag files extracted into euroc folder structure. I just fed the front camera's RGB and Depth images, no IMU.

Here is the evo plot and its rmse scores:
translational rmse = 0.088584 meter
rotational rmse = 118.454700 degree

As you can see in the rotational error plot, there is an offset/bias in the estimates.

I couldn't figure out the source of this rotational offset/bias in the pose estimates.

Since the dataset processing executables are tailored to the datasets, I thought I might have missed or duplicated some transformations during that dataset-specific tuning of the algorithm. That's why I decided to feed the RGBD images into rtabmap using the ROS2 pipeline. In this case, I just set the RGBD camera transformation as the optical transformation.

  # optical rotation
  data: [0.0, 0.0, 1.0, 0.0,
        -1.0, 0.0, 0.0, 0.0,
         0.0, -1.0, 0.0, 0.0,
         0.0, 0.0, 0.0, 1.0]

In this case, odometry tracking is lost after the first turn, and the rotation offset/bias persists. Here I attach the evo results for the ROS2 outputs.

translaional rmse: 0.200719 meter
rotational rmse: 119.669258 degree

I also attached the ROS2 launch command output.
rtabmap.ros2.log

I have two questions based on these results.
The first one, which is my main concern, what am I missing to get those good translational but biased rotational estimates both for ROS2 and the dataset executable?
The second one, why ROS2 pipeline cannot keep tracking?

Thanks in advance for your comments, and help.

[matlabbe]

The main thing that is different between the offline CLI tool and online ROS package is the number of frames that is actually processed. With the CLI tool, all frames are processed, even if it is not real-time. For the ros package, it would skip frames when processing time is over the camera frame rate. That could cause the lost issue when running VO online, in particular if the computer cannot process frames fast enough and the camera is moving/rotating fast.

I'll take a look at this dataset and come back with a example launch file that would work.

[matlabbe]

I created a launch file to run rtabmap_ros with the rosbags of this dataset. Rtabmap should be built with OpenGV for multi-camera config.

Usage:

roslaunch rtabmap_examples multi_rgbd_inertial_dataset.launch
rosbag play --clock Indoor.bag

Results:

Here how the nodes are connected to input 4 cameras to rtabmap_ros (open image in new tab to zoom in):

Comparing with ground truth (with evo tools):

# export the poses in TUM format:
$ rtabmap-export --poses --poses_format 10 ~/.ros/rtabmap.db

# Compare APE translation:
$ evo_ape tum -a -p --plot_mode xy gt_indoor.txt ~/.ros/rtabmap_poses.txt
APE w.r.t. translation part (m)
(with SE(3) Umeyama alignment)

       max	0.118361
      mean	0.068072
    median	0.067671
       min	0.010991
      rmse	0.072186
       sse	0.307435
       std	0.024021

# Compare APE rotation:
$ evo_ape tum -r angle_deg -a -p --plot_mode xy gt_indoor.txt ~/.ros/rtabmap_poses.txt
APE w.r.t. rotation angle in degrees (deg)
(with SE(3) Umeyama alignment)

       max	2.484153
      mean	1.968670
    median	1.988270
       min	1.293006
      rmse	1.987854
       sse	233.142158
       std	0.275498

The angle error looks okay in this example. To use the dataset with a modified RGBDdataset executable, how did you convert the rosbags to a directory of images?

[matlabbe]

Update: I ported the example on ros2 for convenience: multi_rgbd_inertial_dataset.launch.py