Our Best VIO Solution, Demo on Disney’s Fastest Roller Coaster

[borongyuan]

Factor_Perception_Tron.mp4

[matlabbe]

I did this roller coaster last Summer, that was one of my highlight at Disney haha. Very cool to see this. I am impressed by the tracking in dark areas, and in particular where there are abrupt changes between high vs low exposures:

How much does vision actually help in this case? it feels the IMU may do most of the work. Thank you for sharing this!

[borongyuan]

Yes, IMU plays a leading role in the whole system. But visual information is also important. I used RGBD data for better visualization. But what is actually used for VIO is mono and depth images. OAK-D Pro W's mono cameras capture infrared light and have a larger FOV, so they can still provide a lot of information in the dark.

I released the world's first roller coaster SLAM dataset last year. Although I was able to run VIO on that sequence, there were still some flaws. For example, my demo last year actually had an obvious scale issue in the indoor section. So I spent about a year developing a new solution. I used a neural network to extract and track features, which can handle noisy images better. For IMU data, I used a more reasonable method for state estimation. In particular, the bias can be estimated more accurately during the initialization process. So current systems can actually tolerate complete loss of visual features for several seconds.

[borongyuan]

When I recorded the data last year I used an older OAK-D Pro W camera with a BMI270. It can only stably output 100 Hz data. When I retake this year, I used a new camera equipped with BNO086, and the IMU data rate was increased to 200 Hz.

[sun-under-bird]

Are you using rtabmap?

[sun-under-bird]

Can rtabmap's dual vision with IMU achieve this effect? If it is rtabmap, did it use the robot_localization EKF?