How to start making map from real-time camera image

[IvanS297]

Hi everyone, I've put together a package for ros2 jazzy.
I have these topics:
/camera/camera_info
/camera/image_raw
/camera/image_raw/compressed
I'd like to know how to create a real-time map from a camera image instead of a recorded video. Will the map be published in a thread? Can it be converted to a 2D map using rtabmap https://github.com/introlab/rtabmap_ros?

[Mechazo11]

Hi @IvanS297 you will need to first add an image transport node (by making a launch file or starting it manually) of this package to convert the CompressedImage message into Image message. Ideally you need modify the python node to consume this message and also make a Frame id, i.e. an integer number. See run_py_node method for more details.

Will the map be published in thread? Not necessarily, you can get the odometry for the passed image by publishing it on a certain topic.

Can it be converted to a 2D map using rtabmap? As far as I recall, ORB-SLAM3 provides the keyframe management and pose graph optimization backend. It does not make a 2D OccupancyMap unlike Nav2's slam_toolbox. That framework, in my opinion is a composite software where the Karto is akin to ORB-SLAM3 backend with additional ray tracing logic that manages creation of the 2D occupancy map.

This was a long term misconception of mine as well, ORB-SLAM3's map is only meant to solve the incremental pose graph o ptimization to give you better odometry estimate, beyond that its map is not very useful.

I hope this answers your question