Segmentation Costmap Plugin for ROS2 Nav2 (Humble)

This package provides a custom costmap plugin for ROS2 Nav2 (Humble) that integrates NVIDIA Isaac ROS Segment Anything instance segmentation masks with Intel RealSense D435 depth camera. The plugin projects segmented objects into 3D, transforms them into the map frame, and marks obstacles in the navigation costmap. A speed-filter rule set can be applied per-detected object via config/object_rules.yaml.

---

Features

• Instance segmentation integration (/segment_anything/raw_segmentation_mask)
• Aligned RealSense depth input (/camera/aligned_depth_to_color/image_raw)
• Optional YOLO fusion (/detections_output)
• Dynamic costmap updates by the plugin (circle/rectangle obstacles by class)
• Speed-limit rules via config/object_rules.yaml

---

Topics

• /segment_anything/raw_segmentation_mask — isaac_ros_tensor_list_interfaces/msg/TensorList
• /camera/aligned_depth_to_color/image_raw — sensor_msgs/msg/Image
• /detections_output — vision_msgs/msg/Detection2DArray (optional)
• /camera/color/camera_info — sensor_msgs/msg/CameraInfo
• /segmentation/detected_objects — segmentation_costmap_plugin/msg/DetectedObjects (published by the plugin)

---

Quick start (Robotino)

1. Build workspace:

cd ~/ros2_ws
colcon build --symlink-install --packages-select segmentation_costmap_plugin robotino4

2. Launch for Robotino (uses nav2_seg.launch.py included in this package):

ros2 launch robotino4 nav2_seg.launch.py 

The launch file starts the RealSense driver (aligned depth), the segmentation node, and Nav2 bringup with the segmentation costmap plugin enabled and speed rules set from config/object_rules.yaml.

---

Configuration

• config/nav2_params.yaml — Nav2 params(Robotino package) that include the segmentation costmap plugin in local/global costmaps.
• config/object_rules.yaml — Speed or behavior rules for detected classes (example below).

Example costmap plugin snippet (in nav2_params.yaml):

local_costmap:
  plugins:
    - segmentation_costmap_plugin::InstanceSegmentationCostmapPlugin

  segmentation_costmap_plugin:
    plugin: "segmentation_costmap_plugin::InstanceSegmentationCostmapPlugin"
    rules_file: "config/object_rules.yaml"
    inflation_radius: 0.30
    depth_neighborhood_size: 3
    use_depth_median: true

Example config/object_rules.yaml:

rules:
  - classes: [0] #person
    min_distance: 0.1
    max_distance: 1.2
    min_speed: 20.0
    max_speed: 50.0

  - classes: [56] #chair
    min_distance: 0.1
    max_distance: 1.3
    min_speed: 30.0
    max_speed: 60.0

---

Launch file

This package includes launch/nav2_seg.launch.py to simplify Robotino usage. The launch file loads RealSense (aligned depth), the segmentation node, and Nav2 with the plugin configured to use config/object_rules.yaml.

---

Troubleshooting

• Confirm topics exist: ros2 topic list
• Verify camera info received before processing: ros2 topic echo /camera/color/camera_info
• Check TF tree: ros2 run tf2_tools view_frames and inspect generated frames.pdf
• If AMCL cannot initialize due to TF, ensure object points use depth-image timestamps and that static transforms (camera↔base_link) are published.

---

License

Apache 2.0

---