Skip to content

Commit 21a7145

Browse files
jaiveersinghNVjaiveersinghNV
committed
Isaac ROS 0.20.0 (DP2)
1 parent afffb79 commit 21a7145

File tree

66 files changed

+2732
-463
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+2732
-463
lines changed

‎CONTRIBUTING.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Isaac ROS Contribution Rules
2+
3+
Any contribution that you make to this repository will
4+
be under the Apache 2 License, as dictated by that
5+
[license](http://www.apache.org/licenses/LICENSE-2.0.html):
6+
7+
> **5. Submission of Contributions.** Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
8+
9+
Contributors must sign-off each commit by adding a `Signed-off-by: ...`
10+
line to commit messages to certify that they have the right to submit
11+
the code they are contributing to the project according to the
12+
[Developer Certificate of Origin (DCO)](https://developercertificate.org/).
13+
14+
[//]: # (202201002)

‎LICENSE‎

Lines changed: 201 additions & 65 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 180 additions & 74 deletions
Large diffs are not rendered by default.

‎docs/bi3d-example.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Example Usage of Isaac ROS Proximity Segmentation
22

3-
This is an example use case of Isaac ROS Proximity Segmentation, which includes monitoring the forward zones of an autonomous mobile robot (AMR) using a HAWK stereo camera. Consider three zones in front of the robot (Zones 1-3), corresponding to different alerts and actions the robot must take if an object breaches the zone. Zone 3 generates a warning, Zone 2 sends a signal for the robot to slow, and Zone 1 results in the robot stopping immediately.
3+
This is an example use case of Isaac ROS Proximity Segmentation, which includes monitoring the forward zones of an autonomous mobile robot (AMR) using a HAWK stereo camera. Consider three zones in front of the robot (Zones 1-3), corresponding to different alerts and actions the robot must take if an object breaches the zone. Zone 3 generates a warning, Zone 2 sends a signal for the robot to slow, and Zone 1 results in the robot stopping immediately.
44

55
<div align="center"><img src="../resources/safety_zones.png" width="400px"/></div>
66

‎docs/tutorial-bi3d-freespace-isaac-sim.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Tutorial for Freespace Segmentation with Isaac Sim
2+
<div align="center"><img src="../resources/Isaac_sim_tutorial.gif" width="600px"/></div></br>
3+
4+
## Overview
5+
This tutorial demonstrates how to use a [Isaac Sim](https://developer.nvidia.com/isaac-sim) and [isaac_ros_bi3d_freespace](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation) to create a local occupancy grid.
6+
7+
## Tutorial Walkthrough
8+
1. Complete steps 1-7 listed in the [Quickstart section](../README.md#quickstart) of the main README.
9+
2. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
10+
3. Open the Isaac ROS Common USD scene (using the **Content** window) located at:
11+
12+
`omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd`.
13+
14+
And wait for it to load completely.
15+
> **Note:** To use a different server, replace `localhost` with `<your_nucleus_server>`
16+
4. Go to the **Stage** tab and select `/World/Carter_ROS/ROS_Cameras/ros2_create_camera_right_info`. In the **Property** tab, change the **Compute Node -> Inputs -> stereoOffset -> X** value from `0` to `-175.92`.
17+
<div align="center"><img src="../resources/Isaac_sim_set_stereo_offset.png" width="500px"/></div></br>
18+
5. Enable the right camera for a stereo image pair. Go to the **Stage** tab and select `/World/Carter_ROS/ROS_Cameras/enable_camera_right`, then tick the **Condition** checkbox.
19+
<div align="center"><img src="../resources/Isaac_sim_enable_stereo.png" width="500px"/></div></br>
20+
6. Project the `base_link` frame to the ground floor so that we can anchor our occupancy grid. Go to the **Stage** tab and select `/World/Carter_ROS/chassis_link/base_link`. In the **Property** tab, change the **Transform -> Translate -> Z** value from `0` to `-0.24`.
21+
<div align="center"><img src="../resources/Isaac_sim_change_base_link.png" width="500px"/></div></br>
22+
23+
7. Disable the clock reset when simulation is stopped. Go to the **Stage** tab and select `/World/Clock/isaac_read_simulation_time`, then untick the **Reset On Stop** checkbox.
24+
<div align="center"><img src="../resources/Isaac_sim_disable_clock_reset.png" width="500px"/></div></br>
25+
8. Press **Play** to start publishing data from the Isaac Sim application.
26+
<div align="center"><img src="../resources/Isaac_sim_play.png" width="800px"/></div></br>
27+
9. Open a second terminal and attach to the container:
28+
```bash
29+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
30+
./scripts/run_dev.sh
31+
```
32+
10. In the second terminal, start the `isaac_ros_bi3d` node using the launch files:
33+
```bash
34+
ros2 launch isaac_ros_bi3d_freespace isaac_ros_bi3d_freespace_isaac_sim.launch.py \
35+
featnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_featnet.plan \
36+
segnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_segnet.plan \
37+
max_disparity_values:=32
38+
```
39+
You should see a RViz window, as shown below:
40+
<div align="center"><img src="../resources/Isaac_sim_rviz.png" width="500px"/></div></br>
41+
42+
11. Optionally, you can run the visualizer script to visualize the disparity image.
43+
```bash
44+
ros2 run isaac_ros_bi3d isaac_ros_bi3d_visualizer.py --disparity_topic bi3d_mask
45+
```
46+
<div align="center"><img src="../resources/Visualizer_isaac_sim.png" width="500px"/></div>

‎docs/tutorial-bi3d-freespace-realsense.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Tutorial for Freespace Segmentation using a RealSense Camera
2+
3+
<div align="center"><img src="../resources/realsense_example.gif" width="600px"/></div>
4+
5+
## Overview
6+
7+
This tutorial demonstrates how to use a [RealSense](https://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html) camera and [isaac_ros_bi3d_freespace](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation) to create a local occupancy grid.
8+
9+
> Note: This tutorial has been tested with a RealSense D455/D435 connected to an x86 PC with an NVIDIA graphics card, as well as a Jetson Xavier AGX.
10+
11+
## Tutorial Walkthrough
12+
13+
1. Complete the [RealSense setup tutorial](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/realsense-setup.md).
14+
2. Complete steps 1-7 described in the [Quickstart Guide](../README.md#quickstart).
15+
3. Open a new terminal and launch the Docker container using the `run_dev.sh` script:
16+
17+
```bash
18+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
19+
./scripts/run_dev.sh
20+
```
21+
22+
4. Build and source the workspace:
23+
24+
```bash
25+
cd /workspaces/isaac_ros-dev && \
26+
colcon build --symlink-install && \
27+
source install/setup.bash
28+
```
29+
30+
5. Please set your camera as shown in the image below, which is on a tripod ~10cm tall and parallel to the ground. Or you can change the static transform in launch file [here](../isaac_ros_bi3d_freespace/launch/isaac_ros_bi3d_freespace_realsense.launch.py#L144-157), according to the placement of your camera with respect to a occupancy grid origin frame.
31+
32+
<div align="center"><img src="../resources/realsense_camera_position.jpg" width="400px"/></div>
33+
34+
6. Run the launch file, which launches the example:
35+
36+
```bash
37+
ros2 launch isaac_ros_bi3d_freespace isaac_ros_bi3d_freespace_realsense.launch.py featnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_featnet.plan \
38+
segnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_segnet.plan \
39+
max_disparity_values:=16
40+
```
41+
42+
7. Open a second terminal and attach to the container:
43+
44+
```bash
45+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
46+
./scripts/run_dev.sh
47+
```
48+
49+
8. Optionally, you can run the visualizer script to visualize the disparity image.
50+
51+
```bash
52+
ros2 run isaac_ros_bi3d isaac_ros_bi3d_visualizer.py --disparity_topic bi3d_mask
53+
```
54+
55+
<div align="center"><img src="../resources/visualizer_realsense.png" width="500px"/></div>
56+
57+
<div align="center"><img src="../resources/visualizer_realsense_mono_pair.png" width="500px"/></div>
58+
> Note: For more information on how to interpret the output, refer to the [interpreting the output section](../README.md#interpreting-the-output) of the main readme.
59+
60+
9. Open a third terminal and attach to the container:
61+
62+
```bash
63+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
64+
./scripts/run_dev.sh
65+
```
66+
67+
10. Visualize the occupancy grid in RViz.
68+
69+
Start RViz:
70+
71+
```bash
72+
rviz2
73+
```
74+
75+
In the left pane, change the **Fixed Frame** to `base_link`.
76+
77+
In the left pane, click the **Add** button, then select **By topic** followed by **Map** to add the occupancy grid. You should see an ouput similar to the one shown at the top of this page.

‎docs/tutorial-bi3d-isaac-sim.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Tutorial for Bi3D with Isaac Sim
2+
3+
<div align="center"><img src="../resources/Rviz.png" width="800px"/></div></br>
4+
5+
## Overview
6+
7+
This tutorial walks through setting up a pipeline to [segment stereo image pairs](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation) generated by Isaac Sim by depth disparity levels. For more details on the output please refer to the [interpreting the output](../README.md#interpreting-the-output) section of the [main readme](../README.md).
8+
9+
## Tutorial Walkthrough
10+
11+
1. Complete steps 1-7 listed in the [Quickstart section](../README.md#quickstart) of the main README.
12+
2. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
13+
3. Open the Isaac ROS Common USD scene (using the **Content** window) located at:
14+
15+
`omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd`.
16+
17+
And wait for it to load completely.
18+
> **Note:** To use a different server, replace `localhost` with `<your_nucleus_server>`
19+
4. Go to the **Stage** tab and select `/World/Carter_ROS/ROS_Cameras/ros2_create_camera_right_info`. In the **Property** tab, change the **Compute Node -> Inputs -> stereoOffset -> X** value from `0` to `-175.92`.
20+
<div align="center"><img src="../resources/Isaac_sim_set_stereo_offset.png" width="500px"/></div></br>
21+
22+
5. Enable the right camera for a stereo image pair. Go to the **Stage** tab and select `/World/Carter_ROS/ROS_Cameras/enable_camera_right`, then tick the **Condition** checkbox.
23+
<div align="center"><img src="../resources/Isaac_sim_enable_stereo.png" width="500px"/></div></br>
24+
6. Press **Play** to start publishing data from the Isaac Sim application.
25+
<div align="center"><img src="../resources/Isaac_sim_play.png" width="800px"/></div></br>
26+
7. Open a second terminal and attach to the container:
27+
28+
```bash
29+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
30+
./scripts/run_dev.sh
31+
```
32+
33+
8. In the second terminal, start the `isaac_ros_bi3d` node using the launch files:
34+
35+
```bash
36+
ros2 launch isaac_ros_bi3d isaac_ros_bi3d_isaac_sim.launch.py \
37+
featnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_featnet.plan \
38+
segnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_segnet.plan \
39+
max_disparity_values:=32
40+
```
41+
42+
You should see a RViz window, as shown at the top of this page.
43+
44+
9. Optionally, you can run the visualizer script to visualize the disparity image.
45+
46+
```bash
47+
ros2 run isaac_ros_bi3d isaac_ros_bi3d_visualizer.py
48+
```
49+
50+
<div align="center"><img src="../resources/Visualizer_isaac_sim.png" width="500px"/></div>

‎isaac_ros_bi3d/CMakeLists.txt‎

Lines changed: 16 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,19 @@
1-
# Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
1+
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
2+
# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
23
#
3-
# NVIDIA CORPORATION and its licensors retain all intellectual property
4-
# and proprietary rights in and to this software, related documentation
5-
# and any modifications thereto. Any use, reproduction, disclosure or
6-
# distribution of this software and related documentation without an express
7-
# license agreement from NVIDIA CORPORATION is strictly prohibited.
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
# SPDX-License-Identifier: Apache-2.0
817

918
cmake_minimum_required(VERSION 3.8)
1019
project(isaac_ros_bi3d)
@@ -31,38 +40,11 @@ message(STATUS "Architecture: ${ARCHITECTURE}")
3140
find_package(ament_cmake_auto REQUIRED)
3241
ament_auto_find_build_dependencies()
3342

34-
# CVCore Bi3D extension library
35-
add_library(cvcore_bi3d SHARED IMPORTED)
36-
37-
if(${ARCHITECTURE} STREQUAL "x86_64")
38-
set_property(
39-
TARGET cvcore_bi3d PROPERTY IMPORTED_LOCATION
40-
${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/x86_64/libgxf_cvcore_bi3d.so)
41-
elseif(${ARCHITECTURE} STREQUAL "aarch64")
42-
set_property(
43-
TARGET cvcore_bi3d PROPERTY IMPORTED_LOCATION
44-
${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/jetpack50/libgxf_cvcore_bi3d.so)
45-
endif()
46-
47-
# Isaac ROS Bi3D postprocess extension library
48-
add_library(bi3d_postprocess SHARED IMPORTED)
49-
50-
if(${ARCHITECTURE} STREQUAL "x86_64")
51-
set_property(
52-
TARGET bi3d_postprocess PROPERTY IMPORTED_LOCATION
53-
${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/x86_64/libgxf_bi3d_postprocessor.so)
54-
elseif(${ARCHITECTURE} STREQUAL "aarch64")
55-
set_property(
56-
TARGET bi3d_postprocess PROPERTY IMPORTED_LOCATION
57-
${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/jetpack50/libgxf_bi3d_postprocessor.so)
58-
endif()
59-
6043
# bi3d_node
6144
ament_auto_add_library(bi3d_node SHARED src/bi3d_node.cpp)
6245
target_compile_definitions(bi3d_node
6346
PRIVATE "COMPOSITION_BUILDING_DLL"
6447
)
65-
target_link_libraries(bi3d_node cvcore_bi3d bi3d_postprocess)
6648
rclcpp_components_register_nodes(bi3d_node "nvidia::isaac_ros::bi3d::Bi3DNode")
6749
set(node_plugins "${node_plugins}nvidia::isaac_ros::bi3d::Bi3DNode;$<TARGET_FILE:bi3d_node>\n")
6850

@@ -72,13 +54,6 @@ install(
7254
DESTINATION share/${PROJECT_NAME}
7355
)
7456

75-
# Install extensions directory
76-
if(${ARCHITECTURE} STREQUAL "x86_64")
77-
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/x86_64/ DESTINATION share/${PROJECT_NAME}/cvcore_bi3d)
78-
elseif(${ARCHITECTURE} STREQUAL "aarch64")
79-
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/cvcore/cvcore_bi3d/lib/jetpack50/ DESTINATION share/${PROJECT_NAME}/cvcore_bi3d)
80-
endif()
81-
8257
set_target_properties(bi3d_node PROPERTIES BUILD_WITH_INSTALL_RPATH TRUE)
8358
install(TARGETS bi3d_node
8459
ARCHIVE DESTINATION lib
@@ -88,10 +63,6 @@ install(TARGETS bi3d_node
8863

8964
if(BUILD_TESTING)
9065
find_package(ament_lint_auto REQUIRED)
91-
92-
# Ignore copyright notices since we use custom NVIDIA Isaac ROS Software License
93-
set(ament_cmake_copyright_FOUND TRUE)
94-
9566
ament_lint_auto_find_test_dependencies()
9667

9768
find_package(launch_testing_ament_cmake REQUIRED)
@@ -103,4 +74,4 @@ install(PROGRAMS
10374
DESTINATION lib/${PROJECT_NAME}
10475
)
10576

106-
ament_auto_package(INSTALL_TO_SHARE launch)
77+
ament_auto_package(INSTALL_TO_SHARE launch config)

0 commit comments

Comments
 (0)