Skip to content

Commit c4f2f11

Browse files
jaiveersinghNVjaiveersinghNV
committed
Isaac ROS 0.30.0 (DP3)
1 parent 0869016 commit c4f2f11

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+2407
-3
lines changed

‎.gitattributes‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Ignore Python files in linguist
2+
*.py linguist-detectable=false
3+
4+
# Images
5+
*.gif filter=lfs diff=lfs merge=lfs -text
6+
*.jpg filter=lfs diff=lfs merge=lfs -text
7+
*.png filter=lfs diff=lfs merge=lfs -text
8+
*.psd filter=lfs diff=lfs merge=lfs -text
9+
10+
# Archives
11+
*.gz filter=lfs diff=lfs merge=lfs -text
12+
*.tar filter=lfs diff=lfs merge=lfs -text
13+
*.zip filter=lfs diff=lfs merge=lfs -text
14+
15+
# Documents
16+
*.pdf filter=lfs diff=lfs merge=lfs -text
17+
18+
# Shared libraries
19+
*.so filter=lfs diff=lfs merge=lfs -text
20+
*.so.* filter=lfs diff=lfs merge=lfs -text
21+
22+
# ROS Bags
23+
**/resources/**/*.zstd filter=lfs diff=lfs merge=lfs -text
24+
**/resources/**/*.db3 filter=lfs diff=lfs merge=lfs -text
25+
**/resources/**/*.yaml filter=lfs diff=lfs merge=lfs -text

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Ignore all pycache files
2+
**/__pycache__/**

‎CONTRIBUTING.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Isaac ROS Contribution Rules
2+
3+
Any contribution that you make to this repository will
4+
be under the Apache 2 License, as dictated by that
5+
[license](http://www.apache.org/licenses/LICENSE-2.0.html):
6+
7+
> **5. Submission of Contributions.** Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
8+
9+
Contributors must sign-off each commit by adding a `Signed-off-by: ...`
10+
line to commit messages to certify that they have the right to submit
11+
the code they are contributing to the project according to the
12+
[Developer Certificate of Origin (DCO)](https://developercertificate.org/).
13+
14+
[//]: # (202201002)

‎LICENSE‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
12
Apache License
23
Version 2.0, January 2004
34
http://www.apache.org/licenses/
@@ -198,4 +199,4 @@
198199
distributed under the License is distributed on an "AS IS" BASIS,
199200
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200201
See the License for the specific language governing permissions and
201-
limitations under the License.
202+
limitations under the License.

‎README.md‎

Lines changed: 253 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/tutorial-bi3d-freespace-isaac-sim.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Tutorial for Freespace Segmentation with Isaac Sim
2+
3+
<div align="center"><img src="../resources/Isaac_sim_tutorial.gif" width="600px"/></div><br>
4+
5+
## Overview
6+
7+
This tutorial demonstrates how to use a [Isaac Sim](https://developer.nvidia.com/isaac-sim) and [isaac_ros_bi3d_freespace](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation) to create a local occupancy grid.
8+
9+
Last validated with [Isaac Sim 2022.2.1](https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/release_notes.html#id1)
10+
11+
## Tutorial Walkthrough
12+
13+
1. Complete steps 1-7 listed in the [Quickstart section](../README.md#quickstart) of the main README.
14+
2. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
15+
3. Open the Isaac ROS Common USD scene (using the *Content* tab) located at:
16+
17+
```text
18+
http://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/2022.2.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd
19+
```
20+
And wait for it to load completely.
21+
4. Go to the *Stage* tab and select `/World/Carter_ROS/ROS_Cameras/ros2_create_camera_right_info`, then in *Property* tab *-> OmniGraph Node -> Inputs -> stereoOffset X* change `0` to `-175.92`.
22+
<div align="center"><img src="../resources/Isaac_sim_set_stereo_offset.png" width="500px"/></div>
23+
5. Enable the right camera for a stereo image pair. Go to the *Stage* tab and select `/World/Carter_ROS/ROS_Cameras/enable_camera_right`, then tick the *Condition* checkbox.
24+
<div align="center"><img src="../resources/Isaac_sim_enable_stereo.png" width="500px"/></div>
25+
6. Project the `base_link` frame to the ground floor so that we can anchor our occupancy grid. Go to the Stage tab and select `/World/Carter_ROS/chassis_link/base_link`. In the *Property* tab, change the *Transform -> Translate -> Z* value from `0` to `-0.24`.
26+
<div align="center"><img src="../resources/Isaac_sim_change_base_link.png" width="500px"/></div><br>
27+
7. Disable the clock reset when simulation is stopped. Go to the *Stage* tab and select `/World/Clock/isaac_read_simulation_time`, then untick the *Reset On Stop* checkbox.
28+
<div align="center"><img src="../resources/Isaac_sim_disable_clock_reset.png" width="500px"/></div><br>
29+
8. Press **Play** to start publishing data from the Isaac Sim application.
30+
<div align="center"><img src="../resources/Isaac_sim_play.png" width="800px"/></div><br>
31+
9. Open a second terminal and attach to the container:
32+
33+
```bash
34+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
35+
./scripts/run_dev.sh
36+
```
37+
38+
10. In the second terminal, start the `isaac_ros_bi3d` node using the launch files:
39+
40+
```bash
41+
ros2 launch isaac_ros_bi3d_freespace isaac_ros_bi3d_freespace_isaac_sim.launch.py \
42+
featnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_featnet.plan \
43+
segnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_segnet.plan \
44+
max_disparity_values:=32
45+
```
46+
47+
You should see a RViz window, as shown below:
48+
<div align="center"><img src="../resources/Isaac_sim_rviz.png" width="500px"/></div><br>
49+
50+
11. Optionally, you can run the visualizer script to visualize the disparity image.
51+
52+
```bash
53+
ros2 run isaac_ros_bi3d isaac_ros_bi3d_visualizer.py --disparity_topic bi3d_mask
54+
```
55+
56+
<div align="center"><img src="../resources/Visualizer_isaac_sim.png" width="500px"/></div>

‎docs/tutorial-bi3d-freespace-realsense.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Tutorial for Freespace Segmentation using a RealSense Camera
2+
3+
<div align="center"><img src="../resources/realsense_example.gif" width="600px"/></div>
4+
5+
## Overview
6+
7+
This tutorial demonstrates how to use a [RealSense](https://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html) camera and [isaac_ros_bi3d_freespace](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_proximity_segmentation) to create a local occupancy grid.
8+
9+
> **Note**: This tutorial requires a compatible RealSense camera from the list available [here](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/realsense-setup.md#camera-compatibility).
10+
11+
## Tutorial Walkthrough
12+
13+
1. Complete the [RealSense setup tutorial](https://github.com/NVIDIA-ISAAC-ROS/.github/blob/main/profile/realsense-setup.md).
14+
2. Complete steps 1-7 described in the [Quickstart Guide](../README.md#quickstart).
15+
3. Open a new terminal and launch the Docker container using the `run_dev.sh` script:
16+
17+
```bash
18+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
19+
./scripts/run_dev.sh
20+
```
21+
22+
4. Build and source the workspace:
23+
24+
```bash
25+
cd /workspaces/isaac_ros-dev && \
26+
colcon build --symlink-install && \
27+
source install/setup.bash
28+
```
29+
30+
5. Please set your camera as shown in the image below, which is on a tripod ~10cm tall and parallel to the ground. Or you can change the static transform in launch file [here](../isaac_ros_bi3d_freespace/launch/isaac_ros_bi3d_freespace_realsense.launch.py#L144-157), according to the placement of your camera with respect to a occupancy grid origin frame.
31+
32+
<div align="center"><img src="../resources/realsense_camera_position.jpg" width="400px"/></div>
33+
34+
6. Run the launch file, which launches the example:
35+
36+
```bash
37+
ros2 launch isaac_ros_bi3d_freespace isaac_ros_bi3d_freespace_realsense.launch.py featnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_featnet.plan \
38+
segnet_engine_file_path:=/tmp/models/bi3d/bi3dnet_segnet.plan \
39+
max_disparity_values:=16
40+
```
41+
42+
7. Open a second terminal and attach to the container:
43+
44+
```bash
45+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
46+
./scripts/run_dev.sh
47+
```
48+
49+
8. Optionally, you can run the visualizer script to visualize the disparity image.
50+
51+
```bash
52+
ros2 run isaac_ros_bi3d isaac_ros_bi3d_visualizer.py --disparity_topic bi3d_mask
53+
```
54+
55+
<div align="center"><img src="../resources/visualizer_realsense.png" width="500px"/></div>
56+
57+
<div align="center"><img src="../resources/visualizer_realsense_mono_pair.png" width="500px"/></div>
58+
> Note: For more information on how to interpret the output, refer to the [interpreting the output section](../README.md#interpreting-the-output) of the main readme.
59+
60+
9. Open a third terminal and attach to the container:
61+
62+
```bash
63+
cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
64+
./scripts/run_dev.sh
65+
```
66+
67+
10. Visualize the occupancy grid in RViz.
68+
69+
Start RViz:
70+
71+
```bash
72+
rviz2
73+
```
74+
75+
In the left pane, change the **Fixed Frame** to `base_link`.
76+
77+
In the left pane, click the **Add** button, then select **By topic** followed by **Map** to add the occupancy grid. You should see an ouput similar to the one shown at the top of this page.

‎isaac_ros_bi3d_freespace/CMakeLists.txt‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
2+
# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
# SPDX-License-Identifier: Apache-2.0
17+
18+
cmake_minimum_required(VERSION 3.23.2)
19+
project(isaac_ros_bi3d_freespace LANGUAGES C CXX)
20+
21+
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
22+
add_compile_options(-Wall -Wextra -Wpedantic)
23+
endif()
24+
25+
find_package(ament_cmake_auto REQUIRED)
26+
ament_auto_find_build_dependencies()
27+
28+
# FreespaceSegmentation_node
29+
ament_auto_add_library(freespace_segmentation_node SHARED src/freespace_segmentation_node.cpp)
30+
rclcpp_components_register_nodes(freespace_segmentation_node "nvidia::isaac_ros::bi3d_freespace::FreespaceSegmentationNode")
31+
set(node_plugins "${node_plugins}nvidia::isaac_ros::bi3d_freespace::FreespaceSegmentationNode;$<TARGET_FILE:freespace_segmentation_node>\n")
32+
33+
### Install extensions built from source
34+
35+
# Bi3D Freespace
36+
add_subdirectory(gxf/bi3d_freespace)
37+
install(TARGETS gxf_occupancy_grid_projector DESTINATION share/${PROJECT_NAME}/gxf/lib/bi3d_freespace)
38+
39+
### End extensions
40+
41+
if(BUILD_TESTING)
42+
find_package(ament_lint_auto REQUIRED)
43+
ament_lint_auto_find_test_dependencies()
44+
45+
find_package(launch_testing_ament_cmake REQUIRED)
46+
add_launch_test(test/isaac_ros_freespace_segmentation_pol.py TIMEOUT "1000")
47+
endif()
48+
49+
ament_auto_package(INSTALL_TO_SHARE config launch)

‎isaac_ros_bi3d_freespace/config/freespace_segmentation_node.yaml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
%YAML 1.2
2+
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
3+
# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
# SPDX-License-Identifier: Apache-2.0
18+
---
19+
name: freespace_segmentation
20+
components:
21+
- name: mask_in
22+
type: nvidia::gxf::DoubleBufferReceiver
23+
parameters:
24+
capacity: 12
25+
policy: 0
26+
- name: occupancy_grid_out
27+
type: nvidia::gxf::DoubleBufferTransmitter
28+
parameters:
29+
capacity: 12
30+
policy: 0
31+
- name: allocator
32+
type: nvidia::gxf::UnboundedAllocator
33+
- name: occupancy_grid_projector
34+
type: nvidia::isaac_ros::freespace_segmentation::OccupancyGridProjector
35+
parameters:
36+
mask_receiver: mask_in
37+
output_transmitter: occupancy_grid_out
38+
allocator: allocator
39+
projection_transform: []
40+
intrinsics: []
41+
grid_height: 10
42+
grid_width: 10
43+
grid_resolution: 0.01
44+
- type: nvidia::gxf::MessageAvailableSchedulingTerm
45+
parameters:
46+
receiver: mask_in
47+
min_size: 1
48+
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
49+
parameters:
50+
transmitter: occupancy_grid_out
51+
min_size: 1
52+
---
53+
name: vault
54+
components:
55+
- name: signal
56+
type: nvidia::gxf::DoubleBufferReceiver
57+
parameters:
58+
capacity: 1
59+
policy: 0
60+
- type: nvidia::gxf::MessageAvailableSchedulingTerm
61+
parameters:
62+
receiver: signal
63+
min_size: 1
64+
- name: vault
65+
type: nvidia::gxf::Vault
66+
parameters:
67+
source: signal
68+
max_waiting_count: 1
69+
drop_waiting: false
70+
---
71+
components:
72+
- name: edge0
73+
type: nvidia::gxf::Connection
74+
parameters:
75+
source: freespace_segmentation/occupancy_grid_out
76+
target: vault/signal
77+
---
78+
components:
79+
- type: nvidia::gxf::GreedyScheduler
80+
parameters:
81+
clock: clock
82+
stop_on_deadlock: false
83+
- name: clock
84+
type: nvidia::gxf::RealtimeClock

0 commit comments

Comments
 (0)