Skip to content

Commit 9ee3eac

Browse files
hemalshahNVhemalshahNV
authored
Merge pull request #47 from NVIDIA-ISAAC-ROS/release-3.0
Isaac ROS 3.0.0
2 parents 6b0444e + 177ef85 commit 9ee3eac

File tree

285 files changed

+7569
-2897
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

285 files changed

+7569
-2897
lines changed

‎README.md‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Isaac ROS Image Pipeline
22

3-
Hardware-accelerated Image Pipeline.
3+
NVIDIA-accelerated Image Pipeline.
44

55
<div align="center"><img src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_image_pipeline/100_right.jpg/" width="300px"/>
66
<img src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_image_pipeline/300_right_hallway2_rect.png/" width="300px"/></div>
@@ -49,11 +49,11 @@ This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://dev
4949

5050
## Performance
5151

52-
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Orin<br/><br/> | Orin NX<br/><br/> | Orin Nano 8GB<br/><br/> | x86_64 w/ RTX 4060 Ti<br/><br/> |
53-
|------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|
54-
| [Rectify Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts/isaac_ros_rectify_node.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [819 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-agx_orin.json)<br/><br/><br/>5.4 ms<br/><br/> | [496 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-orin_nx.json)<br/><br/><br/>5.3 ms<br/><br/> | [347 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-orin_nano.json)<br/><br/><br/>6.4 ms<br/><br/> | [900 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-nuc_4060ti.json)<br/><br/><br/>3.9 ms<br/><br/> |
55-
| [Stereo Disparity Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts/isaac_ros_disparity_node.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [160 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-agx_orin.json)<br/><br/><br/>11 ms<br/><br/> | [76.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-orin_nx.json)<br/><br/><br/>19 ms<br/><br/> | [52.4 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-orin_nano.json)<br/><br/><br/>26 ms<br/><br/> | [357 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-nuc_4060ti.json)<br/><br/><br/>7.2 ms<br/><br/> |
56-
| [Stereo Disparity Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts/isaac_ros_disparity_graph.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [155 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-agx_orin.json)<br/><br/><br/>15 ms<br/><br/> | [74.7 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-orin_nx.json)<br/><br/><br/>21 ms<br/><br/> | [49.0 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-orin_nano.json)<br/><br/><br/>31 ms<br/><br/> | [357 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-nuc_4060ti.json)<br/><br/><br/>3.0 ms<br/><br/> |
52+
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Orin<br/><br/> | Orin NX<br/><br/> | Orin Nano 8GB<br/><br/> | x86_64 w/ RTX 4060 Ti<br/><br/> | x86_64 w/ RTX 4090<br/><br/> |
53+
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
54+
| [Rectify Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_image_proc_benchmark/scripts/isaac_ros_rectify_node.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [983 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-agx_orin.json)<br/><br/><br/>2.5 ms @ 30Hz<br/><br/> | [569 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-orin_nx.json)<br/><br/><br/>3.5 ms @ 30Hz<br/><br/> | [394 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-orin_nano.json)<br/><br/><br/>5.2 ms @ 30Hz<br/><br/> | [2500 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-nuc_4060ti.json)<br/><br/><br/>0.88 ms @ 30Hz<br/><br/> | [2500 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_rectify_node-x86_4090.json)<br/><br/><br/>0.66 ms @ 30Hz<br/><br/> |
55+
| [Stereo Disparity Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_stereo_image_proc_benchmark/scripts/isaac_ros_disparity_node.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [168 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-agx_orin.json)<br/><br/><br/>7.5 ms @ 30Hz<br/><br/> | [75.4 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-orin_nx.json)<br/><br/><br/>15 ms @ 30Hz<br/><br/> | [51.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-orin_nano.json)<br/><br/><br/>22 ms @ 30Hz<br/><br/> | [350 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-nuc_4060ti.json)<br/><br/><br/>3.4 ms @ 30Hz<br/><br/> | [814 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_node-x86_4090.json)<br/><br/><br/>1.8 ms @ 30Hz<br/><br/> |
56+
| [Stereo Disparity Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_stereo_image_proc_benchmark/scripts/isaac_ros_disparity_graph.py)<br/><br/><br/><br/> | 1080p<br/><br/><br/><br/> | [157 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-agx_orin.json)<br/><br/><br/>12 ms @ 30Hz<br/><br/> | [72.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-orin_nx.json)<br/><br/><br/>20 ms @ 30Hz<br/><br/> | [49.6 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-orin_nano.json)<br/><br/><br/>28 ms @ 30Hz<br/><br/> | [349 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-nuc_4060ti.json)<br/><br/><br/>2.5 ms @ 30Hz<br/><br/> | [791 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_disparity_graph-x86_4090.json)<br/><br/><br/>2.7 ms @ 30Hz<br/><br/> |
5757

5858
---
5959

@@ -71,12 +71,11 @@ Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/re
7171
* [Replacing `image_pipeline` with `isaac_ros_image_pipeline`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_image_pipeline/index.html#replacing-image-pipeline-with-isaac-ros-image-pipeline)
7272
* [`isaac_ros_image_proc`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_image_proc/index.html)
7373
* [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_image_proc/index.html#quickstart)
74-
* [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_image_proc/index.html#api)
7574
* [`isaac_ros_stereo_image_proc`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_stereo_image_proc/index.html)
7675
* [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_stereo_image_proc/index.html#quickstart)
7776
* [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_stereo_image_proc/index.html#try-more-examples)
7877
* [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_image_pipeline/isaac_ros_stereo_image_proc/index.html#api)
7978

8079
## Latest
8180

82-
Update 2023-10-18: Introduced confidence thresholding for SGM
81+
Update 2024-05-30: Update to be compatible with JetPack 6.0

‎isaac_ros_depth_image_proc/CMakeLists.txt‎

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
2-
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# Copyright (c) 2023-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33
#
44
# Licensed under the Apache License, Version 2.0 (the "License");
55
# you may not use this file except in compliance with the License.
@@ -29,23 +29,34 @@ ament_auto_find_build_dependencies()
2929
ament_auto_add_library(point_cloud_xyz_node SHARED src/point_cloud_xyz_node.cpp)
3030
rclcpp_components_register_nodes(point_cloud_xyz_node "nvidia::isaac_ros::depth_image_proc::PointCloudXyzNode")
3131
set(node_plugins "${node_plugins}nvidia::isaac_ros::depth_image_proc::PointCloudXyzNode;$<TARGET_FILE:point_cloud_xyz_node>\n")
32+
set_target_properties(point_cloud_xyz_node PROPERTIES
33+
BUILD_WITH_INSTALL_RPATH TRUE
34+
BUILD_RPATH_USE_ORIGIN TRUE
35+
INSTALL_RPATH_USE_LINK_PATH TRUE)
3236

3337
# PointCloudXyzrgbNode
3438
ament_auto_add_library(point_cloud_xyzrgb_node SHARED src/point_cloud_xyzrgb_node.cpp)
3539
rclcpp_components_register_nodes(point_cloud_xyzrgb_node "nvidia::isaac_ros::depth_image_proc::PointCloudXyzrgbNode")
3640
set(node_plugins "${node_plugins}nvidia::isaac_ros::depth_image_proc::PointCloudXyzrgbNode;$<TARGET_FILE:point_cloud_xyzrgb_node>\n")
41+
set_target_properties(point_cloud_xyzrgb_node PROPERTIES
42+
BUILD_WITH_INSTALL_RPATH TRUE
43+
BUILD_RPATH_USE_ORIGIN TRUE
44+
INSTALL_RPATH_USE_LINK_PATH TRUE)
3745

38-
### Install extensions built from source
39-
# depth_image_proc(for DepthToPointCloud)
40-
add_subdirectory(gxf/depth_image_proc)
41-
install(TARGETS gxf_depth_image_proc DESTINATION share/${PROJECT_NAME}/gxf/lib/depth_image_proc)
42-
### End extensions
46+
# convert_metric_node
47+
ament_auto_add_library(convert_metric_node SHARED src/convert_metric_node.cpp)
48+
rclcpp_components_register_nodes(convert_metric_node "nvidia::isaac_ros::depth_image_proc::ConvertMetricNode")
49+
target_link_libraries(convert_metric_node cvcuda nvcv_types)
50+
set(node_plugins "${node_plugins}nvidia::isaac_ros::depth_image_proc::ConvertMetricNode;$<TARGET_FILE:convert_metric_node>\n")
51+
set_target_properties(convert_metric_node PROPERTIES
52+
BUILD_WITH_INSTALL_RPATH TRUE
53+
BUILD_RPATH_USE_ORIGIN TRUE
54+
INSTALL_RPATH_USE_LINK_PATH TRUE)
4355

4456
if(BUILD_TESTING)
4557
find_package(ament_lint_auto REQUIRED)
4658
ament_lint_auto_find_test_dependencies()
4759

48-
4960
# The FindPythonInterp and FindPythonLibs modules are removed
5061
if(POLICY CMP0148)
5162
cmake_policy(SET CMP0148 OLD)
@@ -56,6 +67,7 @@ if(BUILD_TESTING)
5667
add_launch_test(test/isaac_ros_depth_image_proc_point_cloud_xyz_OSS_compare.py)
5768
add_launch_test(test/isaac_ros_depth_image_proc_point_cloud_xyzrgb_pol.py)
5869
add_launch_test(test/isaac_ros_depth_image_proc_point_cloud_xyzrgb_OSS_compare.py)
70+
add_launch_test(test/isaac_ros_depth_image_proc_convert_metric_pol.py)
5971
endif()
6072

61-
ament_auto_package(INSTALL_TO_SHARE config)
73+
ament_auto_package(INSTALL_TO_SHARE launch config)

‎isaac_ros_depth_image_proc/config/nitros_point_cloud_xyz_node.yaml‎

Lines changed: 2 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ components:
2222
type: nvidia::gxf::DoubleBufferReceiver
2323
parameters:
2424
capacity: 12
25-
policy: 0
2625
- type: nvidia::gxf::MessageAvailableSchedulingTerm
2726
parameters:
2827
receiver: depth_image_receiver
@@ -31,24 +30,18 @@ components:
3130
type: nvidia::gxf::DoubleBufferReceiver
3231
parameters:
3332
capacity: 12
34-
policy: 0
3533
- type: nvidia::gxf::MessageAvailableSchedulingTerm
3634
parameters:
3735
receiver: depth_cam_info_receiver
3836
min_size: 1
39-
- type: nvidia::isaac_ros::Synchronization
37+
- type: nvidia::gxf::Synchronization
4038
parameters:
41-
depth_image_receiver: depth_image_receiver
42-
depth_cam_info_receiver: depth_cam_info_receiver
43-
depth_image_transmitter: depth_image_transmitter
44-
depth_cam_info_transmitter: depth_cam_info_transmitter
4539
inputs: [depth_image_receiver, depth_cam_info_receiver]
4640
outputs: [depth_image_transmitter, depth_cam_info_transmitter]
4741
- name: depth_image_transmitter
4842
type: nvidia::gxf::DoubleBufferTransmitter
4943
parameters:
5044
capacity: 12
51-
policy: 0
5245
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
5346
parameters:
5447
transmitter: depth_image_transmitter
@@ -57,7 +50,6 @@ components:
5750
type: nvidia::gxf::DoubleBufferTransmitter
5851
parameters:
5952
capacity: 12
60-
policy: 0
6153
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
6254
parameters:
6355
transmitter: depth_cam_info_transmitter
@@ -69,7 +61,6 @@ components:
6961
type: nvidia::gxf::DoubleBufferReceiver
7062
parameters:
7163
capacity: 1
72-
policy: 0
7364
- type: nvidia::gxf::MessageAvailableSchedulingTerm
7465
parameters:
7566
receiver: video_buffer_input
@@ -78,7 +69,6 @@ components:
7869
type: nvidia::gxf::DoubleBufferReceiver
7970
parameters:
8071
capacity: 1
81-
policy: 0
8272
- type: nvidia::gxf::MessageAvailableSchedulingTerm
8373
parameters:
8474
receiver: camera_model_input
@@ -87,7 +77,6 @@ components:
8777
type: nvidia::gxf::DoubleBufferTransmitter
8878
parameters:
8979
capacity: 1
90-
policy: 0
9180
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
9281
parameters:
9382
transmitter: depth_message_output
@@ -105,7 +94,6 @@ components:
10594
type: nvidia::gxf::DoubleBufferReceiver
10695
parameters:
10796
capacity: 1
108-
policy: 0
10997
- type: nvidia::gxf::MessageAvailableSchedulingTerm
11098
parameters:
11199
receiver: depth_receiver
@@ -114,7 +102,6 @@ components:
114102
type: nvidia::gxf::DoubleBufferTransmitter
115103
parameters:
116104
capacity: 1
117-
policy: 0
118105
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
119106
parameters:
120107
transmitter: point_cloud_transmitter
@@ -143,7 +130,6 @@ components:
143130
type: nvidia::gxf::DoubleBufferReceiver
144131
parameters:
145132
capacity: 2
146-
policy: 0
147133
- type: nvidia::gxf::MessageAvailableSchedulingTerm
148134
parameters:
149135
receiver: signal
@@ -175,11 +161,10 @@ components:
175161
components:
176162
- name: clock
177163
type: nvidia::gxf::RealtimeClock
178-
- type: nvidia::gxf::MultiThreadScheduler
164+
- type: nvidia::gxf::EventBasedScheduler
179165
parameters:
180166
clock: clock
181167
stop_on_deadlock: false
182-
check_recession_period_ms: 1
183168
worker_thread_number: 2
184169
- type: nvidia::gxf::JobStatistics
185170
parameters:

0 commit comments

Comments
 (0)