Skip to content

Commit c97c42a

Browse files
Isaac ROS 4.0
1 parent a4f9b05 commit c97c42a

File tree

63 files changed

+744
-1791
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+744
-1791
lines changed

README.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Deep learned, NVIDIA-accelerated 3D object pose estimation
44

5-
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/" width="400px"/></a></div>
5+
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-4.0/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-4.0/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/" width="400px"/></a></div>
66

77
## Overview
88

@@ -20,7 +20,7 @@ estimate the pose of an object. The output prediction can be used by
2020
perception functions when fusing with the corresponding depth to provide
2121
the 3D pose of an object and distance for navigation or manipulation.
2222

23-
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/" width="500px"/></a></div>
23+
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-4.0/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-4.0/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/" width="500px"/></a></div>
2424

2525
`isaac_ros_foundationpose` is used in a graph of nodes to estimate the pose of
2626
a novel object using 3D bounding cuboid dimensions. It’s developed on top of
@@ -34,7 +34,7 @@ The refine model processes initial pose hypotheses, iteratively refining them, t
3434
passes these refined hypotheses to the score model, which selects and finalizes the pose estimation.
3535
Additionally, the refine model can serve for tracking, that updates the pose estimation based on
3636
new image inputs and the previous frame’s pose estimate. This tracking process is more efficient
37-
compared to pose estimation, which speeds exceeding 120 FPS on the Jetson Orin platform.
37+
compared to pose estimation, which speeds exceeding 120 FPS on the Jetson Thor platform.
3838

3939
`isaac_ros_dope` is used in a graph of nodes to estimate the pose of a
4040
known object with 3D bounding cuboid dimensions. To produce the
@@ -77,15 +77,15 @@ Packages in this repository rely on accelerated DNN model inference
7777
using [Triton](https://github.com/triton-inference-server/server) or
7878
[TensorRT](https://developer.nvidia.com/tensorrt) from [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference).
7979
For preprocessing, packages in this rely on the `Isaac ROS DNN Image Encoder`,
80-
which can also be found at [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/isaac_ros_dnn_image_encoder).
80+
which can also be found at [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/release-4.0/isaac_ros_dnn_image_encoder).
8181

8282
## Performance
8383

84-
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Orin<br/><br/> | Orin NX<br/><br/> | Orin Nano Super 8GB<br/><br/> | x86_64 w/ RTX 4090<br/><br/> |
85-
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
86-
| [FoundationPose Pose Estimation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_foundationpose_benchmark/scripts/isaac_ros_foundationpose_node.py)<br/><br/><br/><br/> | 720p<br/><br/><br/><br/> | [1.54 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_foundationpose_node-agx_orin.json)<br/><br/><br/>780 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | –<br/><br/><br/><br/> | [9.56 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_foundationpose_node-x86-4090.json)<br/><br/><br/>110 ms @ 30Hz<br/><br/> |
87-
| [DOPE Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_dope_benchmark/scripts/isaac_ros_dope_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [27.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-agx_orin.json)<br/><br/><br/>54 ms @ 30Hz<br/><br/> | [15.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-orin_nx.json)<br/><br/><br/>73 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | [186 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-x86-4090.json)<br/><br/><br/>12 ms @ 30Hz<br/><br/> |
88-
| [Centerpose Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_centerpose_benchmark/scripts/isaac_ros_centerpose_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [44.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-agx_orin.json)<br/><br/><br/>43 ms @ 30Hz<br/><br/> | [29.0 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-orin_nx.json)<br/><br/><br/>51 ms @ 30Hz<br/><br/> | [29.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-orin_nano.json)<br/><br/><br/>50 ms @ 30Hz<br/><br/> | [50.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-x86-4090.json)<br/><br/><br/>14 ms @ 30Hz<br/><br/> |
84+
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Thor<br/><br/> | x86_64 w/ RTX 5090<br/><br/> |
85+
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
86+
| [FoundationPose Pose Estimation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/benchmarks/isaac_ros_foundationpose_benchmark/scripts/isaac_ros_foundationpose_node.py)<br/><br/> | 720p<br/><br/> | [3.92 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_foundationpose_node-agx_thor.json)<br/><br/><br/>260 ms @ 30Hz<br/><br/> | [10.1 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_foundationpose_node-x86-5090.json)<br/><br/><br/>89 ms @ 30Hz<br/><br/> |
87+
| [DOPE Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/benchmarks/isaac_ros_dope_benchmark/scripts/isaac_ros_dope_graph.py)<br/><br/> | VGA<br/><br/> | [138 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_dope_graph-agx_thor.json)<br/><br/><br/>24 ms @ 30Hz<br/><br/> | [199 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_dope_graph-x86-5090.json)<br/><br/><br/>14 ms @ 30Hz<br/><br/> |
88+
| [Centerpose Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/benchmarks/isaac_ros_centerpose_benchmark/scripts/isaac_ros_centerpose_graph.py)<br/><br/> | VGA<br/><br/> | [50.5 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_centerpose_graph-agx_thor.json)<br/><br/><br/>50 ms @ 30Hz<br/><br/> | [50.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-4.0/results/isaac_ros_centerpose_graph-x86-5090.json)<br/><br/><br/>16 ms @ 30Hz<br/><br/> |
8989

9090
---
9191

@@ -109,10 +109,11 @@ Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/re
109109
* [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_dope/index.html#api)
110110
* [`isaac_ros_foundationpose`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html)
111111
* [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#quickstart)
112+
* [Visualize Results](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#visualize-results)
112113
* [Try More Examples](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#try-more-examples)
113114
* [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#troubleshooting)
114115
* [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#api)
115116

116117
## Latest
117118

118-
Update 2024-12-10: Added pose estimate post-processing utilities
119+
Update 2025-10-24: Added synchronization node tuned for real-time performance and minor FoundationPose model update for TensorRT 10.13

SECURITY.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
## Security
2+
3+
NVIDIA is dedicated to the security and trust of our software products and services, including all source code repositories managed through our organization.
4+
5+
If you need to report a security issue, please use the appropriate contact points outlined below. **Please do not report security vulnerabilities through GitHub.**
6+
7+
## Reporting Potential Security Vulnerability in an NVIDIA Product
8+
9+
To report a potential security vulnerability in any NVIDIA product:
10+
- Web: [Security Vulnerability Submission Form](https://www.nvidia.com/object/submit-security-vulnerability.html)
11+
12+
- We encourage you to use the following PGP key for secure email communication: [NVIDIA public PGP Key for communication](https://www.nvidia.com/en-us/security/pgp-key)
13+
- Please include the following information:
14+
- Product/Driver name and version/branch that contains the vulnerability
15+
- Type of vulnerability (code execution, denial of service, buffer overflow, etc.)
16+
- Instructions to reproduce the vulnerability
17+
- Proof-of-concept or exploit code
18+
- Potential impact of the vulnerability, including how an attacker could exploit the vulnerability
19+
20+
While NVIDIA currently does not have a bug bounty program, we do offer acknowledgement when an externally reported security issue is addressed under our coordinated vulnerability disclosure policy. Please visit our [Product Security Incident Response Team (PSIRT)](https://www.nvidia.com/en-us/security/psirt-policies/) policies page for more information.
21+
22+
## NVIDIA Product Security
23+
24+
For all security-related concerns, please visit NVIDIA's Product Security portal at https://www.nvidia.com/en-us/security

isaac_ros_centerpose/CMakeLists.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,8 @@ if(BUILD_TESTING)
5656
set(CMAKE_IGNORE_PATH "/usr/include/gmock" ${CMAKE_IGNORE_PATH})
5757

5858
# Explicitly use ROS2 vendor googletest/gmock
59-
include_directories(BEFORE /opt/ros/humble/src/gtest_vendor/include)
60-
include_directories(BEFORE /opt/ros/humble/src/gmock_vendor/include)
59+
include_directories(BEFORE /opt/ros/$ENV{ROS_DISTRO}/src/gtest_vendor/include)
60+
include_directories(BEFORE /opt/ros/$ENV{ROS_DISTRO}/src/gmock_vendor/include)
6161

6262
# Gtest for decoder_node
6363
ament_add_gtest(centerpose_decoder_node_test test/centerpose_decoder_node_test.cpp)

isaac_ros_centerpose/config/centerpose_node.yaml

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
%YAML 1.2
22
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
3-
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -32,6 +32,15 @@ components:
3232
capacity: 12
3333
- name: allocator
3434
type: nvidia::gxf::UnboundedAllocator
35+
- name: stream_pool
36+
type: nvidia::gxf::CudaStreamPool
37+
parameters:
38+
dev_id: 0
39+
stream_flags: 1
40+
stream_priority: 0
41+
reserved_size: 1
42+
max_size: 1
43+
nvtx_identifier: "centerpose_decoder_stream_pool"
3544
- name: centerpose_decoder
3645
type: nvidia::isaac::centerpose::CenterPosePostProcessor
3746
parameters:
@@ -42,6 +51,7 @@ components:
4251
output_field_size: []
4352
cuboid_scaling_factor: 0.0
4453
score_threshold: 1.0
54+
stream_pool: stream_pool
4555
- type: nvidia::gxf::MessageAvailableSchedulingTerm
4656
parameters:
4757
receiver: input
@@ -65,12 +75,22 @@ components:
6575
type: nvidia::gxf::DoubleBufferTransmitter
6676
parameters:
6777
capacity: 12
78+
- name: stream_pool
79+
type: nvidia::gxf::CudaStreamPool
80+
parameters:
81+
dev_id: 0
82+
stream_flags: 1
83+
stream_priority: 0
84+
reserved_size: 1
85+
max_size: 1
86+
nvtx_identifier: "centerpose_decoder_to_isaac_stream_pool"
6887
- name: centerpose_decoder_to_isaac
6988
type: nvidia::isaac::centerpose::CenterPoseDetectionToIsaac
7089
parameters:
7190
input: input
7291
output: output
7392
object_name: ""
93+
stream_pool: stream_pool
7494
- type: nvidia::gxf::MessageAvailableSchedulingTerm
7595
parameters:
7696
receiver: input
@@ -116,6 +136,7 @@ components:
116136
stop_on_deadlock: false
117137
check_recession_period_ms: 1
118138
worker_thread_number: 2
139+
worker_thread_name_id: "centerpose_node"
119140
- type: nvidia::gxf::JobStatistics
120141
parameters:
121142
clock: clock

isaac_ros_centerpose/config/centerpose_visualizer_node.yaml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
%YAML 1.2
22
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
3-
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -36,6 +36,15 @@ components:
3636
capacity: 12
3737
- name: allocator
3838
type: nvidia::gxf::UnboundedAllocator
39+
- name: stream_pool
40+
type: nvidia::gxf::CudaStreamPool
41+
parameters:
42+
dev_id: 0
43+
stream_flags: 1
44+
stream_priority: 0
45+
reserved_size: 1
46+
max_size: 1
47+
nvtx_identifier: "centerpose_visualizer_stream_pool"
3948
- name: centerpose_visualizer
4049
type: nvidia::isaac::centerpose::CenterPoseVisualizer
4150
parameters:
@@ -46,6 +55,7 @@ components:
4655
allocator: allocator
4756
show_axes: true
4857
bounding_box_color: 0x00
58+
stream_pool: stream_pool
4959
- type: nvidia::gxf::MessageAvailableSchedulingTerm
5060
parameters:
5161
receiver: video_buffer_input
@@ -94,6 +104,7 @@ components:
94104
stop_on_deadlock: false
95105
check_recession_period_ms: 1
96106
worker_thread_number: 2
107+
worker_thread_name_id: "centerpose_visualizer_node"
97108
- type: nvidia::gxf::JobStatistics
98109
parameters:
99110
clock: clock

isaac_ros_centerpose/package.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
2222
<package format="3">
2323
<name>isaac_ros_centerpose</name>
24-
<version>3.2.10</version>
24+
<version>4.0.0</version>
2525
<description>CenterPose: Pose Estimation using Deep Learning</description>
2626

2727
<maintainer email="[email protected]">Isaac ROS Maintainers</maintainer>

isaac_ros_centerpose/test/test_centerpose_pol.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,8 @@ def generate_test_description():
9494
'kps_heatmap_mean'],
9595
'output_tensor_formats': ['nitros_tensor_list_nhwc_rgb_f32'],
9696
'verbose': False,
97-
'force_engine_update': False
97+
'force_engine_update': False,
98+
'max_workspace_size': 512*1024*1024, # 512MB
9899
}],
99100
)
100101

isaac_ros_dope/config/dope_node.yaml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
%YAML 1.2
22
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
3-
# Copyright (c) 2022-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -32,6 +32,15 @@ components:
3232
capacity: 12
3333
- name: allocator
3434
type: nvidia::gxf::UnboundedAllocator
35+
- name: stream_pool
36+
type: nvidia::gxf::CudaStreamPool
37+
parameters:
38+
dev_id: 0
39+
stream_flags: 1
40+
stream_priority: 0
41+
reserved_size: 1
42+
max_size: 1
43+
nvtx_identifier: "dope_decoder_stream_pool"
3544
- name: dope_decoder
3645
type: nvidia::isaac_ros::dope::DopeDecoder
3746
parameters:
@@ -41,6 +50,7 @@ components:
4150
allocator: allocator
4251
object_dimensions: []
4352
object_name: ""
53+
stream_pool: stream_pool
4454
- type: nvidia::gxf::MessageAvailableSchedulingTerm
4555
parameters:
4656
receiver: tensorlist_in
@@ -85,6 +95,7 @@ components:
8595
stop_on_deadlock: false
8696
check_recession_period_ms: 1
8797
worker_thread_number: 2
98+
worker_thread_name_id: "dope_node"
8899
- type: nvidia::gxf::JobStatistics
89100
parameters:
90101
clock: clock

0 commit comments

Comments
 (0)