Skip to content

Commit 9caca61

Browse files
Enhance FoundationPose dynamic mesh reloading support
1 parent 2faf5ad commit 9caca61

15 files changed

+295
-129
lines changed

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Deep learned, NVIDIA-accelerated 3D object pose estimation
44

5-
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/" width="400px"/></a></div>
5+
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-3.2/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-3.2/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/dope_objects.png/" width="400px"/></a></div>
66

77
## Overview
88

@@ -20,7 +20,7 @@ estimate the pose of an object. The output prediction can be used by
2020
perception functions when fusing with the corresponding depth to provide
2121
the 3D pose of an object and distance for navigation or manipulation.
2222

23-
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/" width="500px"/></a></div>
23+
<div align="center"><a class="reference internal image-reference" href="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-3.2/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/"><img alt="image" src="https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/release-3.2/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_pose_estimation_nodegraph.png/" width="500px"/></a></div>
2424

2525
`isaac_ros_foundationpose` is used in a graph of nodes to estimate the pose of
2626
a novel object using 3D bounding cuboid dimensions. It’s developed on top of
@@ -77,15 +77,15 @@ Packages in this repository rely on accelerated DNN model inference
7777
using [Triton](https://github.com/triton-inference-server/server) or
7878
[TensorRT](https://developer.nvidia.com/tensorrt) from [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference).
7979
For preprocessing, packages in this rely on the `Isaac ROS DNN Image Encoder`,
80-
which can also be found at [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/main/isaac_ros_dnn_image_encoder).
80+
which can also be found at [Isaac ROS DNN Inference](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/release-3.2/isaac_ros_dnn_image_encoder).
8181

8282
## Performance
8383

84-
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Orin<br/><br/> | Orin NX<br/><br/> | Orin Nano Super 8GB<br/><br/> | x86_64 w/ RTX 4090<br/><br/> |
85-
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
86-
| [FoundationPose Pose Estimation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_foundationpose_benchmark/scripts/isaac_ros_foundationpose_node.py)<br/><br/><br/><br/> | 720p<br/><br/><br/><br/> | [1.54 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_foundationpose_node-agx_orin.json)<br/><br/><br/>780 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | –<br/><br/><br/><br/> | [9.56 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_foundationpose_node-x86-4090.json)<br/><br/><br/>110 ms @ 30Hz<br/><br/> |
87-
| [DOPE Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_dope_benchmark/scripts/isaac_ros_dope_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [27.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-agx_orin.json)<br/><br/><br/>54 ms @ 30Hz<br/><br/> | [15.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-orin_nx.json)<br/><br/><br/>73 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | [186 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_dope_graph-x86-4090.json)<br/><br/><br/>12 ms @ 30Hz<br/><br/> |
88-
| [Centerpose Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/benchmarks/isaac_ros_centerpose_benchmark/scripts/isaac_ros_centerpose_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [44.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-agx_orin.json)<br/><br/><br/>43 ms @ 30Hz<br/><br/> | [29.0 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-orin_nx.json)<br/><br/><br/>51 ms @ 30Hz<br/><br/> | [29.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-orin_nano.json)<br/><br/><br/>50 ms @ 30Hz<br/><br/> | [50.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_centerpose_graph-x86-4090.json)<br/><br/><br/>14 ms @ 30Hz<br/><br/> |
84+
| Sample Graph<br/><br/> | Input Size<br/><br/> | AGX Orin<br/><br/> | Orin NX<br/><br/> | Orin Nano Super 8GB<br/><br/> | x86_64 w/ RTX 4090<br/><br/> |
85+
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
86+
| [FoundationPose Pose Estimation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/benchmarks/isaac_ros_foundationpose_benchmark/scripts/isaac_ros_foundationpose_node.py)<br/><br/><br/><br/> | 720p<br/><br/><br/><br/> | [1.54 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_foundationpose_node-agx_orin.json)<br/><br/><br/>780 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | –<br/><br/><br/><br/> | [9.56 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_foundationpose_node-x86-4090.json)<br/><br/><br/>110 ms @ 30Hz<br/><br/> |
87+
| [DOPE Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/benchmarks/isaac_ros_dope_benchmark/scripts/isaac_ros_dope_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [27.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_dope_graph-agx_orin.json)<br/><br/><br/>54 ms @ 30Hz<br/><br/> | [15.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_dope_graph-orin_nx.json)<br/><br/><br/>73 ms @ 30Hz<br/><br/> | –<br/><br/><br/><br/> | [186 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_dope_graph-x86-4090.json)<br/><br/><br/>12 ms @ 30Hz<br/><br/> |
88+
| [Centerpose Pose Estimation Graph](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/benchmarks/isaac_ros_centerpose_benchmark/scripts/isaac_ros_centerpose_graph.py)<br/><br/><br/><br/> | VGA<br/><br/><br/><br/> | [44.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_centerpose_graph-agx_orin.json)<br/><br/><br/>43 ms @ 30Hz<br/><br/> | [29.0 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_centerpose_graph-orin_nx.json)<br/><br/><br/>51 ms @ 30Hz<br/><br/> | [29.8 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_centerpose_graph-orin_nano.json)<br/><br/><br/>50 ms @ 30Hz<br/><br/> | [50.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/release-3.2/results/isaac_ros_centerpose_graph-x86-4090.json)<br/><br/><br/>14 ms @ 30Hz<br/><br/> |
8989

9090
---
9191

isaac_ros_foundationpose/package.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
2222
<package format="3">
2323
<name>isaac_ros_foundationpose</name>
24-
<version>3.2.10</version>
24+
<version>3.2.14</version>
2525
<description>FoundationPose 6D pose estimation</description>
2626

2727
<maintainer email="[email protected]">Isaac ROS Maintainers</maintainer>
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
2+
# Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
# SPDX-License-Identifier: Apache-2.0
17+
18+
import os
19+
import subprocess
20+
21+
22+
# Model and engine configuration.
23+
MESH_FILE_NAME = 'textured_simple.obj'
24+
25+
REFINE_MODEL_NAME = 'dummy_refine_model.onnx'
26+
REFINE_ENGINE_NAME = 'dummy_refine_trt_engine.plan'
27+
SCORE_MODEL_NAME = 'dummy_score_model.onnx'
28+
SCORE_ENGINE_NAME = 'dummy_score_trt_engine.plan'
29+
30+
REFINE_ENGINE_PATH = '/tmp/' + REFINE_ENGINE_NAME
31+
SCORE_ENGINE_PATH = '/tmp/' + SCORE_ENGINE_NAME
32+
33+
34+
def generate_tensorrt_engine(engine_path, model_name, trtexec_args):
35+
"""
36+
Generate TensorRT engine file from ONNX model using trtexec.
37+
38+
Parameters
39+
----------
40+
engine_path : str
41+
Path where the engine file will be saved
42+
model_name : str
43+
Name of the model for logging purposes
44+
trtexec_args : list
45+
List of trtexec command arguments
46+
47+
"""
48+
if not os.path.isfile(engine_path):
49+
print(f'Generating an engine file for the {model_name} model...')
50+
# Prepend 'trtexec' to the arguments list.
51+
cmd = ['/usr/src/tensorrt/bin/trtexec'] + trtexec_args
52+
print('Generating model engine file by command: ', ' '.join(cmd))
53+
result = subprocess.run(
54+
cmd,
55+
env=os.environ,
56+
stdout=subprocess.PIPE,
57+
stderr=subprocess.PIPE
58+
)
59+
if result.returncode != 0:
60+
raise Exception(
61+
f'Failed to convert with status: {result.returncode}.\n'
62+
f'stderr:\n' + result.stderr.decode('utf-8')
63+
)
64+
print(f'{model_name} model engine file generation was finished')
65+
66+
67+
def generate_foundationpose_engines():
68+
"""
69+
Generate both Refine and Score TensorRT engine files for FoundationPose tests.
70+
71+
This function should be called at the beginning of generate_test_description()
72+
in FoundationPose test files.
73+
"""
74+
# Get correct model paths (relative to test directory).
75+
base_path = os.path.dirname(__file__)
76+
refine_model_path = os.path.join(base_path, '../../test/models', REFINE_MODEL_NAME)
77+
score_model_path = os.path.join(base_path, '../../test/models', SCORE_MODEL_NAME)
78+
79+
# Generate Refine engine.
80+
refine_trtexec_args = [
81+
f'--onnx={refine_model_path}',
82+
f'--saveEngine={REFINE_ENGINE_PATH}',
83+
'--minShapes=input1:1x160x160x6,input2:1x160x160x6',
84+
'--optShapes=input1:1x160x160x6,input2:1x160x160x6',
85+
'--maxShapes=input1:42x160x160x6,input2:42x160x160x6',
86+
'--fp16',
87+
'--skipInference',
88+
]
89+
generate_tensorrt_engine(REFINE_ENGINE_PATH, 'Refine', refine_trtexec_args)
90+
91+
# Generate Score engine.
92+
score_trtexec_args = [
93+
f'--onnx={score_model_path}',
94+
f'--saveEngine={SCORE_ENGINE_PATH}',
95+
'--fp16',
96+
'--minShapes=input1:1x160x160x6,input2:1x160x160x6',
97+
'--optShapes=input1:1x160x160x6,input2:1x160x160x6',
98+
'--maxShapes=input1:252x160x160x6,input2:252x160x160x6',
99+
'--skipInference',
100+
]
101+
generate_tensorrt_engine(SCORE_ENGINE_PATH, 'Score', score_trtexec_args)
102+
103+
104+
def get_engines():
105+
"""
106+
Get model and engine names and paths for FoundationPose tests.
107+
108+
Returns
109+
-------
110+
dict
111+
Dictionary containing model and engine names and paths
112+
113+
"""
114+
# Calculate correct model paths (relative to test directory).
115+
base_path = os.path.dirname(__file__)
116+
refine_model_path = os.path.join(base_path, '../../test/models', REFINE_MODEL_NAME)
117+
score_model_path = os.path.join(base_path, '../../test/models', SCORE_MODEL_NAME)
118+
119+
return {
120+
'refine_model_name': REFINE_MODEL_NAME,
121+
'refine_engine_name': REFINE_ENGINE_NAME,
122+
'score_model_name': SCORE_MODEL_NAME,
123+
'score_engine_name': SCORE_ENGINE_NAME,
124+
'refine_model_path': refine_model_path,
125+
'refine_engine_path': REFINE_ENGINE_PATH,
126+
'score_model_path': score_model_path,
127+
'score_engine_path': SCORE_ENGINE_PATH,
128+
}

0 commit comments

Comments
 (0)