Skip to content

Commit e5f327c

Browse files
jaiveersinghNVjaiveersinghNV
committed
Isaac ROS 0.31.0 (DP3.1)
1 parent 169cdba commit e5f327c

File tree

14 files changed

+82
-62
lines changed

14 files changed

+82
-62
lines changed

‎README.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,9 @@ This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://dev
2626

2727
The following table summarizes the per-platform performance statistics of sample graphs that use this package, with links included to the full benchmark output. These benchmark configurations are taken from the [Isaac ROS Benchmark](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark#list-of-isaac-ros-benchmarks) collection, based on the [`ros2_benchmark`](https://github.com/NVIDIA-ISAAC-ROS/ros2_benchmark) framework.
2828

29-
| Sample Graph | Input Size | AGX Orin | Orin NX | x86_64 w/ RTX 3060 Ti |
30-
| ---------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
31-
| [Proximity Segmentation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts//isaac_ros_bi3d_node.py) | 576p | [45.9 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-agx_orin.json)<br>32 ms | [26.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-orin_nx.json)<br>65 ms | [148 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-x86_64_rtx_3060Ti.json)<br>22 ms |
29+
| Sample Graph | Input Size | AGX Orin | Orin NX | x86_64 w/ RTX 4060 Ti |
30+
| ---------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
31+
| [Proximity Segmentation Node](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/scripts//isaac_ros_bi3d_node.py) | 576p | [49.3 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-agx_orin.json)<br>32 ms | [24.2 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-orin_nx.json)<br>65 ms | [159 fps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_benchmark/blob/main/results/isaac_ros_bi3d_node-nuc_4060ti.json)<br>23 ms |
3232

3333
## Table of Contents
3434

@@ -66,7 +66,7 @@ The following table summarizes the per-platform performance statistics of sample
6666

6767
## Latest Update
6868

69-
Update 2023-04-05: Source available GXF extensions
69+
Update 2023-05-25: Performance improvements.
7070

7171
## Supported Platforms
7272

@@ -328,6 +328,7 @@ Check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference/blob/ma
328328

329329
| Date | Changes |
330330
| ---------- | ------------------------------------------------------------------ |
331+
| 2023-05-25 | Performance improvements |
331332
| 2023-04-05 | Source available GXF extensions |
332333
| 2022-08-31 | Update to use latest model and to be compatible with JetPack 5.0.2 |
333334
| 2022-06-30 | Initial release |

‎isaac_ros_bi3d/config/bi3d_node.yaml‎

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
%YAML 1.2
22
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
3-
# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# Copyright (c) 2022-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
#
55
# Licensed under the Apache License, Version 2.0 (the "License");
66
# you may not use this file except in compliance with the License.
@@ -252,7 +252,7 @@ components:
252252
featnet_engine_file_path: "featnet_engine_file_path"
253253
featnet_input_layers_name: ["input.1"]
254254
featnet_output_layers_name: ["97"]
255-
255+
256256
segnet_engine_file_path: "segnet_engine_file_path"
257257
segnet_input_layers_name: ["input.1"]
258258
segnet_output_layers_name: ["278"]
@@ -306,7 +306,7 @@ components:
306306
type: nvidia::gxf::BlockMemoryPool
307307
parameters:
308308
storage_type: 1
309-
block_size: 2211840
309+
block_size: 2211840
310310
num_blocks: 12
311311
- name: forward_pool
312312
type: nvidia::gxf::BlockMemoryPool
@@ -348,7 +348,7 @@ components:
348348
featnet_engine_file_path: "featnet_engine_file_path"
349349
featnet_input_layers_name: ["input.1"]
350350
featnet_output_layers_name: ["97"]
351-
351+
352352
segnet_engine_file_path: "segnet_engine_file_path"
353353
segnet_input_layers_name: ["input.1"]
354354
segnet_output_layers_name: ["278"]
@@ -408,7 +408,7 @@ components:
408408
parameters:
409409
rx: rx
410410
tx: tx
411-
---
411+
---
412412
name: bi3d_postprocess
413413
components:
414414
- name: pool
@@ -442,7 +442,7 @@ components:
442442
disparity_tensor_name: "disparity"
443443
disparity_values_tensor_name: "bi3d_inference_disparities"
444444
---
445-
name: bi3d_output_vault
445+
name: bi3d_output_sink
446446
components:
447447
- name: input
448448
type: nvidia::gxf::DoubleBufferReceiver
@@ -453,12 +453,10 @@ components:
453453
parameters:
454454
receiver: input
455455
min_size: 1
456-
- name: vault
457-
type: nvidia::gxf::Vault
456+
- name: sink
457+
type: nvidia::isaac_ros::MessageRelay
458458
parameters:
459459
source: input
460-
max_waiting_count: 1
461-
drop_waiting: false
462460
---
463461
components:
464462
- name: connection
@@ -556,7 +554,7 @@ components:
556554
type: nvidia::gxf::Connection
557555
parameters:
558556
source: bi3d_postprocess/data_transmitter
559-
target: bi3d_output_vault/input
557+
target: bi3d_output_sink/input
560558
---
561559
name: scheduler
562560
components:
@@ -567,4 +565,3 @@ components:
567565
clock: clock
568566
worker_thread_number: 8
569567
stop_on_deadlock: false
570-

‎isaac_ros_bi3d/gxf/bi3d/3dv/src/Bi3DPreProcessor.cpp‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030

3131
namespace cvcore { namespace bi3d {
3232

33-
struct Bi3DPreProcessor::Bi3DPreProcessorImpl
33+
struct Bi3DPreProcessor::Bi3DPreProcessorImpl
3434
{
3535
mutable detail::InputImage m_leftImageDevice;
3636
mutable detail::InputImage m_rightImageDevice;
@@ -46,7 +46,7 @@ struct Bi3DPreProcessor::Bi3DPreProcessorImpl
4646
ImagePreProcessingParams m_preProcessorParams;
4747
ModelInputParams m_modelParams;
4848

49-
Bi3DPreProcessorImpl(const ImagePreProcessingParams & preProcessorParams,
49+
Bi3DPreProcessorImpl(const ImagePreProcessingParams & preProcessorParams,
5050
const ModelInputParams & modelInputParams,
5151
cudaStream_t stream)
5252
: m_preProcessorParams{preProcessorParams},
@@ -82,7 +82,7 @@ struct Bi3DPreProcessor::Bi3DPreProcessorImpl
8282
// Bi3D Frontend
8383
// =============================================================================
8484

85-
Bi3DPreProcessor::Bi3DPreProcessor(const ImagePreProcessingParams & preProcessorParams,
85+
Bi3DPreProcessor::Bi3DPreProcessor(const ImagePreProcessingParams & preProcessorParams,
8686
const ModelInputParams & modelInputParams,
8787
cudaStream_t stream)
8888
: m_pImpl{new Bi3DPreProcessorImpl(preProcessorParams, modelInputParams, stream)} {}
@@ -107,6 +107,7 @@ void Bi3DPreProcessor::execute(detail::PreprocessedImage & preprocessedLeftImage
107107
m_pImpl->resizeBuffers(leftImage.getWidth(), leftImage.getHeight());
108108

109109
// Ensuring data is on the GPU
110+
// FIXME: Rename so this is not confused with cvcore::Copy
110111
Copy(m_pImpl->m_leftImageDevice, leftImage, stream);
111112
Copy(m_pImpl->m_rightImageDevice, rightImage, stream);
112113

@@ -144,6 +145,7 @@ void Bi3DPreProcessor::execute(detail::PreprocessedImage & preprocessedLeftImage
144145
// Ensuring the buffers are appropreately allocated
145146
m_pImpl->resizeBuffers(leftImage.getWidth(), leftImage.getHeight());
146147

148+
// FIXME: Rename so this is not confused with cvcore::Copy
147149
Copy(m_pImpl->m_preLeftImageDevice, leftImage, stream);
148150
Copy(m_pImpl->m_preRightImageDevice, rightImage, stream);
149151

‎isaac_ros_bi3d/gxf/bi3d/cvcore/include/cv/tensor_ops/Errors.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222

2323
namespace cvcore { namespace tensor_ops {
2424

25+
// TODO: Add doxygen style comments for each of the error codes
2526
enum class TensorOpsErrorCode : std::int32_t
2627
{
2728
SUCCESS = 0,

‎isaac_ros_bi3d/gxf/bi3d/cvcore/src/core/utility/ProfileUtils.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ std::string GetCPUName()
6666
#else
6767
std::string GetCPUName()
6868
{
69+
// TODO: this will only work on linux platform
6970
std::ifstream cpuInfo("/proc/cpuinfo");
7071
if (!cpuInfo.good())
7172
{

‎isaac_ros_bi3d/gxf/bi3d/cvcore/src/tensor_ops/DBScan.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ void DBScan::doClusterAndMerge(Array<BBox> &input, Array<BBox> &output, BBoxMerg
180180
doCluster(input, clusters);
181181
output.setSize(m_numClusters);
182182

183-
// merge bboxes based on different modes
183+
// merge bboxes based on different modes (TODO: might add mininum/average in the future)
184184
if (type == MAXIMUM)
185185
{
186186
MergeMaximumBBoxes(input, clusters, output);

‎isaac_ros_bi3d/gxf/bi3d/cvcore/src/tensor_ops/vpi/VPITensorOperators.cpp‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ std::error_code CreateVPIImageWrapperImpl(VPIImage &vpiImg, VPIImageData &imgdat
101101
return make_error_code(vpiStatus);
102102
}
103103

104-
// helper function to wrap VPI image for planar image types
104+
// helper function to wrap VPI image for planar image types (TODO: not supported in vpi so far)
105105
template<ImageType T, typename std::enable_if<IsPlanarImage<T>::value>::type * = nullptr>
106106
std::error_code CreateVPIImageWrapperImpl(VPIImage &vpiImg, VPIImageData &imgdata, const Image<T> &cvcoreImage, VPIBackend backend)
107107
{
@@ -224,6 +224,7 @@ std::error_code DestroyVPIImageWrapper(VPIImage &image, VPIImageData &imageWrap)
224224
}
225225
std::error_code VPITensorStream::Status() noexcept
226226
{
227+
// TODO Needs to be updated for supporting non blocking function calls.
227228
return ErrorCode::SUCCESS;
228229
}
229230

‎isaac_ros_bi3d/gxf/bi3d/cvcore/src/trtbackend/TRTBackend.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -458,6 +458,7 @@ void TRTImpl::loadFromMemoryPointer(void *engine)
458458

459459
void TRTImpl::setupIO(int batchSize)
460460
{
461+
// @TODO: use getBindingDimensions to avoid re-setting the IO.
461462
m_bindingsCount = m_inferenceEngine->getNbBindings();
462463
for (int i = 0; i < m_bindingsCount; i++)
463464
{
@@ -542,6 +543,7 @@ void TRTBackend::infer(void **buffer)
542543

543544
void TRTBackend::infer(void **buffer, int batchSize, cudaStream_t stream)
544545
{
546+
//@TODO: fix kMin, kOpt, kMax batch size in SetupProfile() call and then add a check here.
545547
m_pImpl->setupIO(batchSize);
546548

547549
bool success = true;

‎isaac_ros_bi3d/gxf/bi3d/extensions/tensor_ops/TensorOps.cpp‎

Lines changed: 45 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -30,46 +30,62 @@
3030
#include "gxf/std/extension_factory_helper.hpp"
3131

3232
GXF_EXT_FACTORY_BEGIN()
33-
GXF_EXT_FACTORY_SET_INFO(0x6eae64ff97a94d9b, 0xb324f85e6a98a75a, "NvCvTensorOpsExtension",
34-
"Generic CVCORE tensor_ops interfaces", "Nvidia_Gxf", "3.1.0", "LICENSE");
33+
GXF_EXT_FACTORY_SET_INFO(
34+
0x6eae64ff97a94d9b, 0xb324f85e6a98a75a, "NvCvTensorOpsExtension",
35+
"Generic CVCORE tensor_ops interfaces", "Nvidia_Gxf", "3.1.0", "LICENSE");
3536

36-
GXF_EXT_FACTORY_ADD(0xd073a92344ba4b81, 0xbd0f18f4996048e9, nvidia::cvcore::tensor_ops::CameraModel,
37-
nvidia::gxf::Component,
38-
"Construct Camera distortion model / Camera intrinsic compatible with CVCORE");
37+
GXF_EXT_FACTORY_ADD(
38+
0xd073a92344ba4b82, 0xbd0f18f4996048e8, nvidia::cvcore::tensor_ops::CameraModel,
39+
nvidia::gxf::Component,
40+
"Construct Camera distortion model / Camera intrinsic compatible with CVCORE");
3941

40-
GXF_EXT_FACTORY_ADD(0x6c9419223e4b4c2b, 0x899a4d65279c6507, nvidia::cvcore::tensor_ops::Frame3D, nvidia::gxf::Component,
41-
"Construct Camera extrinsic compatible with CVCORE");
42+
GXF_EXT_FACTORY_ADD(
43+
0x6c9419223e4b4c2c, 0x899a4d65279c6508, nvidia::cvcore::tensor_ops::Frame3D,
44+
nvidia::gxf::Component,
45+
"Construct Camera extrinsic compatible with CVCORE");
4246

43-
GXF_EXT_FACTORY_ADD(0xd94385e5b35b4634, 0x9adb0d214a3865f6, nvidia::cvcore::tensor_ops::TensorStream,
44-
nvidia::gxf::Component, "Wrapper of CVCORE ITensorOperatorStream/ITensorOperatorContext");
47+
GXF_EXT_FACTORY_ADD(
48+
0xd94385e5b35b4635, 0x9adb0d214a3865f7, nvidia::cvcore::tensor_ops::TensorStream,
49+
nvidia::gxf::Component, "Wrapper of CVCORE ITensorOperatorStream/ITensorOperatorContext");
4550

46-
GXF_EXT_FACTORY_ADD(0xd0c4ddad486a4a91, 0xb69c8a5304b205ef, nvidia::cvcore::tensor_ops::ImageAdapter,
47-
nvidia::gxf::Component, "Utility component for conversion between message and cvcore image type");
51+
GXF_EXT_FACTORY_ADD(
52+
0xd0c4ddad486a4a92, 0xb69c8a5304b205ea, nvidia::cvcore::tensor_ops::ImageAdapter,
53+
nvidia::gxf::Component, "Utility component for conversion between message and cvcore image type");
4854

49-
GXF_EXT_FACTORY_ADD(0xadebc792bd0b4a56, 0x99c1405fd2ea0727, nvidia::cvcore::tensor_ops::StreamUndistort,
50-
nvidia::gxf::Codelet, "Codelet for stream image undistortion in tensor_ops");
55+
GXF_EXT_FACTORY_ADD(
56+
0xadebc792bd0b4a57, 0x99c1405fd2ea0728, nvidia::cvcore::tensor_ops::StreamUndistort,
57+
nvidia::gxf::Codelet, "Codelet for stream image undistortion in tensor_ops");
5158

52-
GXF_EXT_FACTORY_ADD(0xa58141ac7eca4ea5, 0x9b545446fe379a11, nvidia::cvcore::tensor_ops::Resize, nvidia::gxf::Codelet,
53-
"Codelet for image resizing in tensor_ops");
59+
GXF_EXT_FACTORY_ADD(
60+
0xa58141ac7eca4ea6, 0x9b545446fe379a12, nvidia::cvcore::tensor_ops::Resize, nvidia::gxf::Codelet,
61+
"Codelet for image resizing in tensor_ops");
5462

55-
GXF_EXT_FACTORY_ADD(0xeb8b5f5b36d44b48, 0x81f959fd28e6f677, nvidia::cvcore::tensor_ops::StreamResize,
56-
nvidia::gxf::Codelet, "Codelet for stream image resizing in tensor_ops");
63+
GXF_EXT_FACTORY_ADD(
64+
0xeb8b5f5b36d44b49, 0x81f959fd28e6f678, nvidia::cvcore::tensor_ops::StreamResize,
65+
nvidia::gxf::Codelet, "Codelet for stream image resizing in tensor_ops");
5766

58-
GXF_EXT_FACTORY_ADD(0x4a7ff422de3841bc, 0x9e743ac10d9294b6, nvidia::cvcore::tensor_ops::CropAndResize,
59-
nvidia::gxf::Codelet, "Codelet for crop and resizing operation in tensor_ops");
67+
GXF_EXT_FACTORY_ADD(
68+
0x4a7ff422de3841bd, 0x9e743ac10d9294b7, nvidia::cvcore::tensor_ops::CropAndResize,
69+
nvidia::gxf::Codelet, "Codelet for crop and resizing operation in tensor_ops");
6070

61-
GXF_EXT_FACTORY_ADD(0x7018f0b9034c462b, 0xa9fbaf7ee012974f, nvidia::cvcore::tensor_ops::Normalize, nvidia::gxf::Codelet,
62-
"Codelet for image normalization in tensor_ops");
71+
GXF_EXT_FACTORY_ADD(
72+
0x7018f0b9034c462c, 0xa9fbaf7ee012974a, nvidia::cvcore::tensor_ops::Normalize,
73+
nvidia::gxf::Codelet,
74+
"Codelet for image normalization in tensor_ops");
6375

64-
GXF_EXT_FACTORY_ADD(0x269d4237f3c3479d, 0xbcca9ecc44c71a70, nvidia::cvcore::tensor_ops::InterleavedToPlanar,
65-
nvidia::gxf::Codelet, "Codelet for convert interleaved image to planar image in tensor_ops");
76+
GXF_EXT_FACTORY_ADD(
77+
0x269d4237f3c3479e, 0xbcca9ecc44c71a71, nvidia::cvcore::tensor_ops::InterleavedToPlanar,
78+
nvidia::gxf::Codelet, "Codelet for convert interleaved image to planar image in tensor_ops");
6679

67-
GXF_EXT_FACTORY_ADD(0xfc4d7b4d8fcc4daa, 0xa286056e0fcafa78, nvidia::cvcore::tensor_ops::ConvertColorFormat,
68-
nvidia::gxf::Codelet, "Codelet for image color conversion in tensor_ops");
80+
GXF_EXT_FACTORY_ADD(
81+
0xfc4d7b4d8fcc4dab, 0xa286056e0fcafa79, nvidia::cvcore::tensor_ops::ConvertColorFormat,
82+
nvidia::gxf::Codelet, "Codelet for image color conversion in tensor_ops");
6983

70-
GXF_EXT_FACTORY_ADD(0x5ab4a4d8f7a34552, 0xa90be52660b076fd, nvidia::cvcore::tensor_ops::StreamConvertColorFormat,
71-
nvidia::gxf::Codelet, "Codelet for stream image color conversion in tensor_ops");
84+
GXF_EXT_FACTORY_ADD(
85+
0x5ab4a4d8f7a34553, 0xa90be52660b076fe, nvidia::cvcore::tensor_ops::StreamConvertColorFormat,
86+
nvidia::gxf::Codelet, "Codelet for stream image color conversion in tensor_ops");
7287

73-
GXF_EXT_FACTORY_ADD(0x26789b7d5a8d4e84, 0x86b845ec5f4cd12a, nvidia::cvcore::tensor_ops::Reshape, nvidia::gxf::Codelet,
74-
"Codelet for image reshape in tensor_ops");
88+
GXF_EXT_FACTORY_ADD(
89+
0x26789b7d5a8d4e85, 0x86b845ec5f4cd12b, nvidia::cvcore::tensor_ops::Reshape, nvidia::gxf::Codelet,
90+
"Codelet for image reshape in tensor_ops");
7591
GXF_EXT_FACTORY_END()

‎isaac_ros_bi3d/package.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0
2121
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
2222
<package format="3">
2323
<name>isaac_ros_bi3d</name>
24-
<version>0.30.0</version>
24+
<version>0.31.0</version>
2525
<description>Bi3D inference network for ROS</description>
2626

2727
<maintainer email="[email protected]">Hemal Shah</maintainer>

0 commit comments

Comments
 (0)