Merge pull request #18 from NVIDIA-ISAAC-ROS/release-dp3

Isaac ROS 0.30.0 (DP3)

Author: jaiveersinghNV

.gitattributes

@@ -1,3 +1,6 @@
+# Ignore Python files in linguist
+*.py linguist-detectable=false
+
 # Images
 *.gif filter=lfs diff=lfs merge=lfs -text
 *.jpg filter=lfs diff=lfs merge=lfs -text
@@ -17,5 +20,6 @@
 *.so.* filter=lfs diff=lfs merge=lfs -text
 
 # ROS Bags
+**/resources/**/*.zstd filter=lfs diff=lfs merge=lfs -text
 **/resources/**/*.db3 filter=lfs diff=lfs merge=lfs -text
 **/resources/**/*.yaml filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,10 +1,28 @@
 # Isaac ROS Argus Camera
 
-<div align="center"><img src="resources/100_library_left.JPG" width="350" title="HAWK Stereo Left"/><img src="resources/100_library_right.JPG" width="350" title="HAWK Stereo Right"/></div>
+<div align="center"><img src="resources/isaac_ros_argus_sample_raw.png" width="300" title="raw image"/> <img src="resources/isaac_ros_argus_sample_isp.png" width="300" title="isp processed image"/></div>
 
 ## Overview
+The Isaac ROS Argus Camera module contains an ROS 2 package for sensor processing to output images. Image sensors are connected on CSI and GMSL hardware interfaces to Jetson platforms. This package uses dedicated hardware engines to accelerate image processing. Output images are used in graphs of nodes for AI and CV perception packages, image compression for capture to disk by event recorders, and live-stream visuals for remote robot teleoperation.
+
+Isaac ROS Argus Camera provides with several sensor capture and processing features, including AWB (auto-white-balance), AE (auto-exposure), and noise reduction. Leveraging hardware engines in Jetson, Argus provides multi-camera frame synchronization, with very high precision frame acquisition timestamping and jitter less than 100us.
+
+<div align="center"><img src="resources/isaac_ros_argus_camera_nodegraph.png" width="800" title="graph of nodes with Argus Camera"/></div>
+
+In the example graph of nodes above, the Argus Camera module processes sensor image data from the camera for input to vision-based perception graphs, including DNN stereo disparity, AprilTag, VSLAM, and H.264 encode. Each of the nodes in green is GPU accelerated for a high-performance compute graph from Argus camera to vision-based perception functions.
+
+<div align="center"><img src="resources/isaac_ros_argus_camera_zeromemcpy.png" width="800" title="zero CPU memcpy with Argus Camera"/></div>
+
+Argus Camera uses dedicated hardware engines to access the full memory bandwidth in Jetson. Raw camera images are delivered via CSI or GMSL interfaces directly to the GPU accelerated memory. The ISP hardware processes the raw image directly into a GPU accelerated output image topic.
+
+Widely available USB and Ethernet plug-in cameras can be used for robotics applications, but there is performance cost for this convenience. The I/O interface (USB or Ethernet) places the image from the camera directly into CPU-accessible memory. The camera driver makes a copy from the I/O interface using the CPU to make the image available to other applications. The Camera driver wrapper node in ROS performs another memcpy with the CPU from the driver to publish the image in ROS. Before a USB or Ethernet image arrives as a published topic, two CPU memcpy actions have been perfromed for every pixel. In contrast, the Argus Camera module processes sensor data into output image topics in ROS without the CPU touching a single pixel in the image.
+
+> **Note**: Argus Camera outputs sensor_msgs/Image at the sensor data rate, subject to performance capabilites of the Jetson platform(s). For example, a [Hawk camera](https://www.leopardimaging.com/li-ar0234cs-stereo-gmsl2-hawk/) configured for 30fps (frames per second) stereo 1920x1080 will output time-synchronized left and right camera frames sensor_msgs/Image at 30fps.
+
+> **Note**: Argus Camera is not supported on x86_64 platforms with discrete GPUs that do not have a CSI or GMSL interface to connect to a camera.
+
+> **Note**: See [Argus](https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html) for more information on camera processing.
 
-This repository provides the Isaac ROS Argus node, which enables ROS developers to use monocular and stereo cameras connected to Jetson platforms over CSI. The node uses the [Libargus API](https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html) internally for acquiring images and associated metadata from camera devices.
 
 ### Isaac ROS NITROS Acceleration
 
@@ -41,24 +59,24 @@ This package is powered by [NVIDIA Isaac Transport for ROS (NITROS)](https://dev
 
 ## Latest Update
 
-Update 2022-10-19: Updated OSS licensing
+Update 2023-03-23: Update to be compatible with JetPack 5.1.1
 
 ## Supported Platforms
 
-This package is designed and tested to be compatible with ROS2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) with off-the-shelf cameras from NVIDIA partners (see the [Reference Cameras](#reference-cameras) section for more details).
+This package is designed and tested to be compatible with ROS 2 Humble running on [Jetson](https://developer.nvidia.com/embedded-computing) with off-the-shelf cameras from NVIDIA partners (see the [Reference Cameras](#reference-cameras) section for more details).
 > **Note**: x86_64 system is not supported.
 
-> **Note**: Versions of ROS2 earlier than Humble are **not** supported. This package depends on specific ROS2 implementation features that were only introduced beginning with the Humble release.
+> **Note**: Versions of ROS 2 earlier than Humble are **not** supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
 
 | Platform | Hardware                                                                                                                                                                                                 | Software                                                       | Notes                                                                                                                                                                                                                                                                                                                                                       |
 | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Jetson   | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) <br> [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.0.2](https://developer.nvidia.com/embedded/jetpack) | The [CSI](https://en.wikipedia.org/wiki/Camera_Serial_Interface) camera device needs to be connected and presented as a video device (e.g. `/dev/video0`). <br><br> For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
+| Jetson   | [Jetson Orin](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) <br> [Jetson Xavier](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/) | [JetPack 5.1.1](https://developer.nvidia.com/embedded/jetpack) | The [CSI](https://en.wikipedia.org/wiki/Camera_Serial_Interface) camera device needs to be connected and presented as a video device (e.g. `/dev/video0`). <br><br> For best performance, ensure that [power settings](https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html) are configured appropriately. |
 
 ### Docker
 
 To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following [these steps](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md). This will streamline your development environment setup with the correct versions of dependencies on Jetson platforms.
 
-> **Note:** All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
+> **Note**: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
   
 ### Reference Cameras
 
@@ -194,7 +212,7 @@ Refer to [this page](https://docs.nvidia.com/jetson/l4t-multimedia/classArgus_1_
 
 > **Note**: Each camera module should have stored the calibration parameters in internal memory like EEPROM, and the device driver must support the Argus Ext API to extract those parameters. Contact your camera vendor to get the required drivers.
 
-> **Note:** If your camera does not support the Argus Ext API, you can also specify a URL to a camera info `.ini` file parseable by the ROS [CameraInfoManager](http://wiki.ros.org/camera_info_manager) using the `camera_info_url` parameter on the Isaac ROS Argus node. This will allow you to provide parameters you may have calibrated using the ROS Camera Calibration package, for example.
+> **Note**: If your camera does not support the Argus Ext API, you can also specify a URL to a camera info `.ini` file parseable by the ROS [CameraInfoManager](http://wiki.ros.org/camera_info_manager) using the `camera_info_url` parameter on the Isaac ROS Argus node. This will allow you to provide parameters you may have calibrated using the ROS Camera Calibration package, for example.
 
 > **Note**: When the `camera_info_url` is provided, the loaded parameters override the `CameraInfo` from Argus Ext API.
 
@@ -218,6 +236,7 @@ Exit the Docker container and restart the `nvargus` daemon by running `sudo serv
 
 | Date       | Changes                                    |
 | ---------- | ------------------------------------------ |
+| 2023-03-23 | Update to be compatible with JetPack 5.1.1 |
 | 2022-10-19 | Updated OSS licensing                      |
 | 2022-08-31 | Update to be compatible with JetPack 5.0.2 |
 | 2022-06-30 | Support NITROS acceleration                |

isaac_ros_argus_camera/CMakeLists.txt

@@ -1,5 +1,5 @@
 # SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -15,118 +15,41 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-cmake_minimum_required(VERSION 3.8)
-project(isaac_ros_argus_camera)
-
-# Default to C++17
-if(NOT CMAKE_CXX_STANDARD)
-  set(CMAKE_CXX_STANDARD 17)
-endif()
+cmake_minimum_required(VERSION 3.23.2)
+project(isaac_ros_argus_camera LANGUAGES C CXX)
 
 if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
   add_compile_options(-Wall -Wextra -Wpedantic)
 endif()
 
-# Default to Release build
-if(NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "")
-  set(CMAKE_BUILD_TYPE "Release" CACHE STRING "" FORCE)
-endif()
-message( STATUS "CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}" )
-
-execute_process(
-  COMMAND uname -m
-  COMMAND tr -d '\n'
-  OUTPUT_VARIABLE ARCHITECTURE)
-message(STATUS "Architecture: ${ARCHITECTURE}")
+execute_process(COMMAND uname -m COMMAND tr -d '\n' OUTPUT_VARIABLE ARCHITECTURE)
+message( STATUS "Architecture: ${ARCHITECTURE}" )
 
 find_package(ament_cmake_auto REQUIRED)
 ament_auto_find_build_dependencies()
+
 # Only build Argus nodes on Jetson platform
 if(${ARCHITECTURE} STREQUAL "aarch64")
-  # Find VPI dependency
-  find_package(vpi REQUIRED)
-  # Base class
-  ament_auto_add_library(base_node SHARED src/argus_camera_node.cpp)
-  # Mono camera node
-  ament_auto_add_library(mono_node SHARED src/argus_camera_mono_node.cpp)
-  # Stereo camera node
-  ament_auto_add_library(stereo_node SHARED src/argus_camera_stereo_node.cpp)
-
-  target_include_directories(
-    mono_node
-    PRIVATE /usr/lib/aarch64-linux-gnu/tegra
-            /usr/src/jetson_multimedia_api/argus/include
-            /usr/src/jetson_multimedia_api/argus/samples/utils
-            /usr/src/jetson_multimedia_api/include
-            include)
-
-  target_include_directories(
-    stereo_node
-    PRIVATE /usr/lib/aarch64-linux-gnu/tegra
-            /usr/src/jetson_multimedia_api/argus/include
-            /usr/src/jetson_multimedia_api/argus/samples/utils
-            /usr/src/jetson_multimedia_api/include
-            include)
-
-  target_compile_definitions(mono_node PRIVATE "COMPOSITION_BUILDING_DLL")
+# aarch64
 
-  target_compile_definitions(stereo_node PRIVATE "COMPOSITION_BUILDING_DLL")
+ament_auto_add_library(base_node SHARED src/argus_camera_node.cpp)
+ament_auto_add_library(mono_node SHARED src/argus_camera_mono_node.cpp)
+rclcpp_components_register_nodes(mono_node "nvidia::isaac_ros::argus::ArgusMonoNode")
+set(node_plugins "${node_plugins}nvidia::isaac_ros::argus::ArgusMonoNode;$<TARGET_FILE:mono_node>\n")
 
-  target_link_libraries(
-    mono_node
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so
-    vpi)
-
-  target_link_libraries(
-    stereo_node
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
-    /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so
-    vpi)
-
-  rclcpp_components_register_nodes(mono_node "nvidia::isaac_ros::argus::ArgusMonoNode")
-  set(node_plugins
-      "${node_plugins}nvidia::isaac_ros::argus::ArgusMonoNode;$<TARGET_FILE:mono_node>\n"
-  )
-
-  rclcpp_components_register_nodes(stereo_node "nvidia::isaac_ros::argus::ArgusStereoNode")
-  set(node_plugins
-      "${node_plugins}nvidia::isaac_ros::argus::ArgusStereoNode;$<TARGET_FILE:stereo_node>\n"
-  )
-
-  # Install config directory
-  install(
-    DIRECTORY config
-    DESTINATION share/${PROJECT_NAME})
-  install(
-    DIRECTORY launch
-    DESTINATION share/${PROJECT_NAME})
-  install(
-    TARGETS mono_node
-    ARCHIVE DESTINATION lib
-    LIBRARY DESTINATION lib
-    RUNTIME DESTINATION bin)
-
-  install(
-    TARGETS stereo_node
-    ARCHIVE DESTINATION lib
-    LIBRARY DESTINATION lib
-    RUNTIME DESTINATION bin)
-endif()
+ament_auto_add_library(stereo_node SHARED src/argus_camera_stereo_node.cpp)
+rclcpp_components_register_nodes(stereo_node "nvidia::isaac_ros::argus::ArgusStereoNode")
+set(node_plugins "${node_plugins}nvidia::isaac_ros::argus::ArgusStereoNode;$<TARGET_FILE:stereo_node>\n")
 
 if(BUILD_TESTING)
   find_package(ament_lint_auto REQUIRED)
   ament_lint_auto_find_test_dependencies()
 
   find_package(launch_testing_ament_cmake REQUIRED)
-  if(${ARCHITECTURE} STREQUAL "aarch64")
-    add_launch_test(test/isaac_ros_argus_camera_mono_test.py)
-    add_launch_test(test/isaac_ros_argus_camera_stereo_test.py)
-  endif()
+  add_launch_test(test/isaac_ros_argus_camera_mono_test.py)
+  add_launch_test(test/isaac_ros_argus_camera_stereo_test.py)
 endif()
 
-ament_auto_package(INSTALL_TO_SHARE launch)
+endif()  # aarch64
+
+ament_auto_package(INSTALL_TO_SHARE config launch)

isaac_ros_argus_camera/package.xml

@@ -20,7 +20,7 @@
 <?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
 <package format="3">
   <name>isaac_ros_argus_camera</name>
-  <version>0.20.0</version>
+  <version>0.30.0</version>
   <description>Support for Argus Stereo Camera</description>
 
   <maintainer email="[email protected]">Greg Lo</maintainer>
@@ -33,13 +33,16 @@
 
   <depend>camera_info_manager</depend>
   <depend>geometry_msgs</depend>
+  <depend>isaac_ros_image_proc</depend>
   <depend>isaac_ros_nitros</depend>
   <depend>isaac_ros_nitros_camera_info_type</depend>
   <depend>isaac_ros_nitros_image_type</depend>
   <depend>rclcpp</depend>
   <depend>rclcpp_components</depend>
   <depend>tf2_ros</depend>
 
+  <build_depend>isaac_ros_common</build_depend>
+
   <test_depend>ament_lint_auto</test_depend>
   <test_depend>ament_lint_common</test_depend>
   <test_depend>isaac_ros_test</test_depend>

isaac_ros_argus_camera/src/argus_camera_node.cpp

@@ -47,16 +47,17 @@ const std::unordered_map<std::string, DistortionType> g_ros_to_gxf_distortion_mo
         });
 
 const std::vector<std::pair<std::string, std::string>> EXTENSIONS = {
-  {"isaac_ros_nitros", "gxf/std/libgxf_std.so"},
-  {"isaac_ros_nitros", "gxf/cuda/libgxf_cuda.so"},
-  {"isaac_ros_nitros", "gxf/serialization/libgxf_serialization.so"},
-  {"isaac_ros_nitros", "gxf/libgxf_sight.so"},
-  {"isaac_ros_nitros", "gxf/libgxf_atlas.so"},
-  {"isaac_ros_nitros", "gxf/libgxf_isaac_messages.so"},
-  {"isaac_ros_nitros", "gxf/multimedia/libgxf_multimedia.so"},
-  {"isaac_ros_nitros", "gxf/tensorops/libgxf_tensorops.so"},
-  {"isaac_ros_nitros", "gxf/libgxf_argus.so"},
-  {"isaac_ros_nitros", "gxf/libgxf_message_compositor.so"}
+  {"isaac_ros_gxf", "gxf/lib/std/libgxf_std.so"},
+  {"isaac_ros_gxf", "gxf/lib/cuda/libgxf_cuda.so"},
+  {"isaac_ros_gxf", "gxf/lib/serialization/libgxf_serialization.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_gxf_helpers.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_sight.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_atlas.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_isaac_messages.so"},
+  {"isaac_ros_gxf", "gxf/lib/multimedia/libgxf_multimedia.so"},
+  {"isaac_ros_image_proc", "gxf/lib/image_proc/libgxf_tensorops.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_argus.so"},
+  {"isaac_ros_gxf", "gxf/lib/libgxf_message_compositor.so"}
 };
 constexpr char PACKAGE_NAME[] = "isaac_ros_argus_camera";
 const std::vector<std::string> GENERATOR_RULE_FILENAMES = {