Skip to content

v2.3.0

Latest
Latest

Choose a tag to compare

View all tags
@kellyguo11 kellyguo11 released this 28 Oct 21:38
v2.3.0
3c6e67b

What's Changed

The Isaac Lab 2.3.0 release, built on Isaac Sim 5.1, delivers enhancements across dexterous manipulation, teleoperation, and learning workflows. It introduces new dexterous environments with advanced training capabilities, expands surface gripper and teleoperation support for a wider range of robots and devices, and integrates SkillGen with the Mimic imitation learning pipeline to enable GPU-accelerated motion planning and skill-based data generation with cuRobo integration.

Key highlights of this release include:

  • Dexterous RL (DexSuite): Introduction of two new dexterous manipulation environments using the Kuka arm and Allegro hand setup, with addition of support for Automatic Domain Randomization (ADR) and PBT (Population-Based Training).
  • Surface gripper updates: Surface gripper has been extended to support Manager-based workflows, including the addition of SurfaceGripperAction and SurfaceGripperActionCfg, along with several new environments demonstrating teleoperation examples with surface grippers and the RMPFlow controller. New robots and variations are introduced, including Franka and UR10 with robotiq grippers and suction cups, and Galbot and Agibot robots.
  • Mimic - SkillGen: SkillGen support has been added for the Mimic Imitation Learning pipeline, introducing cuRobo integration, integrating GPU motion planning with skill-segmented data generation. Note that cuRobo has proprietary licensing terms, please review the cuRobo license carefully before use.
  • Mimic - Locomanipulation: Added a new G1 humanoid environment combining RL-based locomotion with IK-based manipulation. A full robot navigation stack is integrated to augment demonstrations with randomization of tabletop pick/place locations, destination and ground obstacles. By segmenting tasks into pick-navigate-place phases, this method enables generation of large-scale loco-manipulation datasets from manipulation-only demonstrations.
  • Teleoperation: Upper body inverse kinematics controller is improved by adding a null space posture task that helps enable waist movement on humanoid tasks while regularizing redundant degrees-of-freedom to a preferred upright posture. Additionally, support for Vive and Manus Glove are introduced, providing more options for teleoperation devices.

Full Changelog: v2.2.1...v2.3.0

🔆 Highlighted Features

Dex environments

We have added a new DexSuite, including dexterous Lift and Reorient environments following DextrAH and DexPBT. These environments also demonstrate usage of Automatic Domain Randomization (ADR) and PBT (Population-Based Training)

cube.mp4
rod.mp4

Isaac Lab Mimic - Locomanipulation

We have added a new locomanipulation G1 environment with upper and lower body controllers that enables demonstrations by controlling the arms and fingers, and locomotion by controlling the pelvis instantaneous velocities and height (vx, vy, wz, h). Additionally, the pink-based inverse kinematics controller now supports null-space posture regularization to enable waist DOFs and extend the humanoid robots reach.

Release.Note.-.G1.Locomanipulation.Pick.Place.mp4

We have also extended these in-place locomanipulation tasks with integrated navigation and demonstration augmentation, enabling large-scale pick-navigate-place dataset generation from manipulation-only demonstrations.

locomanipulation-navigation.mp4

💡Isaac Sim 5.1 Updates

  • Introduced support for DGX Spark, including multi-architecture Docker images with support for ARM platforms.
  • PhysX now offers a new joint parameter tuning tutorial for robotic grippers, along with a new feature for solving articulation collision contacts last to improve on gripper penetration issues, especially for cases with sub-optimally tuned joints.
  • Surface grippers has been optimized for better performance. Although support continues to be CPU-only, performance has improved by several orders of magnitude compared to previous releases.
  • Windows 10 support ended on October 14, 2025. Microsoft will no longer provide free security, feature, or technical updates for Windows 10. As a result, we will be dropping support for Windows 10 in future releases of Isaac Sim and Lab to ensure the security and functionality of our software.

✨ New Features

Core

Mimic and Teleoperation

Environments

  • Adds dexterous lift and reorientation manipulation environments by @ooctipus in #3378
  • Adds task Reach-UR10e, an end-effector tracking environment by @ashwinvkNV in #3147
  • Adds a configuration example for Student-Teacher Distillation by @ClemensSchwarke in #3100
  • Adds Locomanipulation Environment with G1 for Mimic workflow by @michaellin6 in #3150
  • Adds teleop support for Unitree G1 with Inspire 5-finger hand, take PickPlace task as an example by @yami007007 in #3242
  • Adds galbot stack cube tasks, with left_arm_gripper and right_arm_suction, using RMPFlow controller by @rebeccazhang0707 in #3210
  • Adds AVP teleop support for Galbot stack tasks by @rwiltz in #3669
  • Adds camera to G1 Steering Wheel environment by @jaybdub in #3549

Infrastructure

  • Adds YAML Resource Specification To Ray Integration by @binw666 in #2847
  • Installs cuda13 on arm builds for Spark by @kellyguo11 in #3396
  • Adds arm64 platform for Pink IK setup by @michaellin6 in #3686
  • Updates torch installation version to 2.9 for Linux-aarch, and updates opset version from 11 to 18. by @ooctipus in #3706

🔧 Improvements

Core and Infrastructure

Mimic and Teleoperation

  • Improves recorder performance and add additional recording capability by @peterd-NV in #3302
  • Optimizes Kit XR Teleop CPU time by @hougantc-nvda in #3487
  • Improves dataset file names and low success rate for trained model on g1 locomanipulation dataset by @michaellin6 in #3503
  • Updates the teleop_se3 and record_demos scripts with more helpful description for teleop_device parameter by @rwiltz in #3642

🐛 Bug Fixes

Core

  • Fixes missing visible attribute in spawn_ground_plane by @kellyguo11 in #3304
  • Moves parameter platform_height to the correct mesh terrain configuration by @Mayankm96 in #3316
  • Fixes invalid callbacks for debug vis when simulation is restarted by @kellyguo11 in #3338
  • Deletes unused asset.py in isaaclab by @fan-ziqi in #3389
  • Moves location of serve file check to the correct module by @Mayankm96 in #3368
  • Fixes SurfaceGripper API to accommodate for Isaac Sim 5.1 changes by @AntoineRichard in #3528
  • Fixes keyboard unsubscribe carb call by @rwiltz in #3662
  • Fixes GCC error for raycaster demo when running in conda by @kellyguo11 in #3712
  • Corrects materials and objects imports in check_terrain_importer.py by @PeterL-NV in #3411
  • Fixes tensor construction warning in events.py by @louislelay in #3251
  • Fixes skrl train/play script configurations when using the --agent argument and rename agent configuration variable by @Toni-SM in #3643
  • Fixes TiledCamera data types and rlgames training on CPU by @kellyguo11 in #3808

Mimic and Teleoperation

  • Updates the Path to Isaaclab Dir in SkillGen Documentation by @njawale42 in #3483
  • Fixes the reach task regression with teleop devices returning the gripper by @rwiltz in #3327
  • Fixes teleop G1 with Inspire hand issues by @yami007007 in #3440
  • Updates default viewer pose to see the whole scene for Agibot environment by @rebeccazhang0707 in #3525
  • Fixes XR UI when used with teleop devices other than "handtracking" by @rwiltz in #3566
  • Fixes manus joint indices mapping for teleoperation by @rwiltz in #3592
  • Updates gr1t2 dex pilot hand scaling by @rwiltz in #3607
  • Fixes unreal surface_gripper behavior by @rebeccazhang0707 in #3679
  • Fixes G1 finger PD gains configs for locomanipulation by @michaellin6 in #3749
  • Fixes the bug of right_arm suction cup passing through cubes by @rebeccazhang0707 in #3764
  • Updates the xr anchor for g1 tasks to me more natural for standing teleop by @rwiltz in #3775
  • Suppresses dex_retargeting::yourdfpy warnings for G1 by @rwiltz in #3798
  • Refines height of xr view for G1 envs by @rwiltz in #3813

Infrastructure

📜 Documentation

Core

  • Updates documentation to explain known issue of missing references when uses URDF importer by @ooctipus in #3729
  • Fixes symbol in training_jetbot_reward_exploration.rst by @dougfulop in #2722
  • Clarifies asset classes' default_inertia tensor coordinate frame by @preist-nvidia in #3405
  • Adds limitation note in docs for Multi Node Training on DGX Spark by @matthewtrepte in #3806
  • Updates locomanip task name and link in docs by @fan-ziqi in #3342

Mimic and Teleoperation

  • Fixes G1 dataset link in teleop_imitation tutorial by @michaellin6 in #3463
  • Updates dataset instruction in teleop_imitation.rst (#3462) by @peterd-NV in #3489
  • Fixes teleop doc in Isaac Lab by @tifchen-nvda in #3539
  • Updates cloudxr teleop doc in Isaac Lab by @tifchen-nvda in #3540
  • Adds instructions on how to position the lighthouse for manus+vive by @cathyliyuanchen in #3548
  • Corrects versions for the cloudxr teleop doc by @tifchen-nvda in #3580
  • Adds link to IsaacLabEvalTasks repo from mimic section in doc (#3621) by @xyao-nv in #3627
  • Fixes ordering of docs for imitation learning by @shauryadNv in #3634
  • Updates documentation for manus teleop by @rwiltz in #3605
  • Updates SkillGen documentation for data gen command and success rates by @njawale42 in #3703
  • Fixes typo in mimic teleop documentation for locomanipulation by @michaellin6 in #3704
  • Updates dataset paths in teleop documentation and adds note in documentation to adjusting AR Anchors by @michaellin6 in #3707
  • Adds pysurvive installation instructions by @rwiltz in #3747
  • Adds to mimic documentation expected generation and training timings and success rates by @michaellin6 in #3742
  • Adds data gen and policy learning times in SkillGen documentation by @njawale42 in #3774
  • Updates doc to describe ways to clean up orphaned container and check connectivity for teleop by @yanziz-nvidia in #3787
  • Updates cloudxr teleop doc to explain openxr plugin by @tifchen-nvda in #3786
  • Updates Mimic docs to clarify CPU mode usage and DGX Spark support by @peterd-NV in #3794
  • Updates cuRobo installation instructions and added VRAM baseline perf to SkillGen docs by @njawale42 in #3797
  • Adds dgx spark limitations link to teleop docs by @lotusl-code in #3805
  • Adds Cosmos Transfer1 limitation for DGX spark by @shauryadNv in #3817
  • Updates DGX spark limitations for SkillGen in the documentation by @njawale42 in #3748
  • Adds the Isaac-PickPlace-G1-InspireFTP-Abs-v0 Task into Envs Docs by @yami007007 in #3479

Infrastructure

💔 Breaking Changes

  • Removes unused 'relevant_link_name' parameter in nutpour and exhaust pipe envs by @michaellin6 in #3651
  • Moves IO descriptor log dir to logs by @kellyguo11 in #3434

Known Issues

  • The ROS2 docker image is not currently expected to work due to the update to Python 3.11. We are actively working on
    a fix to resolve this.
  • We have received reports of performance regressions in the previous Isaac Sim release for both physics and rendering workflows. We are still working on addressing some of these, but have also found some workarounds. For viewport regressions, Omniverse settings can be set by adding --kit_args="--/app/usdrt/hierarchy/partialGpuUpdate=1 --/rtx/post/dlss/execMode=0 --/app/runLoops/main/rateLimitEnabled=false --/app/runLoops/main/manualModeEnabled=true --enable omni.kit.loop-isaac". Additionally, Isaac Sim 5.0 introduced new actuator models for PhysX, including drive model and friction model improvements. These improvements also introduced a small performance regression. We have observed up to ~20% slowdown in some state-based environments.

🤗 New Contributors

Contributors

dougfulop, jaybdub, and 33 other contributors
Assets 2
Loading