[Question] Using wheel-based mobile robots in simulation

[junminkyung]

Dear Isaac Lab Team,

I hope this message finds you well.

I am currently working on a project involving reinforcement learning for mobile robots, and I am interested in using Isaac Lab for this purpose. I have a few questions regarding the support for mobile robots:

1. Are there any plans to update Isaac Lab to better support reinforcement learning for mobile robots? If so, could you provide an estimated timeline for these updates?

2. In the meantime, are there any existing examples or resources within Isaac Lab that you would recommend for getting started with reinforcement learning on mobile robots?

Thank you very much for your time and assistance. I look forward to your response.

[KyleM73]

I'm not sure if this has changed in a more recent release of PhysX but previously there was not yet a cylinder prim object for the GPU pipeline, which defaulted to an n-gon (I believe it was 18 but could be configured up to 64). As a result directly simulating wheels for RL was not physically realistic. Instead, the supported workflow was to simulate the robot as the end effector of a virtual arm parallel to the ground, and then use the action space for that virtual arm (pose deviation, velocity, acceleration, whatever you prefer) to control the physical robot with some low level pose/velocity/etc. controller. The idea was that for most indoor environment "locomotion" is not the difficult task that must be learned with RL, but rather (potentially perceptive) navigation, as well as navigation with medium-to-long horizon manipulation. I'm not sure if an official example of this workflow exists, and of course if the supported workflow has changed please someone correct me.

[Mayankm96]

The PhysX team is working on proper wheel simulation. Once that's ready, we'll add some concrete examples on wheel-based mobile robots. Stay tuned :)

[umangdighe]

The PhysX team is working on proper wheel simulation. Once that's ready, we'll add some concrete examples on wheel-based mobile robots. Stay tuned :)

Hi Mayank!

Any update on a wheeled robot example? Thanks :)

[junminkyung]

Got it! Looking forward to it :)

By the way, I’ve tried creating and using a differential drive-type action myself, and I’m really excited to see how PhysX will improve wheel simulation for mobile robots.

Also, are there any plans for developing RTX LiDAR? I imagine there would be a lot of interest in that as well!

[ronaldluc]

All the official Isaac Sim vehicle examples use the n-gon directly driven by a joint (e.g. Nova Carter). Thus we've implemented this approach, as it's native to Isaac Lab workflow. But we are now hitting a wall with regards to sim2real when trying to do differential drive, mecanum wheels and a tank drive. The main issue is the inability to turn in-place at low effort and the joint speed not matching with the output robot speed. Given the wheel is n-gon, the mapping of linear/angular speed to actually experienced linear/angular speed does not seem like a polynomial function.

Is PhysX Vehicle API a good temporary patch?

On the other hand, the Isaac Sim PhysX examples have vehicles using PhysX Vehicle API : Ackerman, Differential drive, Tank. Should we use these for now, or is that not part of the GPU pipeline?

Thanks

[ronaldluc]

Please, do I understand correctly, that parts of the overall Isaac Sim are not on GPU pipeline (like these omni.isaac.wheeled_robots extension controllers), thus they cannot be used in Isaac Lab?

[sinaqahremani]

I am curious if the wheel simulation you mentioned is the vehicle plugin or something new?

[junminkyung]

I mean mobile robot like clearpath husky A200

[AlistairKeiller]

PhysX added Cylinder support!!!! NVIDIAGameWorks/PhysX#50

Any chance that mobile robot examples are around the corner?

[geiko246]

Hi! This is a bit of an old thread but I wanted to plug our work on wheeled robots here since I happened upon this thread during our development. We've found that it fills a large gap in Isaac Lab (and the robotics community as a whole):

Please do leave feedback or suggestions if you try it out!

Website: https://uwrobotlearning.github.io/WheeledLab/
Repository: https://github.com/UWRobotLearning/WheeledLab
Paper: https://arxiv.org/pdf/2502.07380