MuJoCo Mode / Environment creation

[gautam1858]

Hi I just want to know how to create environment or models ? Is it recommended to use CAD software, design robotic arm and then convert to XML file ? I want to create a environment like having a table where the robotic arm can do certain things like pick and place etc. Is it possible to create xml file directly ? please advice on this.

[sanbens1234]

I have a similar question. If I want to model a terrain with specific height functions h(x) with respect to x-axis, is there any way to construct such terrain, see the picture below. It seems that hfield is not possible, because it is discrete and not smooth enough.

[gautam1858]

Hi I just want to know how to create environment or models ? Is it recommended to use CAD software, design robotic arm and then convert to XML file ? I want to create a environment like having a table where the robotic arm can do certain things like pick and place etc. Is it possible to create xml file directly ? please advice on this.

@kevinzakka @yuvaltassa Can you please help ?

[Balint-H]

@gautam1858 While you wait till Kevin or Yuval weighs in:

Currently the way you describe, CAD->URDF->XML is indeed often the most convenient and precise method. Here are some of the alternatives to consider:

• For something simple like a table, you can design it directly in the xml out of primitives

• You can find good example scenes online you can use a starting point, here are a couple recommendations from me:
  (prosthetic hand + table/other environments:)
  https://www.roboti.us/forum/index.php?resources/modular-prosthetic-limb-shap-test-suites.19/
  (other robotic arms, particularly take a look at ALOHA)
  https://github.com/google-deepmind/mujoco_menagerie

• The DeepMind control suite has good helper functionality to programmatically edit MJCF files. If you already know what kind of scene you want to create and can describe it in python, it can enable you to automate MJCF creation. https://github.com/google-deepmind/dm_control

• I have a very rudimentary Blender plugin I used to create a custom robotic arm + hand MJCF, available here. I do have a guide on using it, but its functionality is quite barebones and relies on learning Blender basics: https://github.com/Balint-H/blender-mjcf

• If you have proficiency with Unity already, the Unity plugin is also great at quickly and visually building Mujoco Scenes, then exporting them.

[sanjay-swain]

I have tried some CAD convertors in the past but honestly URDF is pretty bad in my opinion. For any kind of complex designs URDF is simply not enough. It doesn't have any support for closed kinematic links and also the CAD to URDF convertors available online are terrible.. I have tried Solidworks to URDF but I am pretty sure that one is unmaintained and couldn't find any kind of support.. Countless hours have been wasted with no results.

Also do you have any experiences or aware of any other tools to make this process less painful? I will look into your blender project. Thanks for that.

[Balint-H]

One of my collaborators went through the CAD->MJCF process (after which I joined them to help clean up the .xml), I asked them and will get back to you.

It's worth noting that the the method of modelling closed links is not self-evident in MJCF either, as only kinematic trees are supported hierarchy definitions. Usually you can get good implementations of closed loops in MuJoCo though using equality constraints to link the two ends of the chains!

I know there's already perhaps too many options, but yet another approach to consider would be @kevinzakka's OBJ->MJCF tool.

[Balint-H]

Sorry for the delay, I ended up confirming that my collaborator constructed the URDF manually from their CAD model, double checking values and entering them by hand. This we then converted to MJCF using the MuJoCo compile tools. This may be feasible for small models or one-off situations, but obviously doesn't scale.

I asked a couple other people in the MuJoCo team, and heard that some MuJoCo users use Onshape, which has decent public conversion tools to URDF as well.

[Balint-H]

@sanbens1234 Is there a reason why a higher resolution hfield would not work for this case (although I am aware the colliding object and the resolution of the hfield should be of comparable scale)? If you need higher order smoothness, then you might be able to create a custom SDF, but in practice I'd expect a higher-res hfield to work for this application.