Interest in a dm_control like API for continuous control tasks?

[ethanluoyc]

Is there any interest in a dm_control like API for pybullet's robotic tasks?

DeepMind recently released their suite dm_control which they use for continuous control. It features
a more customizable set of API which might be more useful for researchers working on RL for continuous control.

https://github.com/deepmind/dm_control

This this would allow for much more flexibility in engineering for state observations, how tasks are specified. They distinguish between domain and tasks. For example, given the same pendulum dynamics, we may have different sets of tasks to solve (swing up or something else).
Currently, the way OpenAI Gym handles these things is not so flexible. I will probably write something recently, I am posting this here just to see if anyone else is actually interesting in this.

[erwincoumans]

Yes, I'm surely interested in this. I'd like to re-create (some of) their environments in PyBullet, ideally in a way that models trained in dm_control are compatible in pybullet_control and vise-versa.
Please fork Bullet and prototype a simple example env for some feedback (say in Bullet/examples/pybullet/pybullet_control.

For most of our PyBullet Gym enviroments we already separate the robot from the environment. For example see gym_locomotion_envs.py versus robot_locomotors.py and also minitaur_duck_gym_env.py versus minitaur.py.

[benelot]

I am currently working on releasing the dm_control envs in my repository: https://github.com/benelot/pybullet-gym. If you are interested in collaborating, just leave me a note!

…

On Wed, Jan 17, 2018 at 5:10 PM erwincoumans ***@***.***> wrote: Yes, I'm surely interested in this. I'd like to re-create (some of) their environments in PyBullet, ideally in a way that models trained in dm_control are compatible in pybullet_control and vise-versa. Please fork Bullet and prototype a simple example env for some feedback (say in Bullet/examples/pybullet/pybullet_control. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1520 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC97q6Ef-T4dych-pd_LMBvk5rdA6lohks5tLhuPgaJpZM4RhjUF> .

[erwincoumans]

Since dm_control doesn't use the gym interface, it would be nice to pick naming carefully. If the environments are relatively simple it can become part of PyBullet as pybullet_controls.

[benelot]

It seems that there are already wrappers for dm_control to be used with gym. But that is definitely true, maybe we should rethink the naming there.

…

On Wed, Jan 17, 2018 at 5:19 PM erwincoumans ***@***.***> wrote: Since dm_control doesn't use the gym interface, it would be nice to pick naming carefully. If the environments are relatively simple it can become part of PyBullet as pybullet_controls. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1520 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC97q7Huc-VR8JsB0P4OhyYifvVSI_bRks5tLh2UgaJpZM4RhjUF> .

[ethanluoyc]

@erwincoumans I find the naming robot a bit confusing. After some digging, I think this sort of correspond to Physics in dm_control (https://github.com/deepmind/dm_control/blob/master/dm_control/mujoco/engine.py#L92).
@benelot I will make a prototype and ping you after that :D

[erwincoumans]

Wow, I find the name 'Physics' much more confusing that 'robot'. It seems dm_control 'Physics' is the python binding to MuJoCo, which implies it is similar to 'pybullet'?

[ethanluoyc]

I think we can keep it as Robot. The thing I want to improve is to add ease when users want to customize their rewards and state representation.

[ethanluoyc]

@erwincoumans I encountered some issues when importing the MJCF files from dm_control.
Some of the tags not yet supported:

• include
• light
• camera
• geom with type plane (it has really different rendering)
• zaxis does not seem to be handled correctly
  I guess it would be a good idea to keep track of a list of these things not yet supported?

[ethanluoyc]

Example for dm_control's cartpole, the rails are not handled correctly. This might be due to some issues with handling zaxis.

[erwincoumans]

Indeed, not all fields are supported, since MJCF is evolving and a very complex format compared to URDF. The MJCF parser is only tested with the files that ship with Bullet/Pybullet, those are from OpenAI Gym and Roboschool. It shouldn't be hard to adapt the DeepMind dm_control files. You could have a look at Bullet/data/mjcf and compare the xml files. 'include' tag can be done manually by replacing the include line with the content of the included file. I didn't port light/camera, since it was not needed for our control tasks yet. Once I find some time, I'll check out dm_control and make the MJCF files load in PyBullet. Your help is appreciated.

I think the best approach is to fork https://github.com/deepmind/dm_control and add PyBullet support next to MuJoCo in the forked repository. The DeepMind dm_control seems more interesting than Gym.