Add gymnasium-robotics simulation

[talregev]

What this does

Explain what this PR does. Feel free to tag your PR with the appropriate label(s).

Examples:

Title	Label
Adds new simulation	(🦾 gymnasium-robotics)

How it was tested

pip install "lerobot[gymnasium-robotics,smolvla] @ git+https://github.com/talregev/lerobot.git@TalR/gym_robotics" --extra-index-url https://download.pytorch.org/whl/cpu

        lerobot-eval \
            --policy.path=lerobot/smolvla_base \
            --policy.device=cpu \
            --policy.n_action_steps=1000 \
            --policy.chunk_size=1000 \
            --env.episode_length=1000 \
            --env.max_state_dim=32 \
            --env.type=gymnasium-robotics \
            --env.task=Ant-v5 \
            --eval.n_episodes=1 \
            --eval.batch_size=1 \
            --rename_map='{"observation.image": "observation.images.camera1"}' \
            --policy.output_features='{"action":{"type":"ACTION","shape":[8]}}' \
            --output vla-results/Ant-v5

        lerobot-eval \
            --policy.path=lerobot/smolvla_base \
            --policy.device=cpu \
            --policy.n_action_steps=1000 \
            --policy.chunk_size=1000 \
            --env.episode_length=1000 \
            --env.type=gymnasium-robotics \
            --env.task=FetchPickAndPlace-v4 \
            --eval.n_episodes=1 \
            --eval.batch_size=1 \
            --rename_map='{"observation.image": "observation.images.camera1"}' \
            --policy.output_features='{"action":{"type":"ACTION","shape":[4]}}' \
            --output vla-results/FetchPickAndPlace-v4

Explain/show how you tested your changes.

Examples:

• Added examples\smoke_gym_robotics.py.

How to checkout & try? (for the reviewer)

Provide a simple way for the reviewer to try out your changes.

Examples:

python examples\smoke_gym_robotics.py

SECTION TO REMOVE BEFORE SUBMITTING YOUR PR

Note: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.

Note: Before submitting this PR, please read the contributor guideline.

[talregev]

Please let me know how much work I need to be done to complete this PR.
I waiting for review.

[jadechoghari]

@talregev Thank you for this great PR! 🙏
Could you share a few examples of why gymnasium-robotics might be useful to have it inside LeRobot? 🤗

[talregev]

gymnasium-robotics is environment that combine gymnasium with mujoco simulator that you can add robots and new tasks, then create or train on a new dataset for your need. There is many complex tasks that you can choose, and also add yourself.
official website: https://robotics.farama.org/
Beside fetch tasks:
https://robotics.farama.org/envs/fetch/
There is more complex tasks:
https://robotics.farama.org/envs/shadow_dexterous_hand/
https://robotics.farama.org/envs/adroit_hand/

Also because it using gymnasium, all the gymnasium tasks is out of the box too:
https://gymnasium.farama.org/
https://gymnasium.farama.org/environments/mujoco/

[jadechoghari]

Thanks, do we have ready to use dataset such that we can train the evaluate one of our policy on a task from gymnasium-robotics?

[talregev]

Thanks, do we have ready to use dataset such that we can train the evaluate one of our policy on a task from gymnasium-robotics?

I didn't try to train because I don't have a gpu. I tested with evaluation. So for my side it ready for evaluate with smalvla. and maybe ready for train. Do you care to try it out?

You can try evaluate / train online on FetchPickAndPlace-v4 as the example I show for evaluation.
Please start with evaluation first, because I know it working.

[talregev]

I updated the code that it will support episode_length and max_state_dim for the environment.
I am adding 2 videos that you can impress. These video was created from lerobot eval:
clearly the vla doesn't know what to do, but it can be teached. I will try to train it.

FetchPickAndPlace-v4:

eval_episode_0.mp4

Ant-v5:

eval_episode_0.mp4

[talregev]

How I can train only online? without need of dataset. Can I do it without hugging face repo id? only local?
I try like that without success:

          cat > "train_cfg.json" <<'JSON'
          {
              "job_name": "hilserl_fetch_pick_v4_cpu",
              "seed": 0,
              "env": {
                  "type": "gymnasium-robotics",
                  "task": "FetchPickAndPlace-v4",
                  "episode_length": 200,
                  "features_map": {
                      "action": "action",
                      "agent_pos": "observation.state",
                      "top": "observation.image",
                      "pixels/top": "observation.image"
                  },
                  "features": {
                      "action": {
                          "type": "ACTION",
                          "shape": [
                              4
                          ]
                      },
                      "agent_pos": {
                          "type": "STATE",
                          "shape": [
                              4
                          ]
                      },
                      "pixels/top": {
                          "type": "VISUAL",
                          "shape": [
                              480,
                              480,
                              3
                          ]
                      }
                  }
              },
              "policy": {
                  "type": "sac",
                  "device": "cpu",
                  "concurrency": {
                      "actor": "threads",
                      "learner": "threads"
                  },
                  "repo_id": "None",
                  "push_to_hub": false
              },
              "dataset": { 
                  "repo_id": "online-buffer",
                  "root": "${{ github.workspace }}/dataset",
                  "use_imagenet_stats": true
              }
          }
          JSON

          mkdir -p dataset/online-buffer

          export HF_HUB_OFFLINE=1
          export HF_HUB_DISABLE_TELEMETRY=1
          export HF_DATASETS_OFFLINE=1
          export WANDB_MODE=disabled

          # Launch learner and actor (one shell)
          python -m lerobot.rl.learner --config_path "train_cfg.json"
          python -m lerobot.rl.actor   --config_path "train_cfg.json"

[jadechoghari]

Hello @talregev , thank you for the PR!
Currently we are only looking to add support for VLA, imitation learning benchmarks., which means you have a dataset then train and evaluate on a VLA policy. We still need some time to natively support more RL stuff here.
If you think things could be made otherwise let us know! + I see this could be added on the Huggingface hub, then loaded from their, once we merge: #2121