[WIP] Add RLBench simulator environment

[RonPlusSign]

This pull request adds support for the RLBench benchmark to LeRobot, enabling users to train and evaluate policies on RLBench tasks. The integration includes environment configuration, dataset collection, documentation, and dependency management. The most important changes are grouped below.

RLBench Environment Integration

• Added the RLBenchEnv class to src/lerobot/envs/configs.py for configuring RLBench environments, including support for multiple observation types, camera settings, and feature mappings.
• Updated the make_env function in src/lerobot/envs/factory.py to support RLBench environments by importing and calling create_rlbench_envs when the environment type is RLBench.

RLBench Dataset Collection

• Added a new example script examples/dataset/rlbench_collect_dataset.py for collecting RLBench demonstrations and saving them in the LeRobot dataset format, with flexible action representations and camera configurations.

# Usage example:
python examples/dataset/rlbench_collect_dataset.py --repo_id="username/repo_name" --task="put_rubbish_in_bin" --num_episodes=10

Documentation Updates

• Added a comprehensive RLBench documentation page (docs/source/rlbench.mdx) describing the benchmark, its integration, setup instructions, and usage within LeRobot.
• Updated the documentation table of contents (docs/source/_toctree.yml) to include the new RLBench section.

Dependency Management

• Added RLBench as an optional dependency in pyproject.toml and included it in the all dependencies group to ensure easy installation.

Reviews needed

⚠️ This PR is not complete, as I'm currently struggling with multi-threading.
In particular, it breaks when instantiating multiple environments in the for loop of line 413 of rlbench.py, called from line 497 of lerobot_eval.py via make_env().
I think this is due to graphics libraries / QT rendering and how CoppeliaSim handles parallelism, but I currently didn't find any solution.

How it was tested

I tested the code trying to load an evaluation of a model on the given environment, as reported below.

How to checkout & try? (for the reviewer)

After installing the CoppeliaSim as explained in the docs/source/rlbench.mdx and installing the rlbench optional dependency (`pip install -e ".[rlbench]"), you can use rlbench as an env type:

lerobot-eval \
  --policy.path="RonPlusSign/smolvla_PutRubbishInBin" \
  --env.type=rlbench \
  --env.task=FS10_V1 \
  --eval.batch_size=1 \
  --eval.n_episodes=1

Before using RLBench, we need to ensure that the simulator is found, by exporting the following environment variables:

export COPPELIASIM_ROOT=${HOME}/CoppeliaSim
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT
export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT

I opened this PR because I'm working on VLA models on RLBench for my Master's thesis. This follows up an outdated PR (#290) and can be helpful for the issue I opened few days ago: #2259.

@jadechoghari, @fracapuano as reported in my issue #2259, here's the PR 😄
Any help from anybody to fix the concurrency issues would be very welcome!

[Copilot]

Pull Request Overview

This PR adds RLBench support to LeRobot, enabling training and evaluation of policies on RLBench's 100+ vision-guided manipulation tasks. RLBench is a large-scale benchmark built on CoppeliaSim that provides infinite demonstrations via motion planning.

• Implements RLBenchEnv wrapper with support for pixel-based observations and joint position actions
• Adds configuration infrastructure for RLBench task suites (FS10_V1 through MT100_V1) with task descriptions
• Includes dataset collection script for converting RLBench demonstrations to LeRobot format
• Provides comprehensive documentation with installation and usage examples

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
src/lerobot/envs/rlbench.py	Main environment wrapper implementing Gymnasium interface for RLBench tasks
src/lerobot/envs/rlbench_config.json	Configuration file mapping 106 RLBench tasks to IDs and descriptions
src/lerobot/envs/factory.py	Adds RLBench environment creation support to the factory function
src/lerobot/envs/configs.py	Defines RLBenchEnv configuration dataclass with features and observation mappings
pyproject.toml	Adds RLBench as optional dependency
examples/dataset/rlbench_collect_dataset.py	Script for collecting demonstrations and converting to LeRobot dataset format
docs/source/rlbench.mdx	Comprehensive documentation for RLBench integration
docs/source/_toctree.yml	Adds RLBench documentation to table of contents

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Incomplete sentence with extra 'the to' - should likely be 'Empty the container into the container' or 'Empty the container in the container'.

Suggested change

	"empty_container": "Empty the container in the to container",
	"empty_container": "Empty the container into the container",

[Copilot]

The word 'queue' should be 'cue' (a billiards/pool cue). Also, 'in to' should be 'into'.

Suggested change

	"hit_ball_with_queue": "Hit ball with queue in to the goal",
	"hit_ball_with_queue": "Hit ball with cue into the goal",

[Copilot]

Missing object noun after 'Take the' - should be 'Take the meat off the grill'.

Suggested change

	"meat_off_grill": "Take the off the grill",
	"meat_off_grill": "Take the meat off the grill",

[Copilot]

Missing object noun after 'Put the' - should be 'Put the meat on the grill'.

Suggested change

	"meat_on_grill": "Put the on the grill",
	"meat_on_grill": "Put the meat on the grill",

[Copilot]

Missing object noun after 'Pick up the' - should be 'Pick up the block and lift it up to the target' or similar.

Suggested change

	"pick_and_lift_small": "Pick up the and lift it up to the target",
	"pick_and_lift_small": "Pick up the small block and lift it up to the target",

[Copilot]

Incomplete sentence ending with 'place it' - missing destination.

Suggested change

	"take_money_out_safe": "Take the money out of the bottom shelf and place it on",
	"take_off_weighing_scales": "Remove the pepper from the weighing scales and place it",
	"take_money_out_safe": "Take the money out of the bottom shelf and place it on the table",
	"take_off_weighing_scales": "Remove the pepper from the weighing scales and place it on the table",

[Copilot]

Incorrect usage of np.concatenate. The function requires a sequence of arrays as a single argument, not separate arguments. Should be: np.concatenate([raw_obs['joint_positions'], [raw_obs['gripper_open']]])

Suggested change

	agent_pos = np.concatenate(
	raw_obs["joint_positions"],
	[raw_obs["gripper_open"]],
	)
	agent_pos = np.concatenate([raw_obs["joint_positions"], [raw_obs["gripper_open"]]])

[Copilot]

Variable image is undefined in this scope. The code stores images in the images dictionary but then references an undefined image variable. Should use one of the images from the images dictionary or from raw_obs.

Suggested change

	obs = {"pixels": image.copy()}
	elif self.obs_type == "pixels_agent_pos":
	obs = {
	"pixels": image.copy(),
	obs = {"pixels": images.copy()}
	elif self.obs_type == "pixels_agent_pos":
	obs = {
	"pixels": images.copy(),

[Copilot]

Grammatical inconsistency: 'wrist camera' should be plural 'wrist cameras' to match 'over-the-shoulder cameras'.

Suggested change

	- **Rich, Multi-Modal Observations:** The benchmark provides both proprioceptive (joint states) and visual observations. Visual data comes from multiple camera angles, including over-the-shoulder cameras and wrist camera, with options for RGB, depth, and segmentation masks.
	- **Rich, Multi-Modal Observations:** The benchmark provides both proprioceptive (joint states) and visual observations. Visual data comes from multiple camera angles, including over-the-shoulder cameras and wrist cameras, with options for RGB, depth, and segmentation masks.

[Copilot]

The global variable 'TASK_NAME_TO_ID' is not used.

Suggested change

TASK_NAME_TO_ID: dict[str, int] = data.get("TASK_NAME_TO_ID", {})

[fracapuano]

ty for the PR! Can you provide a MRE so that we can take a look at what exactly is going on with multi-processing? it'd be useful to debug

[RonPlusSign]

Sure!

1. Install CoppeliaSim

# set env variables
export COPPELIASIM_ROOT=${HOME}/CoppeliaSim
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT
export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT

wget https://downloads.coppeliarobotics.com/V4_1_0/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz
mkdir -p $COPPELIASIM_ROOT && tar -xf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz -C $COPPELIASIM_ROOT --strip-components 1
rm -rf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz

2. Clone the fork

git clone https://github.com/RonPlusSign/lerobot.git
cd lerobot
git checkout add-rlbench-simulator

3. Create a venv and install LeRobot + RLBench

conda create -n lerobot_rlbench python=3.10 -y
conda activate lerobot_rlbench
pip install pycparser    # needed while cloning rlbench
pip install -e ".[rlbench,smolvla]"    # smolvla is used in next steps for testing, but it's not directly linked with rlbench

4. Run the test

(here is when the problem occours)

lerobot-eval \
  --policy.path="RonPlusSign/smolvla_PutRubbishInBin" \
  --env.type=rlbench \
  --env.task=FS10_V1 \
  --eval.batch_size=1 \
  --eval.n_episodes=1

In particular, it breaks when instantiating multiple environments in the for loop of line 413 of rlbench.py, called from line 497 of lerobot_eval.py via make_env().

Useful links: RLBench#140, RLBench#161, PyRep parallelization.

[jadechoghari]

Hey @RonPlusSign great work! Do you have cases where RLBench work in this PR ?
In lerobot-eval