Merge pull request #3245 from ZhuYaoHui1998/docusaurus-version

fix orbbec camera of lerobot

Author: nfs0619

docs/Robotics/Robot_Kits/Lerobot/Lerobot_SO100Arm_New.md

@@ -589,6 +589,122 @@ The teleoperate command will automatically:
 
 ## Add cameras
 
+
+<details>
+<summary> If using the Orbbec Gemini2 Depth Camera </summary>
+
+<div align="center">
+    <img width={800}
+    src="https://media-cdn.seeedstudio.com/media/catalog/product/cache/bb49d3ec4ee05b6f018e93f896b8a25d/0/-/0-101090144--orbbec-gemini-2-3d-camera.jpg" />
+</div>
+<div class="get_one_now_container" style={{textAlign: 'center'}}>
+<a class="get_one_now_item" href="https://www.seeedstudio.com/Orbbec-Gemini-2-3D-Camera-p-6464.html" target="_blank" rel="noopener noreferrer" >
+            <strong><span><font color={'FFFFFF'} size={"4"}> Get One Now 🖱️</font></span></strong>
+</a></div>
+
+
+- 🚀 Step 1: Install the Orbbec SDK Dependent Environment
+
+1. Clone the `pyorbbec` repository
+   ```bash
+   cd ~/
+   git clone https://github.com/orbbec/pyorbbecsdk.git
+   ```
+
+2. Download and install the corresponding **.whl file** for the SDK  
+   Go to [pyorbbecsdk Releases](https://github.com/orbbec/pyorbbecsdk/releases),  
+   select and install based on your Python version. For example:
+   ```bash
+   pip install pyorbbecsdk-x.x.x-cp310-cp310-linux_x86_64.whl
+   ```
+
+3. Install dependencies in the `pyorbbec` directory
+   ```bash
+   cd ~/pyorbbecsdk
+   pip install -r requirements.txt
+   ```
+
+   Force downgrade the `numpy` version to `1.26.0`
+    ```bash
+    pip install numpy==1.26.0
+    ```
+  Red error messages can be ignored.
+
+4. Clone the Orbbec SDK into the `~/lerobot/src/cameras` directory
+
+  ```bash
+  cd ~/lerobot/src/cameras
+  git clone https://github.com/ZhuYaoHui1998/orbbec.git
+  ```
+
+5. Modify utils.py and __init__.py
+- Find `utils.py` in the `~/lerobot/src/lerobot/cameras` directory, and add the following code at line 40:
+
+```python
+elif cfg.type == "orbbec":
+            from .realsense.camera_orbbec import OrbbecCamera
+
+            cameras[key] = OrbbecCamera(cfg)
+```
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/utils.png" />
+</div>
+
+- Find `__init__.py` in the `~/lerobot/src/lerobot/cameras` directory, and add the following code at line 18:
+
+```python
+from .orbbec.configuration_orbbec import OrbbecCameraConfig
+```
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/init.png" />
+</div>
+
+
+- 🚀 Step 2: Function Call and Examples
+
+In all the following examples, replace `so101_follower` with the actual model of the robotic arm you are using (e.g., `so100` / `so101`).
+
+
+We have added the `focus_area` hyperparameter. Since depth data that is too far away is meaningless for the robotic arm (it cannot reach or grasp objects), depth data less than or greater than the `focus_area` will be displayed in black. The default `focus_area` is (20, 600).  
+Currently, the only supported resolution is width: 640, height: 880.
+
+```bash
+lerobot-teleoperate \
+    --robot.type=so101_follower \
+    --robot.port=/dev/ttyACM0 \
+    --robot.id=my_awesome_follower_arm \
+    --robot.cameras="{ up: {type: orbbec, width: 640, height: 880, fps: 30, focus_area:[60,300]}}" \
+    --teleop.type=so101_leader \
+    --teleop.port=/dev/ttyACM1 \
+    --teleop.id=my_awesome_leader_arm \
+    --display_data=true
+```
+
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/orbbec_result.png" />
+</div>
+
+
+
+For subsequent tasks such as data collection, training, and evaluation, the process is the same as that for regular RGB commands. You only need to replace the relevant part in the regular RGB command with:
+
+```bash
+  --robot.cameras="{ up: {type: orbbec, width: 640, height: 880, fps: 30, focus_area:[60,300]}}" \
+```
+
+You can also add an additional monocular RGB camera afterward.
+
+
+</details>
+
+
+
 :::tip
 The SO100 and SO101 codes are compatible. Users of SO100 can directly utilize SO101's parameters and code for operation.
 :::

docs/Robotics/Robot_Kits/Lerobot/Lerobot_Starai_Arm.md

@@ -294,25 +294,24 @@ sudo chmod 666 /dev/ttyUSB*
 
 ## Calibrate
 
-### For Initial Calibration
+For videos covering the StarAI Robotic Arm from unboxing to teleoperation, you may refer to:
+<div class="video-container">
+<iframe width="900" height="600" src="https://www.youtube.com/embed/02lxxF9Cvy8?si=IGJda5nXkYEbm2N6" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+</div>
 
-Please rotate each joint left and right to the corresponding positions.
+Move the robotic arm to the initial position of the robotic arm (as shown in the figure below) and place it in standby mode, then reconnect the power supply. For the initial position of the new version of the robotic arm, special attention should be paid to ensuring that the servos of Joints 3, 4, and 5 strictly align with the positions in the figure.
 
-### For Re-Calibration
 
-Follow the on-screen prompt: enter the letter "c" and press the Enter key.
+| **Violin Leader Arm** | **Viola Follower Arm** |
+|:---------:|:---------:|
+| ![fig1](https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/violin_rest.jpg) | ![fig2](https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/viola_rest.jpg) |
 
-Below are the reference values. Under normal circumstances, the actual limit reference values should fall within the range of **±10°** of these references.
+Initial position of the old-version robotic arm (special attention should be paid to ensuring that the servos of Joints 3, 4, and 5 strictly align with the positions in the figure; the initial position of the new-version robotic arm can also be used as a reference):
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/Specifications.png" />
+</div>
 
-| Servo ID | Lower Angle Limit (°) | Upper Angle Limit (°) | Notes                                          |
-| -------- | --------------------- | --------------------- | ---------------------------------------------- |
-| motor\_0 | -180°                 | 180°                  | Rotate to the limit position                   |
-| motor\_1 | -90°                  | 90°                   | Rotate to the limit position                   |
-| motor\_2 | -90°                  | 90°                   | Rotate to the limit position                   |
-| motor\_3 | -180°                 | 180°                  | No limit; rotate to the reference angle limits |
-| motor\_4 | -90°                  | 90°                   | Rotate to the limit position                   |
-| motor\_5 | -180°                 | 180°                  | No limit; rotate to the reference angle limits |
-| motor\_6 | 0°                    | 100°                  | Rotate to the limit position                   |
 
 :::tip
 Taking PC (Linux) and Jetson board as examples, the `first` USB device inserted will be mapped to `ttyUSB0`, and the `second` USB device inserted will be mapped to `ttyUSB1`.
@@ -328,6 +327,8 @@ Connect the leader to `/dev/ttyUSB0`, or modify the `--teleop.port` parameter, a
 lerobot-calibrate     --teleop.type=starai_violin --teleop.port=/dev/ttyUSB0 --teleop.id=my_awesome_staraiviolin_arm
 ```
 
+After startup, you will see the encoder values of each joint. You need to manually calibrate each joint one by one: rotate each joint to its maximum and minimum positions. For joints without limit stops, the rotation range must not exceed 180° clockwise or 180° counterclockwise. After calibrating all joints, press Enter to save the settings.
+
 #### Follower Robotic Arm
 
 Connect the follower to `/dev/ttyUSB1`, or modify the `--teleop.port` parameter, and then execute:
@@ -336,7 +337,7 @@ Connect the follower to `/dev/ttyUSB1`, or modify the `--teleop.port` parameter,
 lerobot-calibrate     --robot.type=starai_viola --robot.port=/dev/ttyUSB1 --robot.id=my_awesome_staraiviola_arm
 ```
 
-After running the command, you need to **manually move the robotic arm** to allow each joint to reach its **limit position**. The terminal will display the recorded range data. Once this operation is completed, press Enter.
+After startup, you will see the encoder values of each joint. You need to manually calibrate each joint one by one: rotate each joint to its maximum and minimum positions. For joints without limit stops, the rotation range must not exceed 180° clockwise or 180° counterclockwise. After calibrating all joints, press Enter to save the settings.
 
 :::tip
 The calibration files will be saved to the following paths: `~/.cache/huggingface/lerobot/calibration/robots` and `~/.cache/huggingface/lerobot/calibration/teleoperators`.
@@ -355,6 +356,8 @@ Connect `left_arm_port` to `/dev/ttyUSB0` and `right_arm_port` to `/dev/ttyUSB2`
 lerobot-calibrate     --teleop.type=bi_starai_leader  --teleop.left_arm_port=/dev/ttyUSB0  --teleop.right_arm_port=/dev/ttyUSB2  --teleop.id=bi_starai_leader
 ```
 
+After startup, you will see the encoder values of each joint. You need to manually calibrate each joint one by one: rotate each joint to its maximum and minimum positions. For joints without limit stops, the rotation range must not exceed 180° clockwise or 180° counterclockwise. After calibrating all joints, press Enter to save the settings.
+
 #### Follower Robotic Arm
 
 Connect `left_arm_port` to `/dev/ttyUSB1` and `right_arm_port` to `/dev/ttyUSB3`, or modify the `--robot.left_arm_port` and `--robot.right_arm_port` parameters, and then execute:
@@ -363,6 +366,8 @@ Connect `left_arm_port` to `/dev/ttyUSB1` and `right_arm_port` to `/dev/ttyUSB3`
 lerobot-calibrate     --robot.type=bi_starai_follower  --robot.left_arm_port=/dev/ttyUSB1  --robot.right_arm_port=/dev/ttyUSB3 --robot.id=bi_starai_follower
 ```
 
+After startup, you will see the encoder values of each joint. You need to manually calibrate each joint one by one: rotate each joint to its maximum and minimum positions. For joints without limit stops, the rotation range must not exceed 180° clockwise or 180° counterclockwise. After calibrating all joints, press Enter to save the settings.
+
 :::tip
 
 The difference between single-arm and dual-arm setups lies in the `--teleop.type` and `--robot.type` parameters. Additionally, dual-arm setups require separate USB ports for the left and right arms, totaling four USB ports: `--teleop.left_arm_port`, `--teleop.right_arm_port`, `--robot.left_arm_port`, and `--robot.right_arm_port`.
@@ -376,7 +381,7 @@ If using a dual-arm setup, you need to manually modify the robotic arm file type
 ## Teleoperate
 
 <div class="video-container">
-<iframe width="900" height="600" src="https://www.youtube.com/embed/Uz-x-2P2xaE?si=HJTjALt5yFntR6-s" title="youtube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+<iframe width="900" height="600" src="https://www.youtube.com/embed/02lxxF9Cvy8?si=IGJda5nXkYEbm2N6" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
 </div>
 
 Move the arm to the position shown in the diagram and set it to standby.
@@ -424,10 +429,122 @@ After the program starts, the Hover Lock Technology remains functional.
 
 ## Add cameras
 
+<details>
+<summary> If using the Orbbec Gemini2 Depth Camera </summary>
+
+<div align="center">
+    <img width={800}
+    src="https://media-cdn.seeedstudio.com/media/catalog/product/cache/bb49d3ec4ee05b6f018e93f896b8a25d/0/-/0-101090144--orbbec-gemini-2-3d-camera.jpg" />
+</div>
+<div class="get_one_now_container" style={{textAlign: 'center'}}>
+<a class="get_one_now_item" href="https://www.seeedstudio.com/Orbbec-Gemini-2-3D-Camera-p-6464.html" target="_blank" rel="noopener noreferrer" >
+            <strong><span><font color={'FFFFFF'} size={"4"}> Get One Now 🖱️</font></span></strong>
+</a></div>
+
+
+- 🚀 Step 1: Install the Orbbec SDK Dependent Environment
+
+1. Clone the `pyorbbec` repository
+   ```bash
+   cd ~/
+   git clone https://github.com/orbbec/pyorbbecsdk.git
+   ```
+
+2. Download and install the corresponding **.whl file** for the SDK  
+   Go to [pyorbbecsdk Releases](https://github.com/orbbec/pyorbbecsdk/releases),  
+   select and install based on your Python version. For example:
+   ```bash
+   pip install pyorbbecsdk-x.x.x-cp310-cp310-linux_x86_64.whl
+   ```
+
+3. Install dependencies in the `pyorbbec` directory
+   ```bash
+   cd ~/pyorbbecsdk
+   pip install -r requirements.txt
+   ```
+
+   Force downgrade the `numpy` version to `1.26.0`
+    ```bash
+    pip install numpy==1.26.0
+    ```
+  Red error messages can be ignored.
+
+4. Clone the Orbbec SDK into the `~/lerobot/src/cameras` directory
+
+  ```bash
+  cd ~/lerobot/src/cameras
+  git clone https://github.com/ZhuYaoHui1998/orbbec.git
+  ```
+
+5. Modify utils.py and __init__.py
+- Find `utils.py` in the `~/lerobot/src/lerobot/cameras` directory, and add the following code at line 40:
+
+```python
+elif cfg.type == "orbbec":
+            from .realsense.camera_orbbec import OrbbecCamera
+
+            cameras[key] = OrbbecCamera(cfg)
+```
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/utils.png" />
+</div>
+
+- Find `__init__.py` in the `~/lerobot/src/lerobot/cameras` directory, and add the following code at line 18:
+
+```python
+from .orbbec.configuration_orbbec import OrbbecCameraConfig
+```
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/init.png" />
+</div>
+
+
+- 🚀 Step 2: Function Call and Examples
+
+In all the following examples, replace `starai_viola` with the actual model of the robotic arm you are using (e.g., `so100` / `so101`).
+
+
+We have added the `focus_area` hyperparameter. Since depth data that is too far away is meaningless for the robotic arm (it cannot reach or grasp objects), depth data less than or greater than the `focus_area` will be displayed in black. The default `focus_area` is (20, 600).  
+Currently, the only supported resolution is width: 640, height: 880.
+
+```bash
+lerobot-teleoperate \
+    --robot.type=starai_viola \
+    --robot.port=/dev/ttyUSB1 \
+    --robot.id=my_awesome_staraiviola_arm \
+    --robot.cameras="{ up: {type: orbbec, width: 640, height: 880, fps: 30, focus_area:[60,300]}}" \
+    --teleop.type=starai_violin \
+    --teleop.port=/dev/ttyUSB0 \
+    --teleop.id=my_awesome_staraiviolin_arm \
+    --display_data=true
+```
+
+
+<div align="center">
+    <img width={800}
+    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/starai/orbbec_result.png" />
+</div>
+
+
+For subsequent tasks such as data collection, training, and evaluation, the process is the same as that for regular RGB commands. You only need to replace the relevant part in the regular RGB command with:
+  ```
+  --robot.cameras="{ front: {type: orbbec, width: 640, height: 880, fps: 30, focus_area:(20,600)}}" \
+  ```
+You can also add an additional monocular RGB camera afterward.
+
+
+</details>
+
 <div class="video-container">
 <iframe width="900" height="600" src="https://www.youtube.com/embed/-p8K_-XxW8U?si=UmYWvEyKNPpTRxDC" title="youtube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
 </div>
 
+
+
 After inserting your two USB cameras, run the following script to check the port numbers of the cameras. It is important to remember that the camera must not be connected to a USB Hub; instead, it should be plugged directly into the device. The slower speed of a USB Hub may result in the inability to read image data.
 
 ```bash