NTU CSIE5074 Robotics 2023/24 Final Project, Team 5
This GitHub repository contains the code files and report for the final project competition in the course CSIE5074 (Robotics 2023-24) at National Taiwan University (NTU). Click below image to watch our demo video!
We built a supervisory teleoperation kendo robot system. The robot is able to track the opponent’s pose and wait for the operator’s command to attack.
To detect and track the human body, we utilized a deep-learning-based human pose estimation algorithm (LOGO-CAP, CVPR 2022). By aligning the RGB and DEPTH images from the RealSense D435, we obtained the 3D coordinates of the human body.
An interactive GUI was created using JavaScript for visualization and real-time implementation. With just one click on the GUI button, the operator can send an attack command to ROS (via a WebSocket), which controls the robot arm. The GUI is handcrafted based on ganja.js, a web-programmable platform developed by the geometric algebra community. below image to enter the interactive GUI.
Visualization, inverse kinematics, and trajectory planning are all handled using conformal geometric algebra (CGA), a powerful mathematical framework for representing geometry. CGA provides numerous geometric insights into the robot that are nearly impossible to achieve using traditional matrix methods.
- Wei-Hsuan Cheng (Team Leader): [email protected]/[email protected]
- Wen Perng: [email protected]
- Che-Jung Chuang: [email protected]
- Cheng-Yen Yu: [email protected]