This project contains the visual cognition and motion planning part of a cognitive wiping robot. The task is assumed to be wiping off the words written on a white board.
Install Intel® RealSense™ SDK 2.0 following official instruction
This project uses python 3.7. To install required packages:
pip install -r requirements.txt
Detect the largest contour max_cnt as working area.
Optionally:
- Detect the bounding rectangular or
- Detect the contour of largest white area.
Default: Reserve only contours inside max_cnt as stains to be cleaned. Add centers of all contours (marked as red) as nodes for planning algorithms.
Option 1: Bounding rectangles can be used instead of contours.
However, the size of rectangular kernel needs to be set manually to get a suitable bounding rectangle.
The wiping task can be formed as a Covering Traveling Salesman Problem. To guarantee that the cleaner will pass each pixel of stains, each point on contours c is checked if it lies within an e-neighborhood of current nodes, where the value of e depends on the size of the tool (e.g. radius of the brush) used to accomplish the task. If not, the middle point between c and the nearst node is added to the node_list until c is covered.
As is shown in the image above, blue point stands for the starting point, red points are all the nodes for planning algorithms and green shadows marks the area the brush covers.
The task of passing all the nodes is then formed as a Traveling Salesman Problem, and will be solved by following algorithms.
Starting from the initial position (current position of end effector, marked as blue), select the nearst node to be the next move.
The path is planned by dynamic programming. However, the computation time is relatively long.
The path is planned by ant colony algorithm. Area covered by the brush along the path is marked as blue shadow.
To detect stains and plan the motion of the end-effector:
python viewer.py
Results above were obatined under the assumption that nodes are all connected to each other. This guarantees that a path will be found, but also leads to computational burdens in planning. To speed up the planning process, a graph G=(V,E) is constructed following steps in J.Hess2012. Each node is only connected to top k nearest neighbors within a radius r. In this way, number of edges to be explored is cut down.
