CMP3103 Week 3

Make sure you keep all your code you develop in the workshops and also note down any commands you used (create a README.md file for your notes). Maybe, a good idea is to actually keep all this in your own https://github.com repository (even share within your group). You will need this again as you go along in this module.

In the lecture, you have been introduced to ways to conduct image processing in Python using OpenCV. In this workshop, you shall learn how to

1. retrieve images from ROS topics
2. convert images into the OpenCV format
3. perform image processing operations on the image
4. (optionally) command the robot based on your image processing output

To get you off the ground quickly, all the source code shown in the lecture is available online. In particular, have a look at

• opencv_intro.py which shows you how to load an image in OpenCV without ROS and address it. Also look at the official OpenCV Python tutorials to gain a better understanding
• opencv_bridge.py showing you how to use CvBridge to read image from a topic.
• color_contours.py to get an idea about colour slicing as introduced in the lecture. Also read about Changing Colour Spaces.

Tasks

1. Develop Python code with the following abilities:

   1. Take the example code fragment opencv_bridge.py from the lecture and modify it so you can read from the camera of your (simulated) turtlebots.
   2. read images from your robot, display them using OpenCV methods, and try out colour slicing as presented in the lecture to segment a coloured object of your choice. When trying this in simulation, put some nice coloured objects in front of the robot. Find suitable parameters to robustly segment that blob. You may take color_contours.py as a starting point for your implementation.
   3. Use the output from above to then publish std_msgs/String messages on a Publisher that contains information about the outcome of the operation (e.g. print the mean value of the pixel intensities of the resulting image). (Hint: You'll need to create a Publisher with type std_msgs/String for this: p=rospy.Publisher('/result_topic', String) and then publish to it.

2. Try out the "followbot" presented in the lecture. Take the code from https://github.com/LCAS/teaching/tree/lcas_melodic/ros_book_line_follower/src described in chapter 12 of the "Programming Robots with ROS" book, available also on blackboard. You need to run sudo apt update && sudo apt install ros-melodic-ros-book-line-follower to install the required ROS package, and afterwards either start a new terminal, our run the command source /opt/ros/melodic/setup.bash in your active terminal to ensure gazebo finds all its resources. Then launch the course with roslaunch ros_book_line_follower course.launch in afterwards. Then, in a separate terminal (or VSCode) run one of the files taken from https://github.com/LCAS/teaching/tree/lcas_melodic/ros_book_line_follower/src.

   To summarise the minimal requirements for this week:

   1. Develop Python code that subscribes to the image stream from the robot
   2. Publish the output of some image processing as a std_msgs/String on a topic named /result_topic
   3. Run the line follower and understand its code

3. (Optional) Research about Hough Transform and see how it can be used to detect lines with OpenCV for Python. Understand the concepts of Hough transform from your research and then also look at the circle detection code in hough_circle.py. Make it work with actual image data received via a topic from your (simulated/real) robot.

4. (Optional) Get a real robot and make it find a colourful object. Note: You cannot easily use display the OpenCV Image on the robots from Jupyter. You will have to use the VPN setup to connect your computer to the turtlebots for it to work. You can of course run just some python code without any graphical windows easily in Jupyter.

5. (Optional) The real fun starts here: Develop code that makes the real robot chase a colour.

Also, browse through this collection of useful resources beyond what has been presented in the lecture: OpenCV and ROS