Webcam Navigation

[Illyvia]

Is there a way to get the create3 to be able to navigate around my house, or follow an object using a usb webcam and a raspberry pi?

[alsora]

Yes, it's possible.

The create 3 is only a mobility platform, I.e. it exposes commands to move the robot around (for example drive to a certain pose, drive straight, or pure velocity actuation).

You can do whatever you want with this capabilities, by developing some intelligence algorithms on top of them.
A mobility platform is the basic building block required for all mobile robotics applications.

The docs show how to connect a raspberry pi to the robot https://iroboteducation.github.io/create3_docs/hw/rpi_hookup/

On your raspberry pi you will have to implement your navigation or object following components.

However, note that your question is very broad.
Depending on the expected result, the complexity of the algorithms you will have to develop will range from trivial to very complex.

[Illyvia]

Thanks,
Do you have any recommendations for already made software, such as using OpenCV with teleop_twist_keyboard?

[alsora]

No, I do not have many recommendations.
As I said, it really depends on what you want to do.

Tracking arbitrary objects with a monocular camera (like a webcam) is a complex task: you can find some papers describing the problem here: https://paperswithcode.com/task/monocular-3d-object-detection.
Using fiducials such as apriltags would be much easier. See for example https://github.com/christianrauch/apriltag_ros.

An example of a full pipeline similar to what you are describing can be found here: https://github.com/intel/ros2_object_analytics.
This library is discontinued and it uses a stereo camera, but it will give you an idea of the various pieces involved.

[Illyvia]

Ok, Thanks anyway!

[justinIRBT]

For general create3 teleop, there is an example here:
https://github.com/iRobotEducation/create3_examples/tree/galactic/create3_teleop

For writing a navigation stack for the create, there is an example of a navigation stack that does a coverage algorithm here:
https://github.com/iRobotEducation/create3_examples/tree/galactic/create3_coverage

For your application, you can use those examples for how to drive the create, but you will need to find examples on how to do the camera sensor processing from other source (like OpenCV)

[Illyvia]

Ok, thanks