Combining visual servoing and CNN

[AssanhajiMohamed]

it is possible to use CNN to track moving objects based on their features and combine this with visual servoing to command a robotic arm to grab said objects ?

[fspindle]

Yes, of course.
For example, you can use Yolo v7 to detect objects and use visual servoing to grasp them.

[AssanhajiMohamed]

great thanks for your response! so i need to have opencv and Yolo v7 as a 3rdParty along with the visp library to benefit from the combination of both ?

[fspindle]

yes

[shrutichakraborty]

Hello! I am working on a similar approach. In this case what vidual features would you recommend to use? I would like to position the robot arm perpendicular to the centre of the bounding box of m detected object. Using a detection network such as Yolov8 I am able to get the bounding box or perhaps even segmentation mask of the desired object to track and servo, would you say that using the centre of gravity and the depth would be sufficient? Is it possible to combine different types of features and then compute the interaction matrix? Any code examples/suggestions would be great ! Thank you!