e1 - Manipulator to AGV Calibration (Unsuccessful)

[JorgeFernandes-Git]

Single Calibrate a Manipulator with RGB astra camera on the end-effector, mounted on top of an AGV. (Unsuccessful)

Transformation from camera to end-effector was previously calculated in e0_rgb2ee.

https://github.com/JorgeFernandes-Git/zau_bot/tree/main/e1_arm2agv

---

Summary:

• Using ATOM framework to manipulator to AGV calibration.
• Aims to find the transformation of the base's manipulator w.r.t the AGV.
• The camera used was a depth astra.
• The calibration used a total of 50 collections.
• The optimization didn't converge (see videos).

---

---

ATOM config file:

This is how I define in the config.yaml 'sensor' part:

zau_bot/e1_arm2agv/e1_arm2agv_calibration/calibration/config.yml

Lines 75 to 81 in c2133a5

	base_link:
	link: "base_link"
	parent_link: "base_link_mb"
	child_link: "base_link"
	topic_name: "/camera/rgb/image_raw"
	throttle: 5
	modality: "rgb"

---

Videos

• Set Initial Estimate: https://youtu.be/DbFU10SCOL8
• Run Calibration: https://youtu.be/JhHzo6qumo0

Commands:

Record bag file:

roslaunch e0_rgb2ee_record_bag record_sensor_data.launch

Run script for move the manipulator:

rosrun zau_bot_bringup execute_trajectories 

Play dataset

roslaunch e1_arm2agv_calibration dataset_playback.launch

rosrun atom_calibration dataset_playback -json /home/jorge/datasets/e1_arm2agv/dataset.json -ow

Unfortunately, the calibration didn't go well. I think it felt into a local minima.

---

I recorded several bags, this last one, as a total of 50 collections. Even so, didn't converge.

Any suggestions?

[JorgeFernandes-Git]

Hello @miguelriemoliveira.

I've tried different things today and the problem remains, or the configure file didn't run at all.

When you have the time, take a look, to give me some help.

[miguelriemoliveira]

Hi @JorgeFernandes-Git ,

I think that this calibration cannot be done. I think the experiment is not well defined.
There is no information given to the calibration system that would allow it to accurately estimate the transformation you want to estimate.

It is like doing a hand eye calibration, with the camera mounted on the end effector, and asking for a transformation from the base of the robot to the door of my house. It is not possible to know.
I know I am not being very successful explaining why it cannot be done, but that's my feeling.
@manuelgitgomes do you agree?

[manuelgitgomes]

Hello both @JorgeFernandes-Git and @miguelriemoliveira.

I understand your argument professor, but I believe this was one of the key discoveries Jorge needed to find out in his dissertation.
This was a key transformation needed to be found. If this experiment does not work, I believe it implies the next experiments won't work either, because all depend on the estimation of this transformation. Am I thinking correctly @JorgeFernandes-Git?

@JorgeFernandes-Git, the results seemed to be hitting on a local minimum. What -nig were you using? Try diminishing it to see if it affects the results.

[JorgeFernandes-Git]

I think I understood @miguelriemoliveira .

I notice that, when launching the set initial estimate, a frustum appears in the base_link of the manipulator pointing to the ceiling. In the video, I uncheck it, but it was there.

In the graph tree, the green circle is around the base_link too, when it should be in the camera_rgb_optical_frame.

@manuelgitgomes, I run the calibration with just this commands -v -rv -si -vo. Do you recommend a value for the -nig ? I'm not sure which to use.

Anyway, it was sad to find out. Should I make the other experiments?
Are there any changes I can make in the ATOM to be able to solve this or work it's way around?

[miguelriemoliveira]

Hi @JorgeFernandes-Git ,

let's talk today at 11h. Send me your phone number to my email and I call you.