Calibration Fails to Converge with OV9281 Stereo Cameras on Raspberry Pi 5

[omeredemen]

I’m working with two OV9281 cameras connected to a Raspberry Pi 5. I use libcamera to capture frames and publish them as ROS 2 topics. I’ve also written custom tools to monitor camera timestamps, periods, and frequencies, and can share the relevant code if needed.

My goal is to use these cameras as a stereo pair, so I’m attempting to calibrate them using Kalibr. However, every calibration attempt fails with the following error:
Did not converge in maxIterations... restarting...
full output looks like in following:

Processed 101 of 762 views with 39 views used

Calibration results 
====================
Camera-system parameters:
cam0 (/camera0):
    type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>
    distortion: [-0.4229514   0.16643629  0.00292668  0.01065282] +- [ 0.00076094  0.00033025  0.00020045  0.00019302]
    projection: [ 874.162641    875.01694623  471.02674756  288.2878431 ] +- [ 0.00000081  0.00000048  0.00000036  0.00000035]
    reprojection error: [-0.086532, 0.039480] +- [7.683208, 7.300174]

cam1 (/camera1):
    type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>
    distortion: [-0.40936821  0.19604773  0.00103892 -0.00639483] +- [ 0.00078383  0.00035286  0.00022148  0.00022154]
    projection: [ 861.00675236  851.46585366  483.56410741  292.6431852 ] +- [ 0.0000005   0.00000011  0.00000036  0.00000035]
    reprojection error: [0.089005, -0.046788] +- [7.617974, 7.773887]

baseline T_1_0:
    q: [ 0.01780457  0.01528229  0.00159401  0.99972342] +- [ 0.00031418  0.00031181  0.00018986]
    t: [ 0.08635207 -0.00524665 -0.00003081] +- [ 0.00006497  0.00006674  0.00007053]



Target configuration
====================

  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.024 [m]
    Spacing 0.00699999999984 [m]

------------------------------------------------------------------
  Progress 104 / 762 	 Time remaining: 1m 10s                  [ERROR] [1749108155.806507]: Did not converge in maxIterations... restarting...
[ WARN] [1749108155.818052]: Optimization diverged possibly due to a bad initialization. (Do the models fit the lenses well?)
[ERROR] [1749108155.820721]: Max. attemps reached... Giving up...

This happens consistently, even after trying with multiple ROS bag files recorded under different conditions.

Here is the calibration command I’m using:

rosrun kalibr kalibr_calibrate_cameras \
  --bag ov9281.bag \
  --target aprilgrid.yaml \
  --topics /camera0 /camera1 \
  --models pinhole-equi pinhole-equi \
  --bag-freq 10.0

I've also tested with the pinhole-radtan model, but the problem persists.

• Both cameras run at ~30 Hz and appear synchronized.

Any suggestions for resolving this would be greatly appreciated. I can provide example bags, AprilGrid config, and image samples if needed.

[WorstCodeWay]

hi @omeredemen , have you solved this problem. If so, would you please show your solution here? I meet this problem too.

[omeredemen]

hi @WorstCodeWay, for my case, the important thing was that the timestamps of camera messages should match and period should be stable. If you are using same setup can consider it but there may be some other reasons to trigger this issue. I hope it helps.

[WorstCodeWay]

Thanks @omeredemen. This is indeed a helpful answer. I'll check my timestamps.