Camera Calibration

[roaldarbol]

Hi again!

I'm trying to undistort a video feed where the camera has some a bit of fish-eye distortion. I think a combination of CameraCalibration and Undistort might do the trick, but I can't find much information about how. Do I need to produce a calibration board, if so, are there any requirements? Any help and pointers appreciated!

[banchan86]

Hi @roaldarbol have not tried it myself, but these links might give you a headstart

https://groups.google.com/g/bonsai-users/c/4Dg1NfmVtDA/m/Ciadrh2FAAAJ
https://groups.google.com/g/bonsai-users/c/Rh9ruulQ5xA/m/fC6Cvt1ABAAJ
https://groups.google.com/g/bonsai-users/c/j9Nwbj7sEbQ/m/VoCRWQkCAwAJ

From what I can gather,

1. You need to print this pattern from https://github.com/opencv/opencv/blob/master/doc/pattern.png.
2. You use CameraCapture -> FindChessboardCorners -> CalibrateCameraChessboard to find the camera intrinsics.
3. Finally you can use CameraCapture -> Undistort with possibly a ResizeCanvas before Undistort to avoid dropping pixels from the border.

[bruno-f-cruz]

There seems to be a weird behavior indeed (I have never used this operator, so not sure). In the meantime, this should fix it (albeit rather inefficiently since it seems we are triggering this method

bonsai/Bonsai.Vision/IntrinsicsTransform.cs

Line 138 in b2162b1

internal void UpdateIntrinsics()

on each property change).

[roaldarbol]

Cool, thanks - good to know that it at least is not just on my end. :-) For now, I'll just run it once and then comment out the LoadIntrinsics and PropertyMapping on the subsequent runs. For the above, will it still just update the mapping once, and then apply on the stream? Or is it loading/updating the mapping on each new frame?

[roaldarbol]

@bruno-f-cruz Actually the above doesn't work for me, I get an error saying:

[bruno-f-cruz]

Sorry I should have sent the script since the difference is visually subtle. You want to use an InputMapping, not PropertyMapping:

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.9.0"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns:rx="clr-namespace:Bonsai.Reactive;assembly=Bonsai.Core"
                 xmlns:io="clr-namespace:Bonsai.IO;assembly=Bonsai.System"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:CameraCapture">
          <cv:Index>0</cv:Index>
          <cv:CaptureProperties />
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:LoadIntrinsics">
          <cv:FileName>calib_changed.yaml</cv:FileName>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:WithLatestFrom" />
      </Expression>
      <Expression xsi:type="InputMapping">
        <PropertyMappings>
          <Property Name="ImageSize" Selector="Item2.ImageSize" />
          <Property Name="FocalLength" Selector="Item2.FocalLength" />
          <Property Name="PrincipalPoint" Selector="Item2.PrincipalPoint" />
          <Property Name="RadialDistortion" Selector="Item2.RadialDistortion" />
          <Property Name="TangentialDistortion" Selector="Item2.TangentialDistortion" />
        </PropertyMappings>
        <Selector>Item1</Selector>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:Undistort">
          <cv:ImageSize>
            <cv:Width>1280</cv:Width>
            <cv:Height>1024</cv:Height>
          </cv:ImageSize>
          <cv:Alpha>0.54758981445192234</cv:Alpha>
          <cv:FocalLength>
            <cv:X>1000.4662597430984</cv:X>
            <cv:Y>1803.8508448487521</cv:Y>
          </cv:FocalLength>
          <cv:PrincipalPoint>
            <cv:X>900.11841853024725</cv:X>
            <cv:Y>700.3926564849362</cv:Y>
          </cv:PrincipalPoint>
          <cv:RadialDistortion>
            <cv:X>-1.6</cv:X>
            <cv:Y>-1.7347352548268997</cv:Y>
            <cv:Z>4.75776620580356</cv:Z>
          </cv:RadialDistortion>
          <cv:TangentialDistortion>
            <cv:X>0.0057550980570318464</cv:X>
            <cv:Y>-0.028204764237863653</cv:Y>
          </cv:TangentialDistortion>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="io:WriteLine" />
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:Take">
          <rx:Count>1</rx:Count>
        </Combinator>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="2" Label="Source1" />
      <Edge From="1" To="2" Label="Source2" />
      <Edge From="2" To="3" Label="Source1" />
      <Edge From="2" To="5" Label="Source1" />
      <Edge From="3" To="4" Label="Source1" />
      <Edge From="5" To="6" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

[roaldarbol]

Ah, I see. Unfortunately it seems that it's the incorrect calibration (that I was seeing after restart) that I end up with from that workflow.