Implement PoVar in sliding-window-based VIO

[empty-spacebar]

Hi, thanks for open-sourcing PoVar.

Since you implement PoVar in BAL rather than problems with smaller scale like VIO, my question is, could sliding-window-based VIO benifited from PoVar on speed or accuracy?

Before I replace rootBA on basalt with PoVar, your advise may help me a lot.

Thanks.

[simonwebertum]

Hi, thanks for your message!
The main idea of PoVar (and broadly speaking, variable projection algorithm) is to skip the initialization process of landmarks and camera parameters (via triangulation, rotation averaging, etc). It can be worth to try if this first block is complicated, slow, or leads to bad results (as it can be often the case when considering structure-from-motion). The cost of VarPro is the formulation of the uncalibrated (or calibrated with known intrinsics) problem under a separable non-linear least squares problem -- for some problems the cost is negligible tho.
Also, concerning the solver of VarPro, our proposed PoVar works very well with geometric bundle adjustment, as the Hessian is block-diagonal (and then the inverse of both diagonal blocks, landmarks and pose, is easily derivable). I guess it should be difficult to use the power series in case of dense Hessian (that is, for photometric loss, that is often the case for VIO). For this problem, some solvers like PCG (also implemented in our implementation) should be more efficient.