Regularized Quasi-Newton operators

[MaxenceGollier]

It would be useful to have an implementation of the operator $$v\mapsto Hv + \sigma v$$, where $$H$$ is some quasi-Newton operator and $$\sigma > 0$$. We should be able to get the inverse of this operator cheaply, see https://arxiv.org/pdf/1911.04584 .

cc @MohamedLaghdafHABIBOULLAH, i don't know if you plan to implement this soon ? I might do it on my own as well, I will need it for my research.

[MohamedLaghdafHABIBOULLAH]

If you need it immediately, feel free to open a PR, and we can review it together.

[MaxenceGollier]

Actually, solve_shifted_system! does what I need.

[MohamedLaghdafHABIBOULLAH]

Indeed, it performs the inverse operation, but only for LBFGSOperator.

[dpo]

@farhadrclass Already implemented the exact solve of $$(B + \sigma I) x = b$$ for LBFGS. It can be implemented for other limited-memory quasi-Newton operators but I believe we need to implement the compact storage first (there was an attempt in #95).

Useful references:

• https://www.sciencedirect.com/science/article/pii/S0024379512001358
• https://onlinelibrary.wiley.com/doi/full/10.1002/nla.2186
• https://ieeexplore.ieee.org/abstract/document/6810448
• https://www.tandfonline.com/doi/full/10.1080/10556788.2014.894045

Note that, more generally, it’s possible to solve $(B + G) x = b$ where $G$ is symmetric provided the user supplies a function to perform a solve with $G$. That would be a great contribution.

[MohamedLaghdafHABIBOULLAH]

Interesting @dpo! That will be useful for the Becker/Kanzow proxes as well.