Enhancement : add iteratively regularized Gauss–Newton method

[aTrotier]

BART used the iteratively regularized Gauss–Newton method, it might be interesting to mimics their implementation ?

The BART implementation is described here :
Wang X, Tan Z, Scholand N, Roeloffs V, Uecker M. Physics-based reconstruction methods for magnetic resonance imaging. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2021;379(2200):20200196. doi:[10.1098/rsta.2020.0196](https://doi.org/10.1098/rsta.2020.0196)

However, the high non-convexity of model-based reconstruction makes this method sensible to the initial guess and relative scaling of the derivatives of each parameter map. These issues can often be addressed with a reasonable initial guess and a proper preconditioning. Algorithms to solve the nonlinear inverse problems include gradient descent, the variable projection methods [7,32], the method of nonlinear conjugate gradient [33] and Newton-type methods [34]. Particularly for the examples presented in this paper, we solve equation (3.1) via an iteratively regularized Gauss–Newton method (IRGNM) [34] where the nonlinear problem in equation (3.1) is linearized in each Gauss–Newton step, i.e.

xn+1 = argmin x ‖DF(xn)(x − xn) + F(xn) − y‖22 + ∑ i λiRi(x), (3.2)

with DF(xn) the Jacobian matrix of F at the point xn. The regularized linear subproblem can be further solved by conjugate gradients, FISTA [35] or ADMM [36].