Trust Region Reflective Algorithm

[avik-pal]

Is your feature request related to a problem?

Special Trust Region Algorithm for NLLS Problems with bound constraints.

Describe the solution you’d like

See https://nmayorov.wordpress.com/2015/06/19/trust-region-reflective-algorithm/ for a description of the algorithm.

Additional context

First, we need to add bound constraints to the NLLS Problem in SciMLBase

[vcantarella]

Hi, Thanks for putting the effort. How are developments in this area? In my experience, the trust region reflective algorithm is very useful for not so well posed problems. Many of my ODE models reproducing real world experiments are fitted only on such algorithms.
Your link refers to the Coleman and Li algorithm which is included in Matlab and Scipy. I have looked at the references for the other algorithms included in NonlinearSolve.jl and they come from this revision paper: trust-region revision that reference a new affine scale transformation for bound constraints trust region described in this paper: new affine scaling. I am not an expert in the field, and I dont know how superior/inferior is the new algorithm, but since things are being developed it could be useful to write also this different affine scaling and check the performance . I took a first look at the code in Nonlinearsolve.jl and in principle I think these algorithms can be implemented without much hassle, but I couldn't get through everything on my own. I'm happy to help further to have this implemented, but someone more experienced would be necessary.
Anyway, thanks for the efforts so far.
Cheers,