NonlinearSolve stalls when jacobian is singular

[oscardssmith]

julia> using NonlinearSolve

julia> f(u, p) = [u[1]^2 - 2u[1] + 1, sum(u)]
f (generic function with 1 method)

julia> prob = NonlinearProblem(f, [1.0, 1.0])
NonlinearProblem with uType Vector{Float64}. In-place: false
u0: 2-element Vector{Float64}:
 1.0
 1.0

julia> sol = solve(prob)
retcode: Stalled
u: 2-element Vector{Float64}:
 1.0
 1.0

julia> sol.resid
2-element Vector{Float64}:
 0.0
 2.0

We should probably do something if our initial step doesn't change anything. Possibly epsilon perturbation? possibly warn via the Verbosity system (@jClugstor)

[ChrisRackauckas]

We would want it to be deterministic. There are methods which are made for this kind of thing:

• https://www.jstor.org/stable/2156931
• https://www.sciencedirect.com/science/article/pii/S0377042708001787
• https://ocw.mit.edu/courses/6-336j-introduction-to-numerical-simulation-sma-5211-fall-2003/33912d2b3446ff62fcd51b7238469f10_lec10.pdf
• https://dl.acm.org/doi/10.5555/1090567.1642750
• https://arxiv.org/pdf/2207.12334

But also Levenburg-Marquedt should also be sufficient?

[ChrisRackauckas]

julia> sol = solve(prob, LevenbergMarquardt())
retcode: Success
u: 2-element Vector{Float64}:
  0.9999996541895417
 -0.9999996541895417

[ChrisRackauckas]

julia> using LineSearches

julia> HagerZhang() = LineSearchesJL(; method = LineSearches.HagerZhang())
HagerZhang (generic function with 1 method)

julia> MoreThuente() = LineSearchesJL(; method = LineSearches.MoreThuente())
MoreThuente (generic function with 1 method)

julia> sol = solve(prob, NewtonRaphson(; linesearch = HagerZhang()))
retcode: InternalLineSearchFailed
u: 2-element Vector{Float64}:
 1.0
 1.0

julia> sol = solve(prob, NewtonRaphson(; linesearch = MoreThuente()))
retcode: InternalLineSearchFailed
u: 2-element Vector{Float64}:
 1.0
 1.0

julia> sol = solve(prob, NewtonRaphson(; linesearch = LineSearch.BackTracking()))
retcode: Stalled
u: 2-element Vector{Float64}:
 1.0
 1.0

julia> sol = solve(prob, TrustRegion())
retcode: Stalled
u: 2-element Vector{Float64}:
 1.0
 1.0