Skip to content

At linesearchmax, select alpha with minimum value#185

Merged
pkofod merged 3 commits intoJuliaNLSolvers:masterfrom
kbarros:master
Nov 19, 2025
Merged

At linesearchmax, select alpha with minimum value#185
pkofod merged 3 commits intoJuliaNLSolvers:masterfrom
kbarros:master

Conversation

@kbarros
Copy link
Contributor

@kbarros kbarros commented Nov 12, 2025

This PR modifies the behavior when Hager-Zhang line search hits the maximum number of iterations without being able to bracket the step size alpha. Prior to this PR, HZ returns the smallest alpha (alphas[ia] == 0.0) in its LineSearchException. After this PR, HZ returns the alpha associated with the lowest value, i.e. alphas[argmin(values)].

This failure case can arise when using Optim with the Manifold feature. Consider, for example, a sphere manifold. If the initial condition is the north pole and the optimizer wants to move to the equator, then an "infinite" step size alpha is formally required. This is because the retract! feature of Manifold uses simple projection rather than exponential transport. Since HZ line search cannot actually bracket an infinite alpha, it eventually gives up. When this happens, it's better to accept a very large step size rather than take a zero step size.

Even after this PR is merged, it is still a bit tricky to make the Optim Manifold optimization reliable. If the line search fails, Optim's ConjugateGradient (or similar) will terminate with termination_code(res) == TerminationCode.FailedLinesearch. Some user-side logic is needed to detect this case and restart CG.

Another source of brittleness occurs when line search succeeds, but the step size alpha is divergently large (e.g., alpha of order 1e23 was showing up in some of my tests). In this case, CG's next line search is likely to fail, because its initial guess for alpha will be based on the previous line search's accepted alpha. Unfortunately, here, the HZ line search will not have enough iterations available to decrease alpha back to order 1. I found a good workaround for this case is to explicitly bound the maximum initial alpha by some constant αmax. This is possible with, e.g.,

    method = Optim.ConjugateGradient(; alphaguess=LineSearches.InitialHagerZhang(; αmax=10.0),
                                       manifold=CustomManifold())

For example, when optimizing on a sphere of unit radius, the choice αmax=10.0 seems fine.

Finally, I'll mention something else that does not work. Rather than the bounded alphaguess object, I instead tried passing linesearch=LineSearches.HagerZhang(; alphamax=10.0) to the ConjugateGradient constructor. This just leads to an unrecoverable assertion error, rather than a LineSearchException. So this is not a way to bound the maximum step size in the line search.

Co-authored-by: Patrick Kofod Mogensen <patrick.mogensen@gmail.com>
@pkofod pkofod merged commit 57d805b into JuliaNLSolvers:master Nov 19, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants