|
| 1 | +""" |
| 2 | + Stepsize |
| 3 | +
|
| 4 | +An abstract type for the functors representing step sizes. These are callable |
| 5 | +structures. The naming scheme is `TypeOfStepSize`, for example `ConstantStepsize`. |
| 6 | +
|
| 7 | +Every Stepsize has to provide a constructor and its function has to have |
| 8 | +the interface `(p,o,i)` where a [`AbstractManoptProblem`](@ref) as well as [`AbstractManoptSolverState`](@ref) |
| 9 | +and the current number of iterations are the arguments |
| 10 | +and returns a number, namely the stepsize to use. |
| 11 | +
|
| 12 | +The functor usually should accept arbitrary keyword arguments. Common ones used are |
| 13 | +* `gradient=nothing`: to pass a pre-calculated gradient, otherwise it is computed. |
| 14 | +
|
| 15 | +For most it is advisable to employ a [`ManifoldDefaultsFactory`](@ref). Then |
| 16 | +the function creating the factory should either be called `TypeOf` or if that is confusing or too generic, `TypeOfLength` |
| 17 | +
|
| 18 | +# See also |
| 19 | +
|
| 20 | +[`Linesearch`](@ref) |
| 21 | +""" |
| 22 | +abstract type Stepsize end |
| 23 | + |
| 24 | +get_message(::S) where {S <: Stepsize} = "" |
| 25 | + |
| 26 | +""" |
| 27 | + default_stepsize(M::AbstractManifold, ams::AbstractManoptSolverState) |
| 28 | +
|
| 29 | +Returns the default [`Stepsize`](@ref) functor used when running the solver specified by the |
| 30 | +[`AbstractManoptSolverState`](@ref) `ams` running with an objective on the `AbstractManifold M`. |
| 31 | +""" |
| 32 | +default_stepsize(M::AbstractManifold, sT::Type{<:AbstractManoptSolverState}) |
| 33 | + |
| 34 | +""" |
| 35 | + max_stepsize(M::AbstractManifold, p) |
| 36 | + max_stepsize(M::AbstractManifold) |
| 37 | +
|
| 38 | +Get the maximum stepsize (at point `p`) on manifold `M`. It should be used to limit the |
| 39 | +distance an algorithm is trying to move in a single step. |
| 40 | +
|
| 41 | +By default, this returns $(_link(:injectivity_radius))`(M)`, if this exists. |
| 42 | +If this is not available on the the method returns `Inf`. |
| 43 | +""" |
| 44 | +function max_stepsize(M::AbstractManifold, p) |
| 45 | + s = try |
| 46 | + injectivity_radius(M, p) |
| 47 | + catch |
| 48 | + is_tutorial_mode() && |
| 49 | + @warn "`max_stepsize was called, but there seems to not be an `injectivity_raidus` available on $M." |
| 50 | + Inf |
| 51 | + end |
| 52 | + return s |
| 53 | +end |
| 54 | +function max_stepsize(M::AbstractManifold) |
| 55 | + s = try |
| 56 | + injectivity_radius(M) |
| 57 | + catch |
| 58 | + is_tutorial_mode() && |
| 59 | + @warn "`max_stepsize was called, but there seems to not be an `injectivity_raidus` available on $M." |
| 60 | + Inf |
| 61 | + end |
| 62 | + return s |
| 63 | +end |
| 64 | + |
| 65 | +""" |
| 66 | + Linesearch <: Stepsize |
| 67 | +
|
| 68 | +An abstract functor to represent line search type step size determinations, see |
| 69 | +[`Stepsize`](@ref) for details. One example is the [`ArmijoLinesearchStepsize`](@ref) |
| 70 | +functor. |
| 71 | +
|
| 72 | +Compared to simple step sizes, the line search functors provide an interface of |
| 73 | +the form `(p,o,i,X) -> s` with an additional (but optional) fourth parameter to |
| 74 | +provide a search direction; this should default to something reasonable, |
| 75 | +most prominently the negative gradient. |
| 76 | +""" |
| 77 | +abstract type Linesearch <: Stepsize end |
| 78 | + |
| 79 | +@doc """ |
| 80 | + s = linesearch_backtrack(M, F, p, X, s, decrease, contract η = -X, f0 = f(p); kwargs...) |
| 81 | + s = linesearch_backtrack!(M, q, F, p, X, s, decrease, contract η = -X, f0 = f(p); kwargs...) |
| 82 | +
|
| 83 | +perform a line search |
| 84 | +
|
| 85 | +* on manifold `M` |
| 86 | +* for the cost function `f`, |
| 87 | +* at the current point `p` |
| 88 | +* with current gradient provided in `X` |
| 89 | +* an initial stepsize `s` |
| 90 | +* a sufficient `decrease` |
| 91 | +* a `contract`ion factor ``σ`` |
| 92 | +* a search direction ``η = -X`` |
| 93 | +* an offset, ``f_0 = F(x)`` |
| 94 | +
|
| 95 | +## Keyword arguments |
| 96 | +
|
| 97 | +$(_var(:Keyword, :retraction_method)) |
| 98 | +* `stop_when_stepsize_less=0.0`: to avoid numerical underflow |
| 99 | +* `stop_when_stepsize_exceeds=`[`max_stepsize`](@ref)`(M, p) / norm(M, p, η)`) to avoid leaving the injectivity radius on a manifold |
| 100 | +* `stop_increasing_at_step=100`: stop the initial increase of step size after these many steps |
| 101 | +* `stop_decreasing_at_step=`1000`: stop the decreasing search after these many steps |
| 102 | +* `report_messages_in::NamedTuple = (; )`: a named tuple of [`StepsizeMessage`](@ref)s to report messages in. |
| 103 | + currently supported keywords are `:non_descent_direction`, `:stepsize_exceeds`, `:stepsize_less`, `:stop_increasing`, `:stop_decreasing` |
| 104 | +* `additional_increase_condition=(M,p) -> true`: impose an additional condition for an increased step size to be accepted |
| 105 | +* `additional_decrease_condition=(M,p) -> true`: impose an additional condition for an decreased step size to be accepted |
| 106 | +
|
| 107 | + These keywords are used as safeguards, where only the max stepsize is a very manifold specific one. |
| 108 | +
|
| 109 | +# Return value |
| 110 | +
|
| 111 | +A stepsize `s` and a message `msg` (in case any of the 4 criteria hit) |
| 112 | +""" |
| 113 | +function linesearch_backtrack( |
| 114 | + M::AbstractManifold, f, p, X::T, s, decrease, contract, η::T = (-X), f0 = f(M, p); kwargs... |
| 115 | + ) where {T} |
| 116 | + q = allocate(M, p) |
| 117 | + return linesearch_backtrack!(M, q, f, p, X, s, decrease, contract, η, f0; kwargs...) |
| 118 | +end |
| 119 | + |
| 120 | +""" |
| 121 | + s = linesearch_backtrack!(M, q, F, p, X, s, decrease, contract η = -X, f0 = f(p)) |
| 122 | +
|
| 123 | +Perform a line search backtrack in-place of `q`. |
| 124 | +For all details and options, see [`linesearch_backtrack`](@ref) |
| 125 | +""" |
| 126 | +function linesearch_backtrack!( |
| 127 | + M::AbstractManifold, |
| 128 | + q, |
| 129 | + f::TF, |
| 130 | + p, |
| 131 | + X::T, |
| 132 | + s, |
| 133 | + decrease, |
| 134 | + contract, |
| 135 | + η::T = (-X), |
| 136 | + f0 = f(M, p); |
| 137 | + retraction_method::AbstractRetractionMethod = default_retraction_method(M, typeof(p)), |
| 138 | + additional_increase_condition = (M, p) -> true, |
| 139 | + additional_decrease_condition = (M, p) -> true, |
| 140 | + stop_when_stepsize_less = 0.0, |
| 141 | + stop_when_stepsize_exceeds = max_stepsize(M, p) / norm(M, p, η), |
| 142 | + stop_increasing_at_step = 100, |
| 143 | + stop_decreasing_at_step = 1000, |
| 144 | + report_messages_in::NamedTuple = (;), |
| 145 | + ) where {TF, T} |
| 146 | + ManifoldsBase.retract_fused!(M, q, p, η, s, retraction_method) |
| 147 | + f_q = f(M, q) |
| 148 | + search_dir_inner = real(inner(M, p, η, X)) |
| 149 | + if search_dir_inner >= 0 |
| 150 | + set_message!(report_messages_in, :non_descent_direction, at = 0, value = search_dir_inner) |
| 151 | + end |
| 152 | + |
| 153 | + i = 0 |
| 154 | + # Ensure that both the original condition and the additional one are fulfilled afterwards |
| 155 | + while f_q < f0 + decrease * s * search_dir_inner || !additional_increase_condition(M, q) |
| 156 | + (stop_increasing_at_step == 0) && break |
| 157 | + i = i + 1 |
| 158 | + s = s / contract |
| 159 | + ManifoldsBase.retract_fused!(M, q, p, η, s, retraction_method) |
| 160 | + f_q = f(M, q) |
| 161 | + if i == stop_increasing_at_step |
| 162 | + set_message!(report_messages_in, :stop_increasing, at = i, bound = stop_increasing_at_step, value = s) |
| 163 | + break |
| 164 | + end |
| 165 | + if s > stop_when_stepsize_exceeds |
| 166 | + set_message!(report_messages_in, :stepsize_exceeds, at = i, bound = stop_when_stepsize_exceeds, value = s) |
| 167 | + break |
| 168 | + end |
| 169 | + end |
| 170 | + i = 0 |
| 171 | + # Ensure that both the original condition and the additional one are fulfilled afterwards |
| 172 | + while (f_q > f0 + decrease * s * search_dir_inner) || |
| 173 | + (!additional_decrease_condition(M, q)) |
| 174 | + i = i + 1 |
| 175 | + s = contract * s |
| 176 | + ManifoldsBase.retract_fused!(M, q, p, η, s, retraction_method) |
| 177 | + f_q = f(M, q) |
| 178 | + if i == stop_decreasing_at_step |
| 179 | + set_message!(report_messages_in, :stop_decreasing, at = i, bound = stop_decreasing_at_step, value = s) |
| 180 | + break |
| 181 | + end |
| 182 | + if s < stop_when_stepsize_less |
| 183 | + set_message!(report_messages_in, :stepsize_less, at = i, bound = stop_when_stepsize_less, value = s) |
| 184 | + break |
| 185 | + end |
| 186 | + end |
| 187 | + return s |
| 188 | +end |
0 commit comments