Skip to content

Commit 18c558a

Browse files
committed
Merge branch 'master' into kellertuer/properManopt
# Conflicts: # lib/OptimizationManopt/Project.toml
2 parents 1b4864d + 8c0f4c3 commit 18c558a

File tree

53 files changed

+1249
-234
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+1249
-234
lines changed

Project.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
name = "Optimization"
22
uuid = "7f7a1694-90dd-40f0-9382-eb1efda571ba"
3-
version = "4.6.0"
3+
version = "4.7.0"
44

55
[deps]
66
ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"

docs/src/optimization_packages/ipopt.md

Lines changed: 108 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -42,61 +42,130 @@ The algorithm supports:
4242
- Box constraints via `lb` and `ub` in the `OptimizationProblem`
4343
- General nonlinear equality and inequality constraints via `lcons` and `ucons`
4444

45+
### Basic Usage
46+
47+
```julia
48+
using Optimization, OptimizationIpopt
49+
50+
# Create optimizer with default settings
51+
opt = IpoptOptimizer()
52+
53+
# Or configure Ipopt-specific options
54+
opt = IpoptOptimizer(
55+
acceptable_tol = 1e-8,
56+
mu_strategy = "adaptive"
57+
)
58+
59+
# Solve the problem
60+
sol = solve(prob, opt)
61+
```
62+
4563
## Options and Parameters
4664

47-
### Common Options
65+
### Common Interface Options
4866

49-
The following options can be passed as keyword arguments to `solve`:
67+
The following options can be passed as keyword arguments to `solve` and follow the common Optimization.jl interface:
5068

51-
- `maxiters`: Maximum number of iterations (maps to Ipopt's `max_iter`)
52-
- `maxtime`: Maximum wall time in seconds (maps to Ipopt's `max_wall_time`)
69+
- `maxiters`: Maximum number of iterations (overrides Ipopt's `max_iter`)
70+
- `maxtime`: Maximum wall time in seconds (overrides Ipopt's `max_wall_time`)
5371
- `abstol`: Absolute tolerance (not directly used by Ipopt)
54-
- `reltol`: Convergence tolerance (maps to Ipopt's `tol`)
55-
- `verbose`: Control output verbosity
72+
- `reltol`: Convergence tolerance (overrides Ipopt's `tol`)
73+
- `verbose`: Control output verbosity (overrides Ipopt's `print_level`)
5674
- `false` or `0`: No output
5775
- `true` or `5`: Standard output
58-
- Integer values 0-12: Different verbosity levels (maps to `print_level`)
59-
- `hessian_approximation`: Method for Hessian computation
60-
- `"exact"` (default): Use exact Hessian
61-
- `"limited-memory"`: Use L-BFGS approximation
76+
- Integer values 0-12: Different verbosity levels
6277

63-
### Advanced Ipopt Options
78+
### IpoptOptimizer Constructor Options
6479

65-
Any Ipopt option can be passed directly as keyword arguments. The full list of available options is documented in the [Ipopt Options Reference](https://coin-or.github.io/Ipopt/OPTIONS.html). Common options include:
80+
Ipopt-specific options are passed to the `IpoptOptimizer` constructor. The most commonly used options are available as struct fields:
6681

67-
#### Convergence Options
68-
- `tol`: Desired convergence tolerance (relative)
69-
- `dual_inf_tol`: Dual infeasibility tolerance
70-
- `constr_viol_tol`: Constraint violation tolerance
71-
- `compl_inf_tol`: Complementarity tolerance
82+
#### Termination Options
83+
- `acceptable_tol::Float64 = 1e-6`: Acceptable convergence tolerance (relative)
84+
- `acceptable_iter::Int = 15`: Number of acceptable iterations before termination
85+
- `dual_inf_tol::Float64 = 1.0`: Desired threshold for dual infeasibility
86+
- `constr_viol_tol::Float64 = 1e-4`: Desired threshold for constraint violation
87+
- `compl_inf_tol::Float64 = 1e-4`: Desired threshold for complementarity conditions
7288

73-
#### Algorithm Options
74-
- `linear_solver`: Linear solver to use
89+
#### Linear Solver Options
90+
- `linear_solver::String = "mumps"`: Linear solver to use
7591
- Default: "mumps" (included with Ipopt)
7692
- HSL solvers: "ma27", "ma57", "ma86", "ma97" (require [separate installation](https://github.com/jump-dev/Ipopt.jl?tab=readme-ov-file#linear-solvers))
7793
- Others: "pardiso", "spral" (require [separate installation](https://github.com/jump-dev/Ipopt.jl?tab=readme-ov-file#linear-solvers))
78-
- `nlp_scaling_method`: Scaling method ("gradient-based", "none", "equilibration-based")
79-
- `limited_memory_max_history`: History size for L-BFGS (when using `hessian_approximation="limited-memory"`)
80-
- `mu_strategy`: Update strategy for barrier parameter ("monotone", "adaptive")
94+
- `linear_system_scaling::String = "none"`: Method for scaling linear system. Use "mc19" for HSL solvers.
95+
96+
#### NLP Scaling Options
97+
- `nlp_scaling_method::String = "gradient-based"`: Scaling method for NLP
98+
- Options: "none", "user-scaling", "gradient-based", "equilibration-based"
99+
- `nlp_scaling_max_gradient::Float64 = 100.0`: Maximum gradient after scaling
100+
101+
#### Barrier Parameter Options
102+
- `mu_strategy::String = "monotone"`: Update strategy for barrier parameter ("monotone", "adaptive")
103+
- `mu_init::Float64 = 0.1`: Initial value for barrier parameter
104+
- `mu_oracle::String = "quality-function"`: Oracle for adaptive mu strategy
105+
106+
#### Hessian Options
107+
- `hessian_approximation::String = "exact"`: How to approximate the Hessian
108+
- `"exact"`: Use exact Hessian
109+
- `"limited-memory"`: Use L-BFGS approximation
110+
- `limited_memory_max_history::Int = 6`: History size for L-BFGS
111+
- `limited_memory_update_type::String = "bfgs"`: Quasi-Newton update formula ("bfgs", "sr1")
81112

82113
#### Line Search Options
83-
- `line_search_method`: Line search method ("filter", "penalty")
84-
- `alpha_for_y`: Step size for constraint multipliers
85-
- `recalc_y`: Control when multipliers are recalculated
114+
- `line_search_method::String = "filter"`: Line search method ("filter", "penalty")
115+
- `accept_every_trial_step::String = "no"`: Accept every trial step (disables line search)
86116

87117
#### Output Options
88-
- `print_timing_statistics`: Print detailed timing information ("yes"/"no")
89-
- `print_info_string`: Print user-defined info string ("yes"/"no")
118+
- `print_timing_statistics::String = "no"`: Print detailed timing information
119+
- `print_info_string::String = "no"`: Print algorithm info string
120+
121+
#### Warm Start Options
122+
- `warm_start_init_point::String = "no"`: Use warm start from previous solution
90123

91-
Example with advanced options:
124+
#### Restoration Phase Options
125+
- `expect_infeasible_problem::String = "no"`: Enable if problem is expected to be infeasible
126+
127+
### Additional Options Dictionary
128+
129+
For Ipopt options not available as struct fields, use the `additional_options` dictionary:
92130

93131
```julia
94-
sol = solve(prob, IpoptOptimizer();
95-
maxiters = 1000,
96-
tol = 1e-8,
132+
opt = IpoptOptimizer(
97133
linear_solver = "ma57",
98-
mu_strategy = "adaptive",
99-
print_timing_statistics = "yes"
134+
additional_options = Dict(
135+
"derivative_test" => "first-order",
136+
"derivative_test_tol" => 1e-4,
137+
"fixed_variable_treatment" => "make_parameter",
138+
"alpha_for_y" => "primal"
139+
)
140+
)
141+
```
142+
143+
The full list of available options is documented in the [Ipopt Options Reference](https://coin-or.github.io/Ipopt/OPTIONS.html).
144+
145+
### Option Priority
146+
147+
Options follow this priority order (highest to lowest):
148+
1. Common interface arguments passed to `solve` (e.g., `reltol`, `maxiters`)
149+
2. Options in `additional_options` dictionary
150+
3. Struct field values in `IpoptOptimizer`
151+
152+
Example with multiple option sources:
153+
154+
```julia
155+
opt = IpoptOptimizer(
156+
acceptable_tol = 1e-6, # Struct field
157+
mu_strategy = "adaptive", # Struct field
158+
linear_solver = "ma57", # Struct field (needs HSL)
159+
print_timing_statistics = "yes", # Struct field
160+
additional_options = Dict(
161+
"alpha_for_y" => "primal", # Not a struct field
162+
"max_iter" => 500 # Will be overridden by maxiters below
163+
)
164+
)
165+
166+
sol = solve(prob, opt;
167+
maxiters = 1000, # Overrides max_iter in additional_options
168+
reltol = 1e-8 # Sets Ipopt's tol
100169
)
101170
```
102171

@@ -189,9 +258,9 @@ optfunc = OptimizationFunction(rosenbrock_nd, AutoZygote())
189258
prob = OptimizationProblem(optfunc, x0, p)
190259
191260
# Use L-BFGS approximation for Hessian
192-
sol = solve(prob, IpoptOptimizer();
261+
sol = solve(prob, IpoptOptimizer(
193262
hessian_approximation = "limited-memory",
194-
limited_memory_max_history = 10,
263+
limited_memory_max_history = 10);
195264
maxiters = 1000)
196265
```
197266

@@ -235,8 +304,8 @@ prob = OptimizationProblem(optfunc, w0;
235304
ucons = [0.0, Inf])
236305
237306
sol = solve(prob, IpoptOptimizer();
238-
tol = 1e-8,
239-
print_level = 5)
307+
reltol = 1e-8,
308+
verbose = 5)
240309
241310
println("Optimal weights: ", sol.u)
242311
println("Expected return: ", dot(μ, sol.u))
@@ -249,13 +318,13 @@ println("Portfolio variance: ", sol.objective)
249318

250319
2. **Initial Points**: Provide good initial guesses when possible. Ipopt is a local optimizer and the solution quality depends on the starting point.
251320

252-
3. **Hessian Approximation**: For large problems or when Hessian computation is expensive, use `hessian_approximation = "limited-memory"`.
321+
3. **Hessian Approximation**: For large problems or when Hessian computation is expensive, use `hessian_approximation = "limited-memory"` in the `IpoptOptimizer` constructor.
253322

254323
4. **Linear Solver Selection**: The choice of linear solver can significantly impact performance. For large problems, consider using HSL solvers (ma27, ma57, ma86, ma97). Note that HSL solvers require [separate installation](https://github.com/jump-dev/Ipopt.jl?tab=readme-ov-file#linear-solvers) - see the Ipopt.jl documentation for setup instructions. The default MUMPS solver works well for small to medium problems.
255324

256325
5. **Constraint Formulation**: Ipopt handles equality constraints well. When possible, formulate constraints as equalities rather than pairs of inequalities.
257326

258-
6. **Warm Starting**: When solving a sequence of similar problems, use the solution from the previous problem as the initial point for the next.
327+
6. **Warm Starting**: When solving a sequence of similar problems, use the solution from the previous problem as the initial point for the next. You can enable warm starting with `IpoptOptimizer(warm_start_init_point = "yes")`.
259328

260329
## References
261330

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# OptimizationODE.jl
2+
3+
**OptimizationODE.jl** provides ODE-based optimization methods as a solver plugin for [SciML's Optimization.jl](https://github.com/SciML/Optimization.jl). It wraps various ODE solvers to perform gradient-based optimization using continuous-time dynamics.
4+
5+
## Installation
6+
7+
```julia
8+
using Pkg
9+
Pkg.add("OptimizationODE")
10+
```
11+
12+
## Usage
13+
14+
```julia
15+
using OptimizationODE, Optimization, ADTypes, SciMLBase
16+
17+
function f(x, p)
18+
return sum(abs2, x)
19+
end
20+
21+
function g!(g, x, p)
22+
@. g = 2 * x
23+
end
24+
25+
x0 = [2.0, -3.0]
26+
p = []
27+
28+
f_manual = OptimizationFunction(f, SciMLBase.NoAD(); grad = g!)
29+
prob_manual = OptimizationProblem(f_manual, x0)
30+
31+
opt = ODEGradientDescent(dt=0.01)
32+
sol = solve(prob_manual, opt; maxiters=50_000)
33+
34+
@show sol.u
35+
@show sol.objective
36+
```
37+
38+
## Local Gradient-based Optimizers
39+
40+
All provided optimizers are **gradient-based local optimizers** that solve optimization problems by integrating gradient-based ODEs to convergence:
41+
42+
* `ODEGradientDescent(dt=...)` — performs basic gradient descent using the explicit Euler method. This is a simple and efficient method suitable for small-scale or well-conditioned problems.
43+
44+
* `RKChebyshevDescent()` — uses the ROCK2 solver, a stabilized explicit Runge-Kutta method suitable for stiff problems. It allows larger step sizes while maintaining stability.
45+
46+
* `RKAccelerated()` — leverages the Tsit5 method, a 5th-order Runge-Kutta solver that achieves faster convergence for smooth problems by improving integration accuracy.
47+
48+
* `HighOrderDescent()` — applies Vern7, a high-order (7th-order) explicit Runge-Kutta method for even more accurate integration. This can be beneficial for problems requiring high precision.
49+
50+
You can also define a custom optimizer using the generic `ODEOptimizer(solver; dt=nothing)` constructor by supplying any ODE solver supported by [OrdinaryDiffEq.jl](https://docs.sciml.ai/DiffEqDocs/stable/solvers/ode_solve/).
51+
52+
## DAE-based Optimizers
53+
54+
!!! warn
55+
DAE-based optimizers are still experimental and a research project. Use with caution.
56+
57+
In addition to ODE-based optimizers, OptimizationODE.jl provides optimizers for differential-algebraic equation (DAE) constrained problems:
58+
59+
* `DAEMassMatrix()` — uses the Rodas5P solver (from OrdinaryDiffEq.jl) for DAE problems with a mass matrix formulation.
60+
61+
* `DAEOptimizer(IDA())` — uses the IDA solver (from Sundials.jl) for DAE problems with index variable support (requires `using Sundials`)
62+
63+
You can also define a custom optimizer using the generic `ODEOptimizer(solver)` or `DAEOptimizer(solver)` constructor by supplying any ODE or DAE solver supported by [OrdinaryDiffEq.jl](https://docs.sciml.ai/DiffEqDocs/stable/solvers/ode_solve/) or [Sundials.jl](https://github.com/SciML/Sundials.jl).
64+
65+
## Interface Details
66+
67+
All optimizers require gradient information (either via automatic differentiation or manually provided `grad!`). The optimization is performed by integrating the ODE defined by the negative gradient until a steady state is reached.

lib/OptimizationBBO/Project.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name = "OptimizationBBO"
22
uuid = "3e6eede4-6085-4f62-9a71-46d9bc1eb92b"
33
authors = ["Vaibhav Dixit <[email protected]> and contributors"]
4-
version = "0.4.1"
4+
version = "0.4.2"
55

66
[deps]
77
BlackBoxOptim = "a134a8b2-14d6-55f6-9291-3336d3ab0209"

lib/OptimizationBBO/src/OptimizationBBO.jl

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,12 @@ abstract type BBO end
99

1010
SciMLBase.requiresbounds(::BBO) = true
1111
SciMLBase.allowsbounds(::BBO) = true
12-
SciMLBase.supports_opt_cache_interface(opt::BBO) = true
12+
@static if isdefined(SciMLBase, :supports_opt_cache_interface)
13+
SciMLBase.supports_opt_cache_interface(opt::BBO) = true
14+
end
15+
@static if isdefined(OptimizationBase, :supports_opt_cache_interface)
16+
OptimizationBase.supports_opt_cache_interface(opt::BBO) = true
17+
end
1318

1419
for j in string.(BlackBoxOptim.SingleObjectiveMethodNames)
1520
eval(Meta.parse("Base.@kwdef struct BBO_" * j * " <: BBO method=:" * j * " end"))

lib/OptimizationBase/src/OptimizationBase.jl

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,10 @@ include("cache.jl")
2727
include("OptimizationDIExt.jl")
2828
include("OptimizationDISparseExt.jl")
2929
include("function.jl")
30+
include("solve.jl")
3031

31-
export solve, OptimizationCache, DEFAULT_CALLBACK, DEFAULT_DATA
32+
export solve, OptimizationCache, DEFAULT_CALLBACK, DEFAULT_DATA,
33+
IncompatibleOptimizerError, OptimizerMissingError, _check_opt_alg,
34+
supports_opt_cache_interface
3235

3336
end

lib/OptimizationBase/src/solve.jl

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# This file contains the top level solve interface functionality moved from SciMLBase.jl
2+
# These functions provide the core optimization solving interface
3+
4+
struct IncompatibleOptimizerError <: Exception
5+
err::String
6+
end
7+
8+
function Base.showerror(io::IO, e::IncompatibleOptimizerError)
9+
print(io, e.err)
10+
end
11+
12+
const OPTIMIZER_MISSING_ERROR_MESSAGE = """
13+
Optimization algorithm not found. Either the chosen algorithm is not a valid solver
14+
choice for the `OptimizationProblem`, or the Optimization solver library is not loaded.
15+
Make sure that you have loaded an appropriate Optimization.jl solver library, for example,
16+
`solve(prob,Optim.BFGS())` requires `using OptimizationOptimJL` and
17+
`solve(prob,Adam())` requires `using OptimizationOptimisers`.
18+
19+
For more information, see the Optimization.jl documentation: <https://docs.sciml.ai/Optimization/stable/>.
20+
"""
21+
22+
struct OptimizerMissingError <: Exception
23+
alg::Any
24+
end
25+
26+
function Base.showerror(io::IO, e::OptimizerMissingError)
27+
println(io, OPTIMIZER_MISSING_ERROR_MESSAGE)
28+
print(io, "Chosen Optimizer: ")
29+
print(e.alg)
30+
end
31+
32+
# Algorithm compatibility checking function
33+
function _check_opt_alg(prob::SciMLBase.OptimizationProblem, alg; kwargs...)
34+
!SciMLBase.allowsbounds(alg) && (!isnothing(prob.lb) || !isnothing(prob.ub)) &&
35+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) does not support box constraints. Either remove the `lb` or `ub` bounds passed to `OptimizationProblem` or use a different algorithm."))
36+
SciMLBase.requiresbounds(alg) && isnothing(prob.lb) &&
37+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires box constraints. Either pass `lb` and `ub` bounds to `OptimizationProblem` or use a different algorithm."))
38+
!SciMLBase.allowsconstraints(alg) && !isnothing(prob.f.cons) &&
39+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) does not support constraints. Either remove the `cons` function passed to `OptimizationFunction` or use a different algorithm."))
40+
SciMLBase.requiresconstraints(alg) && isnothing(prob.f.cons) &&
41+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires constraints, pass them with the `cons` kwarg in `OptimizationFunction`."))
42+
# Check that if constraints are present and the algorithm supports constraints, both lcons and ucons are provided
43+
SciMLBase.allowsconstraints(alg) && !isnothing(prob.f.cons) &&
44+
(isnothing(prob.lcons) || isnothing(prob.ucons)) &&
45+
throw(ArgumentError("Constrained optimization problem requires both `lcons` and `ucons` to be provided to OptimizationProblem. " *
46+
"Example: OptimizationProblem(optf, u0, p; lcons=[-Inf], ucons=[0.0])"))
47+
!SciMLBase.allowscallback(alg) && haskey(kwargs, :callback) &&
48+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) does not support callbacks, remove the `callback` keyword argument from the `solve` call."))
49+
SciMLBase.requiresgradient(alg) &&
50+
!(prob.f isa SciMLBase.AbstractOptimizationFunction) &&
51+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires gradients, hence use `OptimizationFunction` to generate them with an automatic differentiation backend e.g. `OptimizationFunction(f, AutoForwardDiff())` or pass it in with `grad` kwarg."))
52+
SciMLBase.requireshessian(alg) &&
53+
!(prob.f isa SciMLBase.AbstractOptimizationFunction) &&
54+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires hessians, hence use `OptimizationFunction` to generate them with an automatic differentiation backend e.g. `OptimizationFunction(f, AutoFiniteDiff(); kwargs...)` or pass them in with `hess` kwarg."))
55+
SciMLBase.requiresconsjac(alg) &&
56+
!(prob.f isa SciMLBase.AbstractOptimizationFunction) &&
57+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires constraint jacobians, hence use `OptimizationFunction` to generate them with an automatic differentiation backend e.g. `OptimizationFunction(f, AutoFiniteDiff(); kwargs...)` or pass them in with `cons` kwarg."))
58+
SciMLBase.requiresconshess(alg) &&
59+
!(prob.f isa SciMLBase.AbstractOptimizationFunction) &&
60+
throw(IncompatibleOptimizerError("The algorithm $(typeof(alg)) requires constraint hessians, hence use `OptimizationFunction` to generate them with an automatic differentiation backend e.g. `OptimizationFunction(f, AutoFiniteDiff(), AutoFiniteDiff(hess=true); kwargs...)` or pass them in with `cons` kwarg."))
61+
return
62+
end
63+
64+
# Base solver dispatch functions (these will be extended by specific solver packages)
65+
supports_opt_cache_interface(alg) = false
66+
67+
function __solve(cache::SciMLBase.AbstractOptimizationCache)::SciMLBase.AbstractOptimizationSolution
68+
throw(OptimizerMissingError(cache.opt))
69+
end
70+
71+
function __init(prob::SciMLBase.OptimizationProblem, alg, args...;
72+
kwargs...)::SciMLBase.AbstractOptimizationCache
73+
throw(OptimizerMissingError(alg))
74+
end
75+
76+
function __solve(prob::SciMLBase.OptimizationProblem, alg, args...; kwargs...)
77+
throw(OptimizerMissingError(alg))
78+
end

lib/OptimizationBase/test/runtests.jl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ using Test
55
include("adtests.jl")
66
include("cvxtest.jl")
77
include("matrixvalued.jl")
8+
include("solver_missing_error_messages.jl")
89
end

0 commit comments

Comments
 (0)