Skip to content

Commit d991bea

Browse files
adds inequality support
1 parent 0f0868e commit d991bea

File tree

5 files changed

+409
-75
lines changed

5 files changed

+409
-75
lines changed

docs/Project.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
33
DifferentialEquations = "0c46a032-eb83-5123-abaf-570d42b7fbaa"
44
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
55
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
6-
Optimization = "7f7a1694-90dd-40f0-9382-eb1efda571ba"
76
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
87
ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
98
NonlinearSolve = "8913a72c-1f9b-4ce2-8d82-65094dcecaec"
109
Optim = "429524aa-4258-5aef-a3af-852621145aeb"
10+
Optimization = "7f7a1694-90dd-40f0-9382-eb1efda571ba"
11+
OptimizationOptimJL = "36348300-93cb-4f02-beb5-3c3902f8871e"
1112
OrdinaryDiffEq = "1dea7af3-3e70-54e6-95c3-0bf5283fa5ed"
1213
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
1314
Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7"

docs/src/tutorials/optimization.md

Lines changed: 70 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,85 @@
11
# Modeling Optimization Problems
22

3-
```julia
4-
using ModelingToolkit, Optimization, OptimizationOptimJL
3+
## Rosenbrock Function in 2D
4+
Let's solve the classical _Rosenbrock function_ in two dimensions.
55

6-
@variables x y
7-
@parameters a b
6+
First, we need to make some imports.
7+
```@example rosenbrock_2d
8+
using ModelingToolkit, Optimization, OptimizationOptimJL
9+
```
10+
Now we can define our optimization problem.
11+
```@example rosenbrock_2d
12+
@variables begin
13+
x, [bounds = (-2.0, 2.0)]
14+
y, [bounds = (-1.0, 3.0)]
15+
end
16+
@parameters a = 1 b = 1
817
loss = (a - x)^2 + b * (y - x^2)^2
9-
@named sys = OptimizationSystem(loss,[x,y],[a,b])
18+
@named sys = OptimizationSystem(loss, [x, y], [a, b])
19+
```
1020

21+
A visualization of the objective function is depicted below.
22+
```@eval
23+
using Plots
24+
x = -2:0.01:2
25+
y = -1:0.01:3
26+
contour(x, y, (x,y) -> (1 - x)^2 + 100 * (y - x^2)^2, fill=true, color=:viridis, ratio=:equal, xlims=(-2, 2))
27+
```
28+
29+
### Explanation
30+
Every optimization problem consists of a set of _optimization variables_. In this case we create two variables. Additionally we assign _box constraints_ for each of them. In the next step we create two parameters for the problem with `@parameters`. While it is not needed to do this it makes it easier to `remake` the problem later with different values for the parameters. The _objective function_ is specified as well and finally everything is used to construct an `OptimizationSystem`.
31+
32+
## Building and Solving the Optimization Problem
33+
Next the actual `OptimizationProblem` can be created. At this stage an initial guess `u0` for the optimization variables needs to be provided via map, using the symbols from before. Concrete values for the parameters of the system can also be provided or changed. However, if the parameters have default values assigned they are used automatically.
34+
```@example rosenbrock_2d
1135
u0 = [
12-
x=>1.0
13-
y=>2.0
36+
x => 1.0
37+
y => 2.0
1438
]
1539
p = [
16-
a => 6.0
17-
b => 7.0
40+
a => 1.0
41+
b => 100.0
42+
]
43+
44+
prob = OptimizationProblem(sys, u0, p, grad=true, hess=true)
45+
solve(prob, GradientDescent())
46+
```
47+
48+
## Rosenbrock Function with Constraints
49+
```@example rosenbrock_2d_cstr
50+
using ModelingToolkit, Optimization, OptimizationOptimJL
51+
52+
@variables begin
53+
x, [bounds = (-2.0, 2.0)]
54+
y, [bounds = (-1.0, 3.0)]
55+
end
56+
@parameters a = 1 b = 100
57+
loss = (a - x)^2 + b * (y - x^2)^2
58+
cons = [
59+
x^2 + y^2 ≲ 1,
1860
]
61+
@named sys = OptimizationSystem(loss, [x, y], [a, b], constraints = cons)
62+
u0 = [
63+
x => 1.0
64+
y => 2.0
65+
]
66+
prob = OptimizationProblem(sys, u0, grad=true, hess=true)
67+
solve(prob, IPNewton())
68+
```
1969

20-
prob = OptimizationProblem(sys,u0,p,grad=true,hess=true)
21-
solve(prob,Newton())
70+
A visualization of the objective function and the inequality constraint is depicted below.
71+
```@eval
72+
using Plots
73+
x = -2:0.01:2
74+
y = -1:0.01:3
75+
contour(x, y, (x,y) -> (1 - x)^2 + 100 * (y - x^2)^2, fill=true, color=:viridis, ratio=:equal, xlims=(-2, 2))
76+
contour!(x, y, (x, y) -> x^2 + y^2, levels=[1], color=:lightblue, line=4)
2277
```
2378

79+
### Explanation
80+
Equality and inequality constraints can be added to the `OptimizationSystem`. An equality constraint can be specified via and `Equation`, e.g., `x^2 + y^2 ~ 1`. While inequality constraints via an `Inequality`, e.g., `x^2 + y^2 ≲ 1`. The syntax is here `\lesssim` and `\gtrsim`.
81+
82+
## Nested Systems
2483
Needs more text but it's super cool and auto-parallelizes and sparsifies too.
2584
Plus you can hierarchically nest systems to have it generate huge
2685
optimization problems.

src/ModelingToolkit.jl

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,15 @@ import Graphs: SimpleDiGraph, add_edge!, incidence_matrix
6666
for fun in [:toexpr]
6767
@eval begin
6868
function $fun(eq::Equation; kw...)
69-
Expr(:(=), $fun(eq.lhs; kw...), $fun(eq.rhs; kw...))
69+
Expr(:call, :(==), $fun(eq.lhs; kw...), $fun(eq.rhs; kw...))
70+
end
71+
72+
function $fun(ineq::Inequality; kw...)
73+
if ineq.relational_op == Symbolics.leq
74+
Expr(:call, :(<=), $fun(ineq.lhs; kw...), $fun(ineq.rhs; kw...))
75+
else
76+
Expr(:call, :(>=), $fun(ineq.lhs; kw...), $fun(ineq.rhs; kw...))
77+
end
7078
end
7179

7280
$fun(eqs::AbstractArray; kw...) = map(eq -> $fun(eq; kw...), eqs)
@@ -85,6 +93,7 @@ abstract type AbstractTimeDependentSystem <: AbstractSystem end
8593
abstract type AbstractTimeIndependentSystem <: AbstractSystem end
8694
abstract type AbstractODESystem <: AbstractTimeDependentSystem end
8795
abstract type AbstractMultivariateSystem <: AbstractSystem end
96+
abstract type AbstractOptimizationSystem <: AbstractTimeIndependentSystem end
8897

8998
"""
9099
$(TYPEDSIGNATURES)
@@ -137,6 +146,7 @@ include("systems/jumps/jumpsystem.jl")
137146
include("systems/nonlinear/nonlinearsystem.jl")
138147
include("systems/nonlinear/modelingtoolkitize.jl")
139148

149+
include("systems/optimization/constraints_system.jl")
140150
include("systems/optimization/optimizationsystem.jl")
141151

142152
include("systems/pde/pdesystem.jl")
@@ -174,7 +184,7 @@ export OptimizationProblem, OptimizationProblemExpr, constraints
174184
export AutoModelingToolkit
175185
export SteadyStateProblem, SteadyStateProblemExpr
176186
export JumpProblem, DiscreteProblem
177-
export NonlinearSystem, OptimizationSystem
187+
export NonlinearSystem, OptimizationSystem, ConstraintsSystem
178188
export alias_elimination, flatten
179189
export connect, @connector, Connection, Flow, Stream, instream
180190
export isinput, isoutput, getbounds, hasbounds, isdisturbance, istunable, getdist, hasdist,
@@ -187,7 +197,7 @@ export Equation, ConstrainedEquation
187197
export Term, Sym
188198
export SymScope, LocalScope, ParentScope, GlobalScope
189199
export independent_variables, independent_variable, states, parameters, equations, controls,
190-
observed, structure, full_equations
200+
observed, structure, full_equations, objective
191201
export structural_simplify, expand_connections, linearize, linearization_function
192202
export DiscreteSystem, DiscreteProblem
193203

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
"""
2+
$(TYPEDEF)
3+
4+
A constraint system of equations.
5+
6+
# Fields
7+
$(FIELDS)
8+
9+
# Examples
10+
11+
```julia
12+
@variables x y z
13+
@parameters a b c
14+
15+
cstr = [0 ~ a*(y-x),
16+
0 ~ x*(b-z)-y,
17+
0 ~ x*y - c*z
18+
x^2 + y^2 ≲ 1]
19+
@named ns = ConstraintsSystem(cstr, [x,y,z],[a,b,c])
20+
```
21+
"""
22+
struct ConstraintsSystem <: AbstractTimeIndependentSystem
23+
"""
24+
tag: a tag for the system. If two system have the same tag, then they are
25+
structurally identical.
26+
"""
27+
tag::UInt
28+
"""Vector of equations defining the system."""
29+
constraints::Vector{Union{Equation, Inequality}}
30+
"""Unknown variables."""
31+
states::Vector
32+
"""Parameters."""
33+
ps::Vector
34+
"""Array variables."""
35+
var_to_name::Any
36+
"""Observed variables."""
37+
observed::Vector{Equation}
38+
"""
39+
Jacobian matrix. Note: this field will not be defined until
40+
[`calculate_jacobian`](@ref) is called on the system.
41+
"""
42+
jac::RefValue{Any}
43+
"""
44+
Name: the name of the system. These are required to have unique names.
45+
"""
46+
name::Symbol
47+
"""
48+
systems: The internal systems
49+
"""
50+
systems::Vector{ConstraintsSystem}
51+
"""
52+
defaults: The default values to use when initial conditions and/or
53+
parameters are not supplied in `ODEProblem`.
54+
"""
55+
defaults::Dict
56+
"""
57+
type: type of the system
58+
"""
59+
connector_type::Any
60+
"""
61+
metadata: metadata for the system, to be used by downstream packages.
62+
"""
63+
metadata::Any
64+
"""
65+
tearing_state: cache for intermediate tearing state
66+
"""
67+
tearing_state::Any
68+
"""
69+
substitutions: substitutions generated by tearing.
70+
"""
71+
substitutions::Any
72+
73+
function ConstraintsSystem(tag, constraints, states, ps, var_to_name, observed, jac,
74+
name,
75+
systems,
76+
defaults, connector_type, metadata = nothing,
77+
tearing_state = nothing, substitutions = nothing;
78+
checks::Union{Bool, Int} = true)
79+
if checks == true || (checks & CheckUnits) > 0
80+
all_dimensionless([states; ps]) || check_units(constraints)
81+
end
82+
new(tag, constraints, states, ps, var_to_name, observed, jac, name, systems,
83+
defaults,
84+
connector_type, metadata, tearing_state, substitutions)
85+
end
86+
end
87+
88+
equations(sys::ConstraintsSystem) = constraints(sys) # needed for Base.show
89+
90+
function ConstraintsSystem(constraints, states, ps;
91+
observed = [],
92+
name = nothing,
93+
default_u0 = Dict(),
94+
default_p = Dict(),
95+
defaults = _merge(Dict(default_u0), Dict(default_p)),
96+
systems = ConstraintsSystem[],
97+
connector_type = nothing,
98+
continuous_events = nothing, # this argument is only required for ODESystems, but is added here for the constructor to accept it without error
99+
discrete_events = nothing, # this argument is only required for ODESystems, but is added here for the constructor to accept it without error
100+
checks = true,
101+
metadata = nothing)
102+
continuous_events === nothing || isempty(continuous_events) ||
103+
throw(ArgumentError("ConstraintsSystem does not accept `continuous_events`, you provided $continuous_events"))
104+
discrete_events === nothing || isempty(discrete_events) ||
105+
throw(ArgumentError("ConstraintsSystem does not accept `discrete_events`, you provided $discrete_events"))
106+
107+
name === nothing &&
108+
throw(ArgumentError("The `name` keyword must be provided. Please consider using the `@named` macro"))
109+
110+
cstr = value.(Symbolics.canonical_form.(scalarize(constraints)))
111+
states′ = value.(scalarize(states))
112+
ps′ = value.(scalarize(ps))
113+
114+
if !(isempty(default_u0) && isempty(default_p))
115+
Base.depwarn("`default_u0` and `default_p` are deprecated. Use `defaults` instead.",
116+
:ConstraintsSystem, force = true)
117+
end
118+
sysnames = nameof.(systems)
119+
if length(unique(sysnames)) != length(sysnames)
120+
throw(ArgumentError("System names must be unique."))
121+
end
122+
123+
jac = RefValue{Any}(EMPTY_JAC)
124+
defaults = todict(defaults)
125+
defaults = Dict(value(k) => value(v) for (k, v) in pairs(defaults))
126+
127+
var_to_name = Dict()
128+
process_variables!(var_to_name, defaults, states′)
129+
process_variables!(var_to_name, defaults, ps′)
130+
isempty(observed) || collect_var_to_name!(var_to_name, (eq.lhs for eq in observed))
131+
132+
ConstraintsSystem(Threads.atomic_add!(SYSTEM_COUNT, UInt(1)),
133+
cstr, states, ps, var_to_name, observed, jac, name, systems,
134+
defaults,
135+
connector_type, metadata, checks = checks)
136+
end
137+
138+
function calculate_jacobian(sys::ConstraintsSystem; sparse = false, simplify = false)
139+
cache = get_jac(sys)[]
140+
if cache isa Tuple && cache[2] == (sparse, simplify)
141+
return cache[1]
142+
end
143+
144+
lhss = generate_canonical_form_lhss(sys)
145+
vals = [dv for dv in states(sys)]
146+
if sparse
147+
jac = sparsejacobian(lhss, vals, simplify = simplify)
148+
else
149+
jac = jacobian(lhss, vals, simplify = simplify)
150+
end
151+
get_jac(sys)[] = jac, (sparse, simplify)
152+
return jac
153+
end
154+
155+
function generate_jacobian(sys::ConstraintsSystem, vs = states(sys), ps = parameters(sys);
156+
sparse = false, simplify = false, kwargs...)
157+
jac = calculate_jacobian(sys, sparse = sparse, simplify = simplify)
158+
return build_function(jac, vs, ps; kwargs...)
159+
end
160+
161+
function calculate_hessian(sys::ConstraintsSystem; sparse = false, simplify = false)
162+
lhss = generate_canonical_form_lhss(sys)
163+
vals = [dv for dv in states(sys)]
164+
if sparse
165+
hess = [sparsehessian(lhs, vals, simplify = simplify) for lhs in lhss]
166+
else
167+
hess = [hessian(lhs, vals, simplify = simplify) for lhs in lhss]
168+
end
169+
return hess
170+
end
171+
172+
function generate_hessian(sys::ConstraintsSystem, vs = states(sys), ps = parameters(sys);
173+
sparse = false, simplify = false, kwargs...)
174+
hess = calculate_hessian(sys, sparse = sparse, simplify = simplify)
175+
return build_function(hess, vs, ps; kwargs...)
176+
end
177+
178+
function generate_function(sys::ConstraintsSystem, dvs = states(sys), ps = parameters(sys);
179+
kwargs...)
180+
lhss = generate_canonical_form_lhss(sys)
181+
pre, sol_states = get_substitutions_and_solved_states(sys)
182+
183+
func = build_function(lhss, value.(dvs), value.(ps); postprocess_fbody = pre,
184+
states = sol_states, kwargs...)
185+
186+
cstr = constraints(sys)
187+
lcons = fill(-Inf, length(cstr))
188+
ucons = zeros(length(cstr))
189+
lcons[findall(Base.Fix2(isa, Equation), cstr)] .= 0.0
190+
191+
return func, lcons, ucons
192+
end
193+
194+
function jacobian_sparsity(sys::ConstraintsSystem)
195+
lhss = generate_canonical_form_lhss(sys)
196+
jacobian_sparsity(lhss, states(sys))
197+
end
198+
199+
function hessian_sparsity(sys::ConstraintsSystem)
200+
lhss = generate_canonical_form_lhss(sys)
201+
[hessian_sparsity(eq, states(sys)) for eq in lhss]
202+
end
203+
204+
"""
205+
Convert the system of equalities and inequalities into a canonical form:
206+
h(x) = 0
207+
g(x) <= 0
208+
"""
209+
function generate_canonical_form_lhss(sys)
210+
lhss = [Symbolics.canonical_form(eq).lhs for eq in constraints(sys)]
211+
end

0 commit comments

Comments
 (0)