Skip to content

Commit 8ef80b3

Browse files
odowodow
authored
Add another OEM update post (#174)
1 parent 7551fbd commit 8ef80b3

File tree

1 file changed

+161
-0
lines changed

1 file changed

+161
-0
lines changed

‎_posts/2025-07-21-oem_update.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
layout: post
3+
title: "An Open Energy Modeling update"
4+
date: 2025-07-21
5+
categories: [open-energy-modeling]
6+
author: "Oscar Dowson, Ivet Galabova, Joaquim Dias Garcia, Julian Hall, Mark Turner"
7+
---
8+
9+
We're now ten months into our [Open Energy Modeling project](/announcements/open-energy-modeling/2024/09/16/oem/).
10+
If you missed them, you can read our [November update](/open-energy-modeling/2024/11/14/oem_update)
11+
and our [January update](/open-energy-modeling/2025/01/27/oem_update).
12+
13+
## Welcome, Mark
14+
15+
In May we welcomed [Dr. Mark Turner](https://scholar.google.com/citations?user=qyD8_b0AAAAJ&hl=en)
16+
to the project. Mark has a Ph.D. from TU Berlin where he worked on SCIP. Mark's
17+
focus is on improving the performance and reliability of the HiGHS MIP solver.
18+
19+
## HiGHS workshop
20+
21+
At the end of June, we held the [2025 HiGHS workshop](https://workshop25.highs.dev).
22+
We had 18 talks about a range of HiGHS-related topics ([view the full
23+
schedule](https://workshop25.highs.dev/schedule) for details).
24+
25+
There were a number of pertinent talks for the energy community. From the JuMP
26+
and HiGHS teams:
27+
28+
* Julian gave an overview of the [state of HiGHS](https://drive.google.com/file/d/19ECB1-duVubkQTgEhcbtnENtKGQbJdXi/view)
29+
and gave some cautionary advice for [navigating the hype of PDLP](https://drive.google.com/file/d/1j_SYpdIlKvDMYwLgL5xi0n4ULrVj7gT4/view).
30+
* Filippo [introduced his new interior point solver](https://drive.google.com/file/d/1ARfYjhb4b9RRpFro8AH3TEvTDjkHaRNH/view).
31+
Initial benchmarks on PyPSA instances show that it is up to to 10x faster
32+
than the current interior point for large linear programs. It is sometimes
33+
slower, particularly for smaller problems, and there are some challenges with
34+
compiling and distributing it, so it may be some time before it becomes the
35+
default option in HiGHS.
36+
* Oscar [talked about JuMP and HiGHS](https://drive.google.com/file/d/17X4kmRFoNyV9nICkBaxgjGzyTnTrYAk-/view).
37+
Our usage statistics show that half of JuMP users are using HiGHS, and that
38+
there are over 1,000 public GitHub repositories that use HiGHS.jl.
39+
* Mark [talked about the MIP solver](https://drive.google.com/file/d/1JIb3aShAetSGfjt7Qj9ElisAFD0lBmvM/view)
40+
and shared his short-, medium-, and long-term development goals.
41+
* Ivet [talked about the build systems](https://drive.google.com/file/d/13lkc-7stetAaPP-0fhnNJEnAh5x2jA6x/view).
42+
Maintaining the various interfaces to HiGHS is no small task!
43+
44+
There were also two energy-related talks. Dimitris Kousoulidis talked about how
45+
Field Energy [use HiGHS to optimize the operation of a battery](https://drive.google.com/file/d/1oHjLeYPFYRwkyiVW3nCsO6QZOycevM02/view),
46+
and Harley Mackenzie [similarly talked about](https://drive.google.com/file/d/1IomRp_w5KIqdnlL14Wjcn3hEdR8Mg65L/view)
47+
how they use HiGHS to optimize the operation of a battery that is co-located
48+
with variable renewable energy. Both companies use JuMP as the modeling layer.
49+
50+
## JuMP-dev 2025
51+
52+
In November we will host [JuMP-dev 2025](meetings/jumpdev2025/) in Auckland, New
53+
Zealand. If you are interested in energy modeling, please come along! We will
54+
all be there, and we're very keep to hear how any feedback you have about using
55+
JuMP and HiGHS.
56+
57+
## MathOptAnalyzer
58+
59+
A key finding from the first part of our energy modeling project is that the
60+
instances people are solving have a range of numerical features (other than size
61+
) that make them difficult to solve. One example are coefficients with a wide
62+
spread of magnitudes (consider a large thermal generator with a power of 20 GW
63+
and a solar plant in the first hour of dawn generating 0.1 MW). Other common
64+
issues make the problem slow to build. For example, many instances have empty
65+
rows (constraints with no variable terms) and empty columns (variables that do
66+
not appear in a constraint). While these can be trivially presolved out by the
67+
solver, there is a cost (in terms of memory and runtime) to adding them to a
68+
model.
69+
70+
To assist modelers in diagnosing and fixing these issues, we have developed
71+
[MathOptAnalyzer.jl](https://github.com/jump-dev/MathOptAnalyzer.jl).
72+
MathOptAnalyzer is a Julia package that takes in a JuMP model and returns a list
73+
of issues for analysis by the user. Use it as follows:
74+
```
75+
using JuMP, MathOptAnalyzer
76+
model = Model()
77+
# ... build model
78+
data = ModelAnalyzer.analyze(ModelAnalyzer.Numerical.Analyzer(), model)
79+
```
80+
81+
In addition to the numerical analyzer, MathOptAnalyzer can analyze a model for
82+
feasibility, and return information on why the model is infeasible. It can also
83+
assess whether a solution returned by the solver satistifies the primal, dual,
84+
and optimality tolerances. This can be useful for identifying bugs in the
85+
solver, and for validating how the solver's tolerances affect the solution.
86+
87+
For more information, go to [MathOptAnalyzer.jl](https://github.com/jump-dev/MathOptAnalyzer.jl).
88+
89+
## MathOptIIS
90+
91+
A common feature request to both JuMP and HiGHS is for an IIS. If a model is
92+
infeasible, the IIS is a subset of variables and constraints such that the
93+
submodel is still infeasible. (The IIS acronym is not standardized. We have seen
94+
Irreducible Infeasible Subsystem and Irreducibly Inconsistent Set, amongst many
95+
others. We use IIS without attempting to define it.)
96+
97+
The IIS is helpful for two common operations:
98+
99+
1. detecting and explaining data input errors when applying an existing model to
100+
new data
101+
2. detecting and explaining syntax errors (for example, a `+` instead of a `-`)
102+
when developing a new model.
103+
104+
Commercial solvers such as CPLEX, Gurobi, and Xpress all have mature IIS
105+
solvers, and these can be accessed from JuMP via `JuMP.compute_conflict!(model)`.
106+
However, many solvers (for example, HiGHS, Cbc, and Ipopt) do not have an IIS
107+
solver. We have heard anecdotal evidence that many practitioners solve the
108+
majority of their problems with HiGHS, but they still have a single Gurobi
109+
license just so they can access the IIS.
110+
111+
As part of this project, we have developed [MathOptIIS.jl](https://github.com/jump-dev/MathOptIIS.jl).
112+
MathOptIIS is a Julia package that implements an IIS solver in MathOptInterface.
113+
It is not intended to be used directly by users (although you can). Instead, we
114+
will add this as a dependency to existing solver wrappers like HiGHS.jl so that
115+
`JuMP.compute_conflict!` works uniformly for solvers with native IIS support and
116+
those without.
117+
118+
Simultaneously, we have been developing a native IIS solver within HiGHS. When
119+
that is released, we will remove MathOptIIS.jl from HiGHS in favor of the native
120+
IIS solver. Even when HiGHS have a native IIS solver, MathOptIIS will be helpful
121+
for other solvers like Ipopt that do not have an IIS solver (and will likely
122+
never have one).
123+
124+
## A parallel MIP solver
125+
126+
A key deliverable for our project is to add multithreading to the HiGHS MIP
127+
solver. We have been exploring two approaches to this.
128+
129+
First, we have been prototyping adding deterministic parallelism to a single
130+
HiGHS MIP solve. In this approach, we extend HiGHS to conduct multiple parallel
131+
dives through the branch and bound tree. As the parallel dives find new
132+
solutions and prove information about feasibility and the dual bound, they
133+
update a shared global state. The information is shared in a deterministic
134+
manner, so that running the parallel MIP solver with the same number of threads
135+
will always find the same solution. This approach is the one taken by commercial
136+
MIP solvers. It offers a consistent performance boost, since it is strictly
137+
superior to the serial MIP solver. However, it requires a lot of careful
138+
engineering time to implement. The current status of this solver is ``work in
139+
progress'' and we do not have an expected date for its completion.
140+
141+
Second, we have implemented a non-deterministic concurrent MIP solver. In this
142+
approach, we start multiple parallel instances of HiGHS in separate threads.
143+
Each instance uses a different random seed. As the threads find new solutions,
144+
they share this information between themselves, and the algorithm terminates
145+
once any thread has found an optimal solution. This approach exploits the
146+
inherent randomness in HiGHS's presolve and MIP solver. The downside to this
147+
approach is that it is not deterministic; repeated runs of the same model are
148+
not guaranteed to find the same optimal solution. In our preliminary testing,
149+
the speed up that can be expected is problem-dependent, but, with eight threads,
150+
we have seen speedups of 2--5x (although some models are slower). The
151+
non-deterministic concurrent MIP solver is implemented and it will be available
152+
in the next release of HiGHS.
153+
154+
## Other changes
155+
156+
With help from Franz Wesselmann from MathWorks, we continue to find, debug, and
157+
fix many bugs in HiGHS. The most common location for these bugs is in the HiGHS
158+
presolve. We have also added new heuristics, such as the Feasibility Jump, and
159+
Mark has been working to add new cutting planes. We're now at an inflection
160+
point where we hope to see rapid improvements in the HiGHS MIP solver that
161+
should be reflected in the Mittelmann benchmarks by the end of the year.

0 commit comments

Comments
 (0)