Skip to content

Commit 7690995

Browse files
committed
solver benchmark: specifying solver-specific options
1 parent ab33f47 commit 7690995

File tree

1 file changed

+31
-0
lines changed
  • tutorials/introduction-to-solverbenchmark

1 file changed

+31
-0
lines changed

tutorials/introduction-to-solverbenchmark/index.jmd

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,3 +221,34 @@ p = profile_solvers(stats, costs, costnames)
221221
Here is a useful tutorial on how to use the benchmark with specific solver:
222222
[Run a benchmark with OptimizationProblems](https://jso.dev/OptimizationProblems.jl/dev/benchmark/)
223223
The tutorial covers how to use the problems from `OptimizationProblems` to run a benchmark for unconstrained optimization.
224+
225+
## Handling `solver_specific` in benchmark runs
226+
227+
`SolverBenchmark` accepts solver-specific options via the keyword argument `solver_specific` on benchmarking functions (for example, when calling the high-level benchmarking helpers such as `bmark_solvers`).
228+
229+
- `solver_specific` is a mapping that associates a solver identifier (typically a `Symbol` or the solver name you use in the benchmark) with a dictionary of options specific to that solver implementation.
230+
- Those options are passed through to the solver when the run is executed and are recorded as part of the run metadata.
231+
- As a result, runs that differ only by solver-specific options are treated as distinct entries in the benchmark results; you can group, filter or compare them in the produced `DataFrame`s.
232+
233+
```julia
234+
using SolverBenchmark, NLPModelsTest, JSOSolvers
235+
236+
solver_specific = Dict(
237+
:IPOPT => Dict("tol" => 1e-8, "max_iter" => 200),
238+
:KNITRO => Dict("maxit" => 500)
239+
)
240+
241+
results = bmark_solvers(solvers, problems; solver_specific = solver_specific)
242+
243+
first(results)
244+
```
245+
246+
Notes and tips:
247+
248+
- If you pass different option sets for the same solver, make sure the keys you use
249+
identify the intended solver variation unambiguously (e.g., distinct Symbols).
250+
- When creating tables or profiles, you can add or extract columns from the per-solver
251+
DataFrames to show which solver-specific options were used for each run.
252+
- If you prefer the solver-specific settings to appear as explicit columns, you can
253+
preprocess the DataFrames (extract the dict entries into columns) before creating
254+
joined tables or profile plots.

0 commit comments

Comments
 (0)