Skip to content

Commit efc329a

Browse files
abelsiqueiraJSOTutorials bot
andauthored
🤖 Tutorials deployment from 8b527f4a0fbf82ffe8ca2b677efa2b327fe19346 (#204)
Co-authored-by: JSOTutorials bot <[email protected]>
1 parent 3046251 commit efc329a

File tree

5 files changed

+41
-5
lines changed

5 files changed

+41
-5
lines changed
10.1 KB
Loading
9.89 KB
Loading
10.1 KB
Loading
1.38 KB
Loading

tutorials/introduction-to-benchmarkprofiles/index.md

Lines changed: 41 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,17 +5,31 @@
55
\preamble{Abel Soares Siqueira and Dominique Orban}
66

77

8-
![JSON 0.21.3](https://img.shields.io/badge/JSON-0.21.3-000?style=flat-square&labelColor=999)
9-
[![BenchmarkProfiles 0.4.3](https://img.shields.io/badge/BenchmarkProfiles-0.4.3-006400?style=flat-square&labelColor=389826)](https://juliasmoothoptimizers.github.io/BenchmarkProfiles.jl/stable/)
10-
![Plots 1.38.5](https://img.shields.io/badge/Plots-1.38.5-000?style=flat-square&labelColor=999)
8+
![JSON 0.21.4](https://img.shields.io/badge/JSON-0.21.4-000?style=flat-square&labelColor=999)
9+
[![BenchmarkProfiles 0.4.6](https://img.shields.io/badge/BenchmarkProfiles-0.4.6-006400?style=flat-square&labelColor=389826)](https://jso.dev/BenchmarkProfiles.jl/stable/)
10+
![Plots 1.41.1](https://img.shields.io/badge/Plots-1.41.1-000?style=flat-square&labelColor=999)
11+
![Random 1.11.0](https://img.shields.io/badge/Random-1.11.0-000?style=flat-square&labelColor=999)
1112

1213

1314

14-
This tutorial is essentially a collection of examples.
15+
This tutorial demonstrates how to use BenchmarkProfiles.jl to visualize and compare solver performance across multiple test problems.
1516

1617
## Performance Profile
1718

18-
Performance profiles are straightforward to use. The input is a matrix `T` with entries `T[i,j]` indicating the cost to solve problem `i` using solver `j`. Cost can be, for instance, elapsed time, or number of evaluations. The cost should be positive. If any cost is zero, all measures will be shifted by 1.
19+
Performance profiles, introduced by Dolan and Moré (2002), provide a graphical way to compare the performance of multiple solvers across a test set. They show the fraction of problems solved by each solver as a function of a performance tolerance.
20+
21+
### Understanding Performance Profiles
22+
23+
The input is a matrix `T` with entries `T[i,j]` indicating the cost to solve problem `i` using solver `j`. Cost can be, for instance, elapsed time, number of iterations, or function evaluations. The cost should be positive. If any cost is zero, all measures will be shifted by 1.
24+
25+
The performance profile plots:
26+
- **x-axis (τ)**: Performance ratio - how much slower a solver is compared to the best solver for each problem. τ=1 means the solver was fastest, τ=2 means it took twice as long as the fastest solver.
27+
- **y-axis (ρ(τ))**: Fraction of problems solved within the performance ratio τ. ρ(2) = 0.8 means the solver solved 80% of problems within twice the time of the best solver.
28+
29+
**Key interpretations**:
30+
- The height at τ=1 (left side) shows the fraction of problems where the solver was fastest
31+
- The right-side height (as τ→∞) shows the fraction of problems successfully solved (robustness)
32+
- Higher curves are better - the solver solves more problems with smaller performance ratios
1933

2034
Basic usage:
2135

@@ -54,9 +68,19 @@ performance_profile(PlotsBackend(), T, ["Solver 1", "Solver 2", "Solver 3"])
5468

5569

5670

71+
### Customization Options
72+
5773
`Plots` arguments can be passed to `performance_profile()` or used as they normally would be with `Plots`.
5874
In the example below, we pass `xlabel` to `performance_profile` and set `ylabel` through `ylabel!`.
5975

76+
Common customization options:
77+
- `lw`: Line width
78+
- `c` or `color`: Line colors
79+
- `linestyles`: Line styles (`:solid`, `:dash`, `:dot`, etc.)
80+
- `xlabel`, `ylabel`: Axis labels
81+
- `title`: Plot title
82+
- `legend`: Legend position (e.g., `:bottomright`, `:topleft`)
83+
6084
```julia
6185
using Plots
6286

@@ -67,3 +91,15 @@ ylabel!("ρ(τ)")
6791
```
6892

6993
![](figures/index_4_1.png)
94+
95+
96+
97+
### Additional Parameters
98+
99+
The `performance_profile` function accepts several optional keyword arguments:
100+
101+
- `logscale::Bool=true`: Use logarithmic scale on the x-axis (default: true). Useful for viewing performance across a wide range of ratios.
102+
- `sampletol::Number=0`: Tolerance for sampling data points. Can reduce plot complexity for large datasets.
103+
- `title::String=""`: Title for the plot
104+
105+
For more details on performance profiles, see: Dolan, E. D., & Moré, J. J. (2002). Benchmarking optimization software with performance profiles. Mathematical Programming, 91(2), 201-213.

0 commit comments

Comments
 (0)