Add tutorial benchmarking subsolvers (#139)

Author: tmigot

tutorials/advanced-jsosolvers/Project.toml

@@ -0,0 +1,15 @@
+[deps]
+ADNLPModels = "54578032-b7ea-4c30-94aa-7cbd1cce6c9a"
+JSOSolvers = "10dff2fc-5484-5881-a0e0-c90441020f8a"
+Krylov = "ba0b0d4f-ebba-5204-a429-3ac8c609bfb7"
+OptimizationProblems = "5049e819-d29b-5fba-b941-0eee7e64c1c6"
+Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
+SolverBenchmark = "581a75fa-a23a-52d0-a590-d6201de2218a"
+
+[compat]
+ADNLPModels = "0.7"
+JSOSolvers = "0.11"
+Krylov = "0.9"
+OptimizationProblems = "0.7"
+Plots = "1"
+SolverBenchmark = "0.6"

tutorials/advanced-jsosolvers/index.jmd

@@ -0,0 +1,95 @@
+---
+title: "Comparing subsolvers for NLS solvers"
+tags: ["solvers", "krylov", "benchmark", "least squares"]
+author: "Tangi Migot"
+---
+
+# Comparing subsolvers for nonlinear least squares JSOSolvers solvers
+
+This tutorial showcases some advanced features of solvers in JSOSolvers.
+
+```julia
+using JSOSolvers
+```
+
+We benchmark different subsolvers used in the solvers TRUNK for unconstrained nonlinear least squares problems.
+The first step is to select a set of problems that are nonlinear least squares.
+
+```julia
+using ADNLPModels
+using OptimizationProblems
+using OptimizationProblems.ADNLPProblems
+df = OptimizationProblems.meta
+names = df[(df.objtype .== :least_squares) .& (df.contype .== :unconstrained), :name]
+ad_problems = (eval(Meta.parse(problem))(use_nls = true) for problem ∈ names)
+```
+
+These problems are [`ADNLSModel`](https://github.com/JuliaSmoothOptimizers/ADNLPModels.jl) so derivatives are generated using automatic differentiation.
+
+```julia
+nls = first(ad_problems)
+typeof(nls)
+```
+
+The solvers TRON and TRUNK are trust-region based methods that compute a search direction by means of solving iteratively a linear least squares problem.
+For this task, several solvers are available.
+
+```julia
+JSOSolvers.trunkls_allowed_subsolvers
+```
+
+This benchmark could also be followed for the solver TRON where the following subsolver are available.
+
+```julia
+JSOSolvers.tronls_allowed_subsolvers
+```
+
+These linear least squares solvers are implemented in the package [Krylov.jl](https://github.com/JuliaSmoothOptimizers/Krylov.jl).
+
+```julia
+using Krylov
+```
+
+We define a dictionary of the different solvers that will be benchmarked.
+We consider here four variants of TRUNK using the different subsolvers.
+
+```julia
+solvers = Dict(
+  :trunk_cgls => model -> trunk(model, subsolver_type = CglsSolver),
+  :trunk_crls => model -> trunk(model, subsolver_type = CrlsSolver),
+  :trunk_lsqr => model -> trunk(model, subsolver_type = LsqrSolver),
+  :trunk_lsmr => model -> trunk(model, subsolver_type = LsmrSolver)
+)
+```
+
+Using [`SolverBenchmark.jl`](https://github.com/JuliaSmoothOptimizers/SolverBenchmark.jl) functionalities, the solvers are executed over all the test problems.
+
+```julia
+using SolverBenchmark
+stats = bmark_solvers(solvers, ad_problems)
+```
+
+The result is stored in a dictionary of `DataFrame` that can be used to analyze the results.
+
+```julia
+first_order(df) = df.status .== :first_order
+unbounded(df) = df.status .== :unbounded
+solved(df) = first_order(df) .| unbounded(df)
+
+costnames = ["time"]
+costs = [df -> .!solved(df) .* Inf .+ df.elapsed_time]
+```
+
+We compare the four variants based on their execution time.
+More advanced comparisons could include the number of evaluations of the objective, gradient, or Hessian-vector products.
+
+```julia
+using Plots
+gr()
+
+profile_solvers(stats, costs, costnames)
+```
+
+The CRLS and CGLS variants are the ones solving more problems, and even though the difference is rather small the CGLS variant is consistently faster which seems to indicate that it is the most appropriate subsolver for TRUNK.
+The size of the problems were rather small here, so this should be confirmed on larger instance.
+Moreover, the results may vary depending on the origin of the test problems.