Skip to content

Commit f383661

Browse files
Fix typos and improve clarity in paper.md
1 parent 66ed713 commit f383661

File tree

1 file changed

+13
-13
lines changed

1 file changed

+13
-13
lines changed

paper/paper.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -46,15 +46,15 @@ The library provides a modular and extensible framework for experimenting with n
4646
- **Levenbergh-Marquardt methods (LM, LMTR)** [@aravkin-baraldi-orban-2024].
4747
- **Augmented Lagrangian methods (AL)** [@demarchi-jia-kanzow-mehlitz-2023].
4848

49-
These methods rely on the gradient and optionnally on the Hessian(-vector) information of the smooth part $f$ and the proximal mapping of the nonsmooth part $h$ in order to compute steps.
49+
These methods rely on the gradient and optionally on the Hessian(-vector) information of the smooth part $f$ and the proximal mapping of the nonsmooth part $h$ in order to compute steps.
5050
Then, the objective function $f + h$ is used only to accept or reject trial points.
5151
Moreover, they can handle cases where Hessian approximations are unbounded [@diouane-habiboullah-orban-2024;@leconte-orban-2023-2], making the package particularly suited for large-scale, ill-conditioned, and nonsmooth problems.
5252

5353
# Statement of need
5454

5555
## Model-based framework for nonsmooth methods
5656

57-
In Julia, \eqref{eq:nlp} can be solved using [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), which implements in-place, first-order, line-search–based methods[@stella-themelis-sopasakis-patrinos-2017;@themelis-stella-patrinos-2017].
57+
In Julia, \eqref{eq:nlp} can be solved using [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), which implements in-place, first-order, line-search–based methods [@stella-themelis-sopasakis-patrinos-2017;@themelis-stella-patrinos-2017].
5858
Most of these methods are generally splitting schemes that alternate between taking steps along the gradient of the smooth part $f$ (or quasi-Newton directions) and applying proximal steps on the nonsmooth part $h$.
5959
Currently, [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl) provides only L-BFGS as a quasi-Newton option.
6060
By contrast, [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) focuses on model-based approaches such as trust-region and quadratic regularization algorithms.
@@ -69,15 +69,17 @@ On the one hand, smooth problems $f$ can be defined via [NLPModels.jl](https://g
6969
Large collections of such problems are available in [Cutest.jl](https://github.com/JuliaSmoothOptimizers/CUTEst.jl) [@orban-siqueira-cutest-2020] and [OptimizationProblems.jl](https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl) [@migot-orban-siqueira-optimizationproblems-2023].
7070
Another option is to use [RegularizedProblems.jl](https://github.com/JuliaSmoothOptimizers/RegularizedProblems.jl), which provides problem instances commonly used in the nonsmooth optimization literature.
7171

72-
On the other hand, Hessian approximations of these functions, including quasi-Newton and diagonal schemes, can be specified through [LinearOperators.jl](https://github.com/JuliaSmoothOptimizers/LinearOperators.jl), which represents Hessians as linear operators and implements efficient Hessian–vector products.
72+
On the other hand, nonsmooth terms $h$ can be modeled using [ProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ProximalOperators.jl), which provides a broad collection of nonsmooth functions, together with [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl), which provides shifted proximal mappings for nonsmooth functions.
7373

74-
Finally, nonsmooth terms $h$ can be modeled using [ProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ProximalOperators.jl), which provides a broad collection of nonsmooth functions, together with [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl), which provides shifted proximal mappings for nonsmooth functions.
74+
## Support for Hessians and Hessian approximations of the smooth part $f$
7575

76-
## Support for Hessians of the smooth part $f$
76+
In contrast to [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) methods such as **R2N** and **TR** methods such as R2N and TR support exact Hessians as well as several Hessian approximations of $f$, which can significantly improve convergence rates—especially for ill-conditioned problems.
7777

78-
In contrast to [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) methods such as **R2N** and **TR** support Hessians of $f$, which can significantly improve convergence rates, especially for ill-conditioned problems.
79-
Hessians can be obtained via automatic differentiation through [ADNLPModels.jl](https://github.com/JuliaSmoothOptimizers/ADNLPModels.jl) or supplied directly as Hessian–vector products $v \mapsto Hv$.
80-
This enables algorithms to exploit second-order information without explicitly forming dense (or sparse) Hessians, which is often prohibitively expensive in both computation and memory, particularly in high-dimensional settings.
78+
Exact Hessians can be obtained via automatic differentiation through [ADNLPModels.jl](https://github.com/JuliaSmoothOptimizers/ADNLPModels.jl) or supplied directly as Hessian–vector products $v \mapsto Hv$.
79+
80+
Hessian approximations (e.g., quasi-Newton and diagonal schemes) can be specified via [LinearOperators.jl](https://github.com/JuliaSmoothOptimizers/LinearOperators.jl), which represents Hessians as linear operators and provides efficient implementations of Hessian–vector products.
81+
82+
This design allows algorithms to exploit second-order information **without** explicitly forming dense (or even sparse) Hessian matrices, which is often prohibitively expensive in time and memory, particularly at large scale.
8183

8284
## Requirements of the RegularizedProblems.jl
8385

@@ -115,7 +117,7 @@ Extensive documentation is provided, including a user guide, API reference, and
115117
Aqua.jl is used to test the package dependencies.
116118
Documentation is built using Documenter.jl.
117119

118-
## Solvers caracteristics
120+
## Solvers characteristics
119121

120122
All solvers in [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) are implemented in an in-place fashion, minimizing memory allocations during the resolution process.
121123
Moreover, they implement non-monotone strategies to accept trial points, which can enhance algorithmic performance in practice [@leconte-orban-2023;@diouane-habiboullah-orban-2024].
@@ -144,10 +146,8 @@ We compare the performance of our solvers with (**PANOC**) solver [@stella-theme
144146
We illustrate the capabilities of [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) on two nonsmooth and nonconvex problems:
145147

146148
- **Support Vector Machine (SVM) with $\ell^{1/2}$ penalty** for image classification [@aravkin-baraldi-orban-2024].
147-
- **Nonnegative Matrix Factorization (NNMF) with $\ell_0$ penalty and constraints** [@kim-park-2008].
149+
- **Nonnegative Matrix Factorization (NNMF) with $\ell_0$ penalty and bound constraints** [@kim-park-2008].
148150

149-
Both problems are of the form $\min f(x) + h(x)$ with $f$ nonconvex and $h$ nonsmooth.
150-
The NNMF problem can be set up in a similar way to the SVM case, with $h$ given by an $\ell_0$ norm and additional nonnegativity constraints.
151151
Below is a condensed example showing how to define and solve such problems:
152152

153153
```julia
@@ -164,7 +164,7 @@ stats = RegularizedExecutionStats(reg_nlp)
164164
solve!(solver, reg_nlp, stats; x=f.meta.x0, atol=1e-4, rtol=1e-4, verbose=0, sub_kwargs=(max_iter=200,))
165165
solve!(solver, reg_nlp, stats; x=f.meta.x0, atol=1e-5, rtol=1e-5, verbose=0, sub_kwargs=(max_iter=200,))
166166
```
167-
The NNMF problem can be set up in a similar way, replacing the model by nnmf_model(...) with bound constraints, $h$ by an $\ell_0$ norm and use an L-BFGS Hessian approximation.
167+
The NNMF problem can be set up in a similar way, replacing the model by nnmf_model(...), $h$ by an $\ell_0$ norm and use an L-BFGS Hessian approximation.
168168

169169
### Numerical results
170170

0 commit comments

Comments
 (0)