Skip to content

Commit 66ed713

Browse files
minor comments
1 parent 1ba1185 commit 66ed713

File tree

2 files changed

+15
-4
lines changed

2 files changed

+15
-4
lines changed

paper/paper.bib

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,3 +137,14 @@ @article{demarchi-jia-kanzow-mehlitz-2023
137137
pages = {863--896},
138138
doi = {10.1007/s10107-022-01922-4},
139139
}
140+
141+
@Article{ themelis-stella-patrinos-2017,
142+
Author = {Themelis, Andreas and Stella, Lorenzo and Patrinos, Panagiotis},
143+
Title = {Forward-Backward Envelope for the Sum of Two Nonconvex Functions: Further Properties and Nonmonotone line seach Algorithms},
144+
Journal = siopt,
145+
Year = 2018,
146+
Volume = 28,
147+
Number = 3,
148+
Pages = {2274--2303},
149+
doi = {10.1137/16M1080240},
150+
}

paper/paper.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,20 +46,20 @@ The library provides a modular and extensible framework for experimenting with n
4646
- **Levenbergh-Marquardt methods (LM, LMTR)** [@aravkin-baraldi-orban-2024].
4747
- **Augmented Lagrangian methods (AL)** [@demarchi-jia-kanzow-mehlitz-2023].
4848

49-
These methods rely solely on the gradient and Hessian(-vector) information of the smooth part $f$ and the proximal mapping of the nonsmooth part $h$ in order to compute steps.
49+
These methods rely on the gradient and optionnally on the Hessian(-vector) information of the smooth part $f$ and the proximal mapping of the nonsmooth part $h$ in order to compute steps.
5050
Then, the objective function $f + h$ is used only to accept or reject trial points.
5151
Moreover, they can handle cases where Hessian approximations are unbounded [@diouane-habiboullah-orban-2024;@leconte-orban-2023-2], making the package particularly suited for large-scale, ill-conditioned, and nonsmooth problems.
5252

5353
# Statement of need
5454

5555
## Model-based framework for nonsmooth methods
5656

57-
There exists a way to solve \eqref{eq:nlp} in Julia using [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), which implements in-place first-order line search–based methods for \eqref{eq:nlp}.
57+
In Julia, \eqref{eq:nlp} can be solved using [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl), which implements in-place, first-order, line-search–based methods[@stella-themelis-sopasakis-patrinos-2017;@themelis-stella-patrinos-2017].
5858
Most of these methods are generally splitting schemes that alternate between taking steps along the gradient of the smooth part $f$ (or quasi-Newton directions) and applying proximal steps on the nonsmooth part $h$.
5959
Currently, [ProximalAlgorithms.jl](https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl) provides only L-BFGS as a quasi-Newton option.
6060
By contrast, [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) focuses on model-based approaches such as trust-region and quadratic regularization algorithms.
6161
As shown in [@aravkin-baraldi-orban-2022], model-based methods typically require fewer evaluations of the objective and its gradient than first-order line search methods, at the expense of solving more involved subproblems.
62-
Although these subproblems may require many proximal iterations, each proximal computation is inexpensive, making the overall approach efficient for large-scale problems.
62+
Although these subproblems may require many proximal iterations, each proximal computation is inexpensive for several commonly used nonsmooth functions, such as separable penalties and bound constraints (see examples below), making the overall approach efficient for large-scale problems.
6363

6464
Building on this perspective, [RegularizedOptimization.jl](https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl) implements state-of-the-art algorithms for solving problems of the form $f(x) + h(x)$, where $f$ is smooth and $h$ is nonsmooth.
6565
The package provides a consistent API to formulate optimization problems and apply different regularization methods.
@@ -164,7 +164,7 @@ stats = RegularizedExecutionStats(reg_nlp)
164164
solve!(solver, reg_nlp, stats; x=f.meta.x0, atol=1e-4, rtol=1e-4, verbose=0, sub_kwargs=(max_iter=200,))
165165
solve!(solver, reg_nlp, stats; x=f.meta.x0, atol=1e-5, rtol=1e-5, verbose=0, sub_kwargs=(max_iter=200,))
166166
```
167-
The NNMF problem can be set up in a similar way, replacing the model by nnmf_model(...) and $h$ by an $\ell_0$ norm and we set LBFGS as Hessian approximation.
167+
The NNMF problem can be set up in a similar way, replacing the model by nnmf_model(...) with bound constraints, $h$ by an $\ell_0$ norm and use an L-BFGS Hessian approximation.
168168

169169
### Numerical results
170170

0 commit comments

Comments
 (0)