rework documentation

Author: MaxenceGollier

docs/make.jl

@@ -16,11 +16,7 @@ makedocs(
   sitename = "RegularizedOptimization.jl",
   pages = [
     "Home" => "index.md", 
-    "User guide" => [
-      joinpath("guide", "introduction.md"),
-      joinpath("guide", "algorithms.md"),
-      joinpath("guide", "custom.md")
-    ], 
+    "Algorithms" => "algorithms.md", 
     "Examples" => [
       joinpath("examples", "bpdn.md"),
       joinpath("examples", "fh.md")

docs/src/guide/algorithms.md docs/src/algorithms.mddocs/src/guide/algorithms.md renamed to docs/src/algorithms.md

@@ -1,4 +1,4 @@
-# Algorithms
+# [Algorithms](@id algorithms)
 
 ## General case
 The algorithms in this package are based upon the approach of [aravkin-baraldi-orban-2022](@cite).
@@ -34,30 +34,6 @@ The models for the smooth part `f` in this package are always quadratic models o
 where $H(x_0)$ is a symmetric matrix that can be either $0$, the Hessian of $f$ (if it exists) or a quasi-Newton approximation.
 Some algorithms require a specific structure for $H$, for an overview, refer to the table below.
 
-For the model of the regularizer $h$, it is less straightforward.
-First, we require the model to locally approximate $h$ sufficiently well, see [aravkin-baraldi-orban-2022; Model Assumption 3.2.](@cite)
-Then, the model $\psi$ should be such that the shifted proximal mapping
-```math
-\text{prox}_{\sigma^{-1}\psi} (q) := \underset{s \in \mathbb{R}^n}{\argmin} \psi(s;x_0) + \frac{\sigma}{2} \| s - q \|_2^2
-```
-can be computed efficiently. 
-
-!!! note
-    While the user can choose whichever regularizer he wants and then choose an appropriate model for it, he also needs to implement the function `prox!(y, ψ, q, σ)` which computes the proximal mapping in place. 
-
-For example, if the regularizer is the Euclidean norm, $h = \|cdot\|_2$, then we can simply choose the model as the function itself, $\psi(s ;x) := h(x + s)$ because
-```math
-\underset{s \in \mathbb{R}^n}{\argmin} \ \|x + s\|_2 + \frac{\sigma}{2} \| s - q \|_2^2 = (1 - \frac{1}{\sigma\|x + q\|})_+ (x + q) - x.
-```
-where $(\cdot)_+ := \max \{0, \cdot \}$.
-
-Another example is when the regularizer is the Euclidean norm composed with a function, $h = \|c(\cdot)\|_2$.
-In that case, taking the model as the function itself would make the proximal mapping intractable. 
-Instead, one can choose the model $\psi(s ;x) := \|c(x) + J(x)s\|_2$, which is a composition of an affine function with the Euclidean norm. 
-This approach was used in [diouane-gollier-orban-2024](@cite).
-
-For more information on custom regularizers and models thereof, please refer to [this section](@ref custom_regularizers).
-
 The following table gives an overview of the available algorithms in the general case.
 
 Algorithm | Quadratic Regularization | Trust Region | Quadratic term for $\varphi$ : H | Reference
@@ -72,7 +48,7 @@ Algorithm | Quadratic Regularization | Trust Region | Quadratic term for $\varph
 This package provides two algorithms, [`LM`](@ref LM) and [`LMTR`](@ref LMTR), specialized for regularized, nonlinear least-squares.
 That is, problems of the form
 ```math
-\underset{x \in \mathbb{R}^n}{\text{minimize}} \quad \frac{1}{2} \frac{1}{2}\|F(x)\|_2^2 + h(x),
+\underset{x \in \mathbb{R}^n}{\text{minimize}} \quad \frac{1}{2}\|F(x)\|_2^2 + h(x),
 ```
 where $F : \mathbb{R}^n \mapsto \mathbb{R}^m$ is continuously differentiable and $h : \mathbb{R}^n \mapsto \mathbb{R} \cup \{\infty\}$ is lower semi-continuous.
 In that case, the model $\varphi$ is defined as 

docs/src/guide/custom.md

@@ -29,12 +29,63 @@ We refer to [ManualNLPModels.jl](https://github.com/JuliaSmoothOptimizers/Manual
 
 ---
 
-## Features
+## Regularizers
 
-- All solvers in RegularizedOptimization.jl have **in-place versions**.  
-  Users can preallocate a workspace and reuse it across solves to avoid memory allocations, which is useful in repetitive scenarios.  
+Regularizers used in this package are based on the [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl) API, which is related to [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl). 
 
-- Solvers work with any floating-point type.
+The solvers in this package work by approximating the regularizer with a *shifted model*.
+That is, at each iterate $x_k$, we approximate $h(x_k + s)$ with a (simpler) function $\psi(s; x_k)$.
+For example, if $h(x) = \|x\|$, then its *shifted model* is simply the function $h$ itself : $\psi(s; x_k) = \|x_k + s\|$.
+On the other hand, if $h$ is the composition of a norm with a function, $h(x) = \|c(x)\|$, then its *shifted model* can be the approximation
+```math
+\psi(s; x_k) = \|c(x_k) + J(x_k)s\| \approx \|c(x_k + s) \| = h(x_k + s),
+```
+where $J(x_k)$ is the Jacobian of $c$ at the point $x_k$.
+
+Basically, we expect a regularizer `h::Foo` to
+
+- Be callable with vectors, i.e. to implement `(h::Foo)(x::AbstractVector)`.
+- Be *shifteable*, that is, to implement a function `shifted(h::Foo, x::AbstractVector)` that returns the shifted model `ψ::ShiftedFoo`.
+
+Next, we expect the shifted model `ψ::ShiftedFoo` to 
+
+- Be callable with vectors, i.e. to implement `(ψ::ShiftedFoo)(x::AbstractVector)`.
+- Be *shifteable*, that is, to implement a function `shifted(ψ::ShiftedFoo, x::AbstractVector)` that returns a shifted model `ψ'::ShiftedFoo`. Moreover, we should be able to change the shift in place, that is, the function `shift!(ψ::ShiftedFoo, x::AbstractVector)` should be implemented as well.
+- Be *proximable*, that is, to implement the inplace proximal mapping `prox!(y::AbstractVector, ψ::ShiftedFoo, q::AbstractVector, σ::Real)`.
+
+The proximal mapping is defined as 
+```math
+\text{prox}(\psi, q, \sigma) := \argmin_y \ \psi(y) + \frac{\sigma}{2} \|y - q\|_2^2.
+```
+
+!!! note
+    The package [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl) mostly implements the shifted models `ψ`. 
+    For the unshifted version, these are often implemented in [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl) so that you might actually need to install the latter. For example, if you wish to use the L0 norm as a regularizer, then you should define `h` as `h = NormL0(1.0)` with [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl), you don't need to do anything else in this case because the shifted model of the L0 norm is already implemented in [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl). 
+
+!!! warning 
+    The shifted model being proximable means that our solvers will not be able to automagically solve with any nonsmooth function that is given to it. Rather, the user is expected to provide an efficient solver for the proximal mapping.
+
+The following table shows which regularizers are readily available and which dependency is required to use the regularizer (the shifted model is always in `ShiftedProximalOperators.jl`).
+
+Regularizer | Shifted Model | Julia | Dependency
+------------|---------------|-------|-----------
+$\lambda ∥x∥_0$ | $\lambda ∥x + s∥_0$ | [`NormL0(λ)`](https://juliafirstorder.github.io/ProximalOperators.jl/stable/functions/#ProximalOperators.NormL0) | [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl)
+$\lambda ∥x∥_1$ | $\lambda ∥x + s∥_1$ | [`NormL1(λ)`](https://juliafirstorder.github.io/ProximalOperators.jl/stable/functions/#ProximalOperators.NormL1) | [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl)
+$\lambda ∥x∥_2$ | $\lambda ∥x + s∥_2$ | [`NormL2(λ)`](https://juliafirstorder.github.io/ProximalOperators.jl/stable/functions/#ProximalOperators.NormL2) | [ProximalOperators.jl](https://github.com/JuliaFirstOrder/ProximalOperators.jl)
+$\lambda ∥c(x)∥_2$ | $\lambda ∥c(x) + J(x)s∥_2$ | [`CompositeNormL2(λ)`](https://jso.dev/ShiftedProximalOperators.jl/dev/reference/#ShiftedProximalOperators.CompositeNormL2) | [ShiftedProximalOperators.jl](https://github.com/JuliaSmoothOptimizers/ShiftedProximalOperators.jl)
+
+---
+
+## Algorithms
+
+A presentation of each algorithm is given [here](@ref algorithms).
+
+---
+
+## Preallocating
+
+All solvers in RegularizedOptimization.jl have **in-place versions**.  
+Users can preallocate a workspace and reuse it across solves to avoid memory allocations, which is useful in repetitive scenarios.  
 
 ---
 
@@ -44,7 +95,7 @@ RegularizedOptimization can be installed through the Julia package manager:
 
 ```julia
 julia> ]
-pkg> add RegularizedOptimization
+pkg> add https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl
 ```
 
 ---