Proposal: Domain-safe L1 penalty wrapper for box constraints (alternative/complement to Fminbox)

[UweAlex]

Proposal: Add a robust domain-safe L1 penalty wrapper for box constraints as alternative/complement to Fminbox

Motivation

Many benchmark and real-world problems have box constraints that define the valid domain of the function (e.g., to avoid log(negative), sqrt(negative), or division by zero).

Fminbox is great but limited:

• Supports only LBFGS as inner solver.
• Gradient projection can cause NaN/instabilities when hitting domain boundaries exactly (seen on functions like bartelsconn, rump, alpinen1, bukin*).
• Pure projection + resetting struggles with some non-smooth/ill-conditioned cases.

A more flexible and robust approach is needed that works with any unconstrained optimizer (including NelderMead) while ensuring domain-safe evaluations.

Proposal: Domain-safe hybrid L1 penalty wrapper

Hybrid method combining:

• Pre-clamping of evaluation point to feasible region (domain-safe f/∇f).
• Fixed large L1 penalty (ρ = 1e6) on violations.
• Gradient override: violated components get +ρ or -ρ (strong inward pull).

Mathematical description (lb ≤ x ≤ ub):

x_clamped = clamp(x, lb, ub)
tilde_f(x) = f(x_clamped) + ρ * sum_i ( max(0, lb_i - x_i) + max(0, x_i - ub_i) )

Wrapped gradient:

• Compute ∇f at x_clamped
• Override violated components with +ρ or -ρ
• Keep ∇f elsewhere (including on bounds)

Key property: For any ρ > 0, no stationary points outside the box (||∇ tilde_f(x)|| ≥ ρ on violations) – stronger than classical exact L1 penalties.

Implementation (~100 lines pure Julia):
https://github.com/UweAlex/NonlinearOptimizationTestFunctions.jl/blob/master/src/l1_penalty_wrapper.jl

Tests:
https://github.com/UweAlex/NonlinearOptimizationTestFunctions.jl/blob/master/test/l1_penalty_wrapper_tests.jl

Usage Example

using Optim, LineSearches

wrapped = with_box_constraints(f, gradient!, lb, ub)

result = optimize(
    wrapped.f, wrapped.gradient!,
    x0,
    LBFGS(linesearch = BackTracking(order=3)),  # works with any optimizer
    Optim.Options(...)
)

Benchmark vs Fminbox (153 bounded functions, 50 feasible starts each ≈ 7650 runs)

--- Robustness ---
L1 more robust     : 15
Fminbox more robust: 6
Equal (>0%)        : 132

--- Efficiency (equal cases) ---
L1 faster          : 90
Fminbox faster     : 42

The wrapper often succeeds where Fminbox fails due to NaNs and uses fewer evaluations when both converge.
(Note: L1 used aggressive BackTracking(order=3) linesearch for non-smooth penalty.)

Full reproducible benchmark:
https://github.com/UweAlex/NonlinearOptimizationTestFunctions.jl/blob/master/examples/benchmark_box_constraints.jl

Test collection:
https://github.com/UweAlex/NonlinearOptimizationTestFunctions.jl

Advantages

• Any unconstrained optimizer (incl. derivative-free).
• No stationary points outside bounds for ρ > 0.
• Full domain safety.
• Simpler than projection/resetting.

Suggestion

Happy to contribute this as an optional wrapper (e.g. with_l1_box_constraints) to complement Fminbox. Ready to open a PR if there's interest.

Thanks!

[pkofod]

I am not at all happy with Fminbox. I will read your proposal in more detail and reply.