Add `ftol` keyword

[matthewelmer-tamu]

I'm transitioning over from NLopt.jl after being majorly burned by the COBYLA bug. One thing I miss is being able to set an ftol. Will this make a return for this updated library?

[amontoison]

Hi @matthewelmer-tamu!
Can you open a PR?
I can merge this feature if you need it.

[matthewelmer-tamu]

Glad to hear you're open to contributions! I'll take a look sometime later.

[emmt]

There are different stopping criteria in any of the PRIMA algorithms:

• convergence in the variables is controlled by the final size of the trust region which is set by the rhoend keyword, the trust region radius being computed according to the scale keyword;
• convergence in the function value is controlled by the ftarget keyword: convergence being assumed when the objective function becomes lower than this limit;
• keyword maxfun specifies the maximum number of function evaluations.

Using ftarget seems to be the closest to your needs except that it is a kind of absolute criterion while I guess that ftol set the stopping criterion in terms of a relative difference in objective function change between successive iterations.

To implement other stopping criteria in a flexible way, we could have the algorithms accept a user-defined callback that is called after every iteration with all needed information about the state of the algorithm. This callback shall returns whether to stop or to continue the iterations. This callback can also do many other things such as print iteration information, etc. This however implies modifying the Fortran version of the algorithms in PRIMA. @zaikunzhang do you think this is doable?

[emmt]

@matthewelmer-tamu Did you tried the different existing options mentioned above?