Using Optimization.MaxSense

[sethaxen]

Currently we need to negate the user-provided function, gradients, and Hessian to to maximize the log-density with Optimization.jl, which by default assumes the optimization problem should be minimized. However, OptimizationProblem can take a sense=Optimization.MaxSense keyword to treat the optimization problem as a maximization one.

This is only worth doing if we can make sure that whether we use Optim.jl or Optimization.jl, the downstream code can be the same.

[sethaxen]

Coming back to this, Optimization/SciMLBase use OptimizationFunction purely to specify the function and derivatives. There's absolutely no information contained there to indicate whether this function will ultimately be maximized or minimized. That information is contained in sense of OptimizationProblem.

Thus, it makes sense for us to remove OptimizationFunction as an allowed type a user can provide and instead just take OptimizationProblem, as we do now. Then we can set MaxSense whenever we construct a problem and otherwise read the sense parameter for a user-provided problem to interpret the function as a positive or negative log-density.

Turing has a utility function for us to construct a problem (they don't set sense, so they default to MinSense), but we need to check that Turing can take our initialization.

[sethaxen]

It seems Optimization is not yet consistent regarding the sign of the current objective and gradient value available to the callback when sense=MaxSense, see SciML/Optimization.jl#858. So using MaxSense would currently require us to recompute these values.