Add API for residuals and Jacobians for least-squares type log-densities

[jeffrey-hokanson-numerica]

Some recent MCMC samplers such as "Randomize-then-optimize" exploit least squares structure in the log-likelihood; i.e., the log-density has the form $f(x) = \sum_j f_j(x)^2$. Could we add this kind of derivative information into the API? As a proposal, I would suggest something like

residual(ℓ, x)
jacobian(ℓ, x)

which is similar to the NLSolversBase.jl API. Non-allocating versions would be helpful as well, e.g., writing into the last argument

residual!(ℓ, x, res)
jacobian!(ℓ, x, jac)

[tpapp]

Sorry for the late reply. I am not entirely sure that this package is the ideal home for an API like that, for the following reasons:

1. it is pretty much orthogonal to our current one
2. either we require that all implementations support it, or introduce further API to query support

I would recommend that a package using that algorithm just implements this API, and then if multiple methods end up using it it can be factored out to an interface package.