callbacks

[longemen3000]

i was looking and comparing the code with Optim.jl, and one of the only missing features is the ability to provide a callback function to short-circuit procedure

[pkofod]

Yes, that is correct. Work has been stopping progress here, but I'll soon make another push.

[longemen3000]

btw, i'm using this package, Mainly for non linear solving, both linesearch and trust region methods (not optimization yet, but ready to use when necessary), and i found some things:

1. retrieval of results of non linear systems are a little inconsistent. I found is that, the zeros of an NEqProblem using linesearch are stored in res.info.solution, whereas the same results using a trust region approach are stored in res.info.zero . i have a helper function to ease the access:

    function x_sol(res::NLSolvers.ConvergenceInfo{<:NLSolvers.LineSearch, <:Any, <:NLSolvers.NEqOptions})
        res.info.solution
    end

    function x_sol(res::NLSolvers.ConvergenceInfo{<:NLSolvers.TrustRegion, <:Any, <:NLSolvers.NEqOptions})
        return res.info.zero
    end

2. i use AD with non linear systems, and the jv!(x), and knowing that ForwardDiff.jl doesn't implement such function, i used an implementation in SparseDiffTools.jl:

struct DeivVecTag end

function jacvec!(dy, f, x, v,
                      cache1 = ForwardDiff.Dual{DeivVecTag}.(x, v),
                      cache2 = ForwardDiff.Dual{DeivVecTag}.(x, v))
    cache1 .= Dual{DeivVecTag}.(x, v)
    f(cache2,cache1)
    dy .= partials.(cache2, 1)
end

function jacvec(f, x, v)
    partials.(f(Dual{DeivVecTag}.(x, v)), 1)
end

#inside the auto AD of the NEqProblem, with f!(result, x) as input
    function jv!(x)
        function JacV(dy,v)
            return jacvec!(dy,f!,x,v)
        end
        return LinearMap(JacV,length(x))
    end

3. Surprisinly, The non linear solving procedure is ForwardDiff-friendly. Ideally dispatching on AbstractVector{<:Dual} and defining frules and rrules could be further advancing on that regard, but the status quo is favorable, as i can't use reverse mode AD with the functions i'm working with.

as always, great work!

[pkofod]

retrieval of results of non linear systems are a little inconsistent. I found is that, the zeros of an NEqProblem using linesearch are stored in res.info.solution, whereas the same results using a trust region approach are stored in res.info.zero . i have a helper function to ease the access:

Good point. Ideally it should be retrieved using a solution_candidate function or similar. (edit: solution_candidate is not a good name as this just means it's a feasible iterate...)

. i use AD with non linear systems, and the `jv!(x)`, and knowing that `ForwardDiff.jl` doesn't implement such function, i used an implementation in `SparseDiffTools.jl`:

I suppose you mean that you're using the krylov methods? Yes that interface is not really polished. Happy to improve here.

1. Surprisinly, The non linear solving procedure is ForwardDiff-friendly. Ideally dispatching on `AbstractVector{<:Dual}` and defining `frule`s and `rrules` could be further advancing on that regard, but the status quo is favorable, as i can't use reverse mode AD with the functions i'm working with.

You mean that you've successfully nested your solve in some outer function that you use ForwardDiff on? That's good to hear. Would maybe be a good idea to test that. I am not too familiar with the frules and rrules setup, but it seems like the community is going in that direction, so I'm all for it!

Thanks for taking this for a spin. I hope to spend some time on it very soon