feat: use JVPCache for FiniteDiff pushforwards

[gdalle]

• For array-to-array pushforwards with FiniteDiff, enable preparation thanks to the new JVPCache
• Add benchmarks proving that sparse Jacobians are now alloc-free for ForwardDiff and FiniteDiff

Related:

• add finite_difference_jvp FiniteDiff.jl#191

[oscardssmith]

Is this sufficient for sparse finitediff jacobians to be supported?

[gdalle]

Yes, but we'll have to benchmark performance to make sure that they are non-allocating

[oscardssmith]

The in place is fully non-allocating.

julia> g(dx,x) = dx.=x*2
g (generic function with 1 method)

julia> cache = FiniteDiff.JVPCache(rand(3), rand(3), Val(:forward))
FiniteDiff.JVPCache{Vector{Float64}, Vector{Float64}, Val{:forward}()}([0.3185378518133124, 0.9795057009072153, 0.007265694202993944], [0.25969842682138, 0.5029073571918922, 0.41967454290961415])

julia> @btime FiniteDiff.finite_difference_jvp!($(ones(3)), g, $(ones(3)), $(ones(3)), $cache)
  25.392 ns (0 allocations: 0 bytes)

[gdalle]

Nice! I'll add a non-allocation test to this PR to prevent regressions

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.00%. Comparing base (0ea7f1d) to head (340e4f3).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #705      +/-   ##
==========================================
+ Coverage   97.96%   98.00%   +0.03%     
==========================================
  Files         119      121       +2     
  Lines        6297     6353      +56     
==========================================
+ Hits         6169     6226      +57     
+ Misses        128      127       -1     

Flag	Coverage Δ
DI	98.96% <100.00%> (+0.03%)	⬆️
DIT	95.91% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.