Purpose of duplicate

[willtebbutt]

@theogf what's the use-case for duplicate? I can see it in a few places around the code base, but it doesn't seem to be tested very often and I'm not really sure what it does.

[theogf]

It was a first attempt to be able to use ForwardDiff for differentiating kernels. The idea was to call duplicate(k::Kernel, theta::Vector) to create a new kernel with a structure identical to k with new parameters theta. But I never concretized it.

[willtebbutt]

Okay. Shall we remove that code then? Can always reinstantiate it later if needs be.

FWIW, I'm quite surprised that you would need to add something like that to make the package work with ForwardDiff.

[theogf]

Yes we can remove the code for now

Well ForwardDiff will only accept an array as an argument. So if you define the mapping from a vector to your kernel structure yourself it would work of course. I was trying to define an automatic way to do it.

[theogf]

duplicate is now useless as one can just use destructure from Flux (which hopefully will be moved to Functors.jl), here is a small example :

using KernelFunctions, Flux, ForwardDiff
k = 3.0 * transform(SqExponentialKernel(), rand(3))
ps, re = Flux.destructure(k)
X = rand(3, 20)
g = ForwardDiff.gradient(ps) do x
  sum(kernelmatrix(re(x), X, obsdim = 2))
end

[devmotion]

I guess Flux.destructure can't just be replaced with Functors.functor in this example since it does not return an array of parameters?

[theogf]

No it requires a bit more work, Flux.destructure is defined here : https://github.com/FluxML/Flux.jl/blob/8b5303328f8fd44d497bdaa7b59097ec54af0d47/src/utils.jl#L421

[devmotion]

Yeah that motivated my question - ForwardDiff requires a parameter vector but in general I'd assume depending on the AD backend it might not be necessary to stack together everything in a parameter vector, and the (named) tuple you get from functor might be enough.

[theogf]

Ah of course, if you can pass a named tuple that's great, but that's not true for most AD right now no? Like ForwardDiff, ReverseDiff etc..