Questions about AD support

[gdalle]

I have a few questions about AD in Turing and this docs page.

For ForwardDiff, pass adtype=AutoForwardDiff(; chunksize) to the sampler constructor. A chunksize of 0 permits the chunk size to be automatically determined.

In ADTypes, the default chunksize is nothing, why adopt a different convention here? If you end up using DifferentiationInterface, chunksize=0 will throw an error on every input.

An additional argument can be provided to AutoReverseDiff to specify whether to to compile the tape only once and cache it for later use (false by default, which means no caching tape). [...] Compiled tapes should only be used if you are absolutely certain that the computation doesn’t change between different executions of your model.

If I understand correctly, this holds for all tapes, compiled or not?

[willtebbutt]

I'll let others address your point about ForwardDiff, however

If I understand correctly, this holds for all tapes, compiled or not?

Maybe there's some terminological confusion here. I think we need to distinguish between "caching" and "compiling". Control-flow related problems are associated to "caching" of a tape, while whether a tape is "compiled" or not (i.e. whether you've called ReverseDiff.compile on a tape that you're re-using each time you call ReverseDiff.gradeint) is irrelevant.

Is this also your mental model @gdalle ? If so, it might be that we should clarify the writing very slightly.

[gdalle]

Yes that is the point I wanted to make. But I'm not sure where in Turing tapes are cached as opposed to re-created, and whether the user can modify that.

[gdalle]

Maybe this information (whether tape reuse is allowed) should go in the backend object from ADTypes.

[willtebbutt]

Thanks for asking about this, and apologies for my confused response. Lets direct conversation to #548 now, rather than this issue.