Section on sensitivity analysis of ODEs is non-sensical

[mhauru]

See here: https://turinglang.org/docs/tutorials/bayesian-differential-equations/#scaling-to-large-models-adjoint-sensitivities

The text claims that the code demonstrates something about sensitivity analysis, but we just import SciMLSensitivity and then MCMC sample with NUTS.

[AoifeHughes]

Do we want this removed, clarify what is actually shown in the example, or give an actual proper example?

[penelopeysm]

I figured this out the other day (I will say that the role of each package in the DiffEq ecosystem is not very well explained). Sampling from an ODE with ForwardDiff 'just works', but to sample from an ODE with Mooncake (or reverse-mode Enzyme), it requires SciMLSensitivity to be loaded. (I think this is what they call 'adjoint sensitivities' i.e. adjoint being a keyword for reverse-mode, and sensitivity being a keyword for gradients.)

So I think the point to be made here in the docs is that:

• If you have a large model (many params), reverse mode is faster, such as Mooncake
• However, differentiating through DEs with Mooncake requires SciMLSensitivity to be loaded
• Hence we load it
• And then that lets us run the ODE example with Mooncake

[mhauru]

Sampling from an ODE with ForwardDiff 'just works', but to sample from an ODE with Mooncake (or reverse-mode Enzyme), it requires SciMLSensitivity to be loaded.

Ewww, this seems far from ideal. Is this some weird side-effect of SciMLSensitivity, or how it's intended to work and its name is just misleading? I get that the two could be related because sensitivity analysis works by taking derivatives, but this seems like a very odd interface.

[penelopeysm]

I'm not sure, SciML or AD folks would have a better understanding of it. From what I've seen I get the impression that that hooking into DEs is really complicated and can't just be solved by defining a few rules in something like MooncakeOrdinaryDiffEqExt.