Update to most recent SSMProblems interface

[charlesknipp]

This fork contains all of the required changes to support the newest SSMProblems interface. I also took the liberty to add some syntactical sugar (see TracedSSM) which I haven't thoroughly tested, so it's sure to break down.

• unit tests pass
• examples run
  • linear Gaussian model
  • stochastic volatility
  • Gaussian process model (fixed, but could be iterated on)
  • Levy process model (minus jump storage)

A Philosophical Note

It seems like AbstractPS exists to define a general form SMC sampler, where the probablistic model need not be limited to state spaces. A look at the first chapter of Elements of Sequential Monte Carlo illustrates this fact. While SMC as it's defined here is conceptually identical to GeneralisedFilters, the optimizations built for SSMs and online nature of filtering diverge significantly from AbstractMCMC. This makes me wonder whether we should even depend on SSMProblems here, and instead leave this to an extension in SSMProblems.

[charlesknipp]

I should note that this requires dev SSMProblems since we have not yet added v0.5.2 to the registry, thus why CI is failing.

[yebai]

JuliaRegistries/General#129392 is blocking us from making a new SSMProblems release. Hopefully, this will get resolved in the next few days.

[charlesknipp]

@FredericWantiez I tried my best at getting the GP to work, but it's cooked. The original implementation didn't have the Markov properties SSMProblems was designed to handle. It should be possible to get working, but I don't know AbstractGPs well enough to make it happen.

@THargreaves I managed to fix the Levy Process SSM, minus jump storage. I can probably get this working, but your expertise is extremely valuable.

[charlesknipp]

Just to clarify, for most SSMProblems compatible models this is exactly how we determine dynamics/observation processes.

Although, overloading this allows us to do some pretty neat stuff with non-Markovian models (see the Gaussian Process example).

[FredericWantiez]

Looking at what I've done for the GP script, not sure it's correct given the non-markovian setup and the PGAS update implemented here

[charlesknipp]

Yeah... I'm not 100% sure this is the proper way to handle things for the GP demo. I only checked the plots and run-times, but from what I saw heuristically it looked good. Apart from some naive checks, this is outside of my expertise.

[FredericWantiez]

A bit of type piracy, can't we just use TracedSSM as the public API ?

model = TracedSSM(LinearGaussianStateSpaceModel(θ), Y)

sample(rng, traced_model, pgas, 500; progress=false)

[charlesknipp]

Totally fine with me. I just figured that if we interface SSMProblems with AbstractMCMC, then we would have to define the observables and state space model tangentially. The syntactical sugar just gets around having to export something that doesn't necessarily need to be public.

[yebai]

JuliaRegistries/General#129392 is blocking us from making a new SSMProblems release.

This is now fixed!

[charlesknipp]

Review Comments

The Gaussian process model is something I struggled to replicate under the new system. Personally, I believe that non-Markovian models should not be handled by the SSMProblems API. With that being said, I think the GP example could be written in such a way that preserves Markov-ness. Any ideas are welcome.

I should note that the existence of TracedSSM seems redundant. It exists because we seek to create a SSMTrace with a StateSpaceModel object, which does not store the trajectories. Since none of the Trace objects are public facing, I abused some type piracy to preserve this feature. Which, as @FredericWantiez pointed out, is bad.

Misc

I added some regards to the description. Contextualizing state space models in the Turing ecosystem really forced me to consider the way we handle sampling via AbstractMCMC. It's part of a much larger discussion, so I don't expect resolution in this PR.

[yebai]

With that being said, I think the GP example could be written in such a way that preserves Markov-ness. Any ideas are welcome.

Sparse GP priors would resolve this, since the GP prior over latent dynamics depend on a set of pseudo datapoints (known as inducing points) instead of the latent states (see Sec 4 of https://arxiv.org/abs/1306.2861). Then these pseudo datapoints are optimised conditioned on realisations of latent states (e.g. sampled trajectories from PMCMC).

[yebai]

Thanks @charlesknipp!