Can we include variation from noise into variation from prior?

[hyunjimoon]

If we were to imagine the last M number of effective MCMC draws, its variation would be the result of added variations from
a. prior
b. process noise
c. measure noise
d. MCMC sampling

@hyunjimoon thinks b, c can be included in a.
@tomfid may I ask your opinion?

[tomfid]

Does "effective" mean "accepted" here?

I think the process noise is potentially tricky, so I'll ignore that for the moment.

It seems like broader priors, or more measurement error in the data, should produce more variation in theta.

If the sampler is performing well, then shouldn't the variation in M draws be reasonably constant? OTOH if the sampler is underperforming (stuck in a corner, or creeping slowly), variation could be nonstationary unless M is very large?

If "included in a" means that part of theta is a parameter describing the scale of measurement error that produced y, I agree.

[hyunjimoon]

Does "effective" mean "accepted" here?

We need a unified terms for here as HMC is not have accept-reject. Hazhir and I agreed that for Stan's M, effective sample size from the this table below is usable for now.

[hyunjimoon]

process noise is potentially tricky

From implementation perspective, driving data and process noise are treated equally (both driving data and random variable's one realization of value are passed through stan data block in this line 7,8 of stancode). This is the only way to implement the random variable concept of SDE for now (and for the future as well, I think). This post explains Stan cannot solve SDE and introduces Mike Bentancourt's remedy.

We can’t do SDEs per se, but if you can transform the problem into a Gaussian process with an appropriate kernel then you can tackle it from the GP perspective. Also, you can do the INLA trick of discretizing the domain of the SDE and modeling it with random effects – that can do very well in many applications

The above comment reminds me of how you explained two ways to model demand data (driving data and parameterizing it with correlation and scale parameter). Viewing alpha and rho from Mike's Gaussian process writing as process noise scale and correlation time of process noise, I can't distinguish driving data and process noise well.. (let alone their equivalent implementation format).

[tomfid]

I think the relevant difference between driving data and process noise is that process noise is not observed directly, so you can only infer its distribution?

[hyunjimoon]

Do you think, between using one realization of data VS using one noise seed's uniform random series, there is much difference in degrees of clamping?