2023 sd conf poster Bridging Statistics and Dynamic Modeling with Vensim, Python and Stan

[hyunjimoon]

five scenarios of workflow with different sbc

• degeneracy example
  
  c=a+b, a, b don't payoff var(C)
  uncertainty of future temp

1. climate sensitivity (amt of outboud radiation)
2. rate of heat transfer into the ocean (how fast the heat taken up by the ocean)
3. size of the aerosol
   estimated parameter (contribution); don't know one parameter much (know the uncertainty of each offset each other)

todo

• 3-5 figures of workflow path with possible sbc diagnostics
• connect new example in sbc paper: three different implementation examples of simplex

questions to discuss next week:
Additionally, by running SBC, we get a simulation study
for free: in this case min is the most efficient in terms of effective sample size per second,
followed by gamma. The correct version of softmax performs worst.

Performance comparison comment is great. I'm just curious how fragile and data-dependent this performance ordering can be. The tradeoff may exist for three model formulations (min, softmax, gamma), meaning each of them may have a specific data space, however narrow, where it dominates the other formulations. For instance, dominance flips from noncentered to centered parameterization with increasing data amount -- could a similar story be told for, say min and gamma formulation (or even softmax)?

My point is performance comparison may misinform some users with specific data settings, and wonder whether there can be a principled workflow for more customized performance comparison, than recommending setting prior well.

[hyunjimoon]

• bayesian inference not normally on dynamic model
  once you do full round trip of synthetic ; test not only model but also inference algorithm in one package (validate prior etc )
• venpy that runs on vensim dll is another option
• pysd supports array, table fucntion
• HMC wouldn't be implemented within vensim in the near future (due to large number of discrete variable)

seems to have high interest in calibration

only word I heard was he practiced sd for 10yrs + saying our work is the future!

hazhir came (happy!) and discussed with Tom on:

• discrete variable (autodiff algorithm); more than n^2, it becomes less scalable -- so in the parameter space, where is the break down point?
• tom explained three steps: step1:?? step2: generate inference data via vensim; step3: (limit, synthetic data)
• some iteration between the shared and unshared model; not doing MCMC for full model (breaking model; start MCMC from country level - iterate over the whole set; shared parameter, pass the posterior (shared parameter - fix for the whole country); problem of scale - country 1 gets 1000)
• country specific (tight); how long does it take? (Powell is the bottleneck?? 3k computation cost), no hope for the whole thing - tom: shared and country-specific crack this then this is frontline

• right person asked "..??.. after all isn't it all optimization algorithm?" and Tom answered by giving example of two stock model (global temperature time series)
• kalman filter is not fully integrated (randomness https://stats.stackexchange.com/questions/435825/kalman-filter-and-monte-carlo) but MCMC has better convergence diagnostics
• connect from vensim to inferencedata right away is one goal

• she mentioned some "bayesian, agent-based modeling" and tom reasked "you're doing sd-thinking with non-sd tool?" but didn't follow

... TOM DRINKING white🍷 ...

seems to be a close friend of Tom, further discussed on ?? which ended up tom introducing simple model translate from vensim to python (venpy)

he introduced as Wane?? (and seems to know a lot about HMC) - his model has 75 parameter

frustrated by the shape of posterior samples ( | |, //, \ shape)

he came back -- must really like our research!

Trevor also came back and we promised to have lunch in Boston!