Skip to content

Commit cf180e3

Browse files
sunxd3sunxd3
committed
add experiment code
1 parent d8874e4 commit cf180e3

15 files changed

+2223
-0
lines changed

‎JuliaBUGS/experiments/Project.toml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
[deps]
2+
ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"
3+
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
4+
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
5+
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
6+
Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6"
7+
JuliaBUGS = "ba9fb4c0-828e-4473-b6a1-cd2560fee5bf"
8+
AbstractPPL = "7a57a42e-76ec-4ea3-a279-07e840d6d9cf"
9+
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
10+
LogDensityProblems = "6fdf6af0-433a-55f7-b3ed-c6c6e0b8df7c"
11+
LogDensityProblemsAD = "996a588d-648d-4e1f-a8f0-a84b347e47b1"
12+
MetaGraphsNext = "fa8bd995-216d-47f1-8a91-f3b68fbeb377"
13+
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
14+
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
15+
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
16+
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
17+
AdvancedHMC = "0bf59076-c3b1-5ca4-86bd-e02cd72cde3d"
18+
AbstractMCMC = "80f14c24-f653-4e6a-9b94-39d6b0f70001"
19+
MCMCChains = "c7f686f2-ff18-58e9-bc7b-31028e88f75d"
20+
LogExpFunctions = "2ab3a3ac-af41-5b50-aa03-7779005ae688"

‎JuliaBUGS/experiments/README.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Experiments Workspace
2+
3+
- Project path: `JuliaBUGS/experiments`
4+
- Scripts live in `JuliaBUGS/experiments/scripts/`
5+
- Shared helpers are in `JuliaBUGS/experiments/utils.jl`
6+
7+
## Running scripts
8+
9+
Always pass the experiments project to Julia so the correct environment loads:
10+
11+
```
12+
julia --project=JuliaBUGS/experiments scripts/hmm_marginal_logp.jl
13+
```
14+
15+
Most scripts accept environment variables to tweak configurations. The simple
16+
HMM example supports:
17+
18+
- `AM_SEED` (default `1`) – RNG seed used for synthetic data.
19+
- `AM_T` (default `50`) – length of the simulated sequence.
20+
21+
Batch sweep (`hmm_correctness_sweep.jl`):
22+
23+
- `AM_SWEEP_SEEDS` (default `1`) – comma-separated list of seeds.
24+
- `AM_SWEEP_K` (default `2,4`) – comma-separated list of state counts.
25+
- `AM_SWEEP_T` (default `50,200`) – comma-separated list of sequence lengths.
26+
27+
GMM sweep (`gmm_correctness_sweep.jl`):
28+
29+
- `AG_SWEEP_SEEDS` (default `1`) – comma-separated list of seeds.
30+
- `AG_SWEEP_K` (default `2,4`) – comma-separated list of mixture counts.
31+
- `AG_SWEEP_N` (default `100,1000`) – comma-separated list of observation counts.
32+
33+
HMM gradient check (`hmm_gradient_check.jl`):
34+
35+
- `AGC_SEED` (default `1`) – RNG seed for synthetic data.
36+
- `AGC_K` (default `2`) – number of HMM states.
37+
- `AGC_T` (default `50`) – length of the simulated sequence.
38+
- `AGC_EPS` (default `1e-5`) – step size for central finite differences.
39+
- `AGC_VERBOSE` (default `0`) – set `1` to print per-θ details.
40+
- `AGC_SWEEP_SEEDS` – comma-separated list of seeds (overrides `AGC_SEED`).
41+
- `AGC_SWEEP_K` – comma-separated list of state counts (overrides `AGC_K`).
42+
- `AGC_SWEEP_T` – comma-separated list of sequence lengths (overrides `AGC_T`).
43+
44+
GMM gradient check (`gmm_gradient_check.jl`):
45+
46+
- `AGG_SEED` (default `1`) – RNG seed for synthetic data.
47+
- `AGG_K` (default `2`) – number of mixture components.
48+
- `AGG_N` (default `200`) – number of observations.
49+
- `AGG_EPS` (default `1e-5`) – step size for central finite differences.
50+
- `AGG_VERBOSE` (default `0`) – set `1` to print per-θ details.
51+
- `AGG_SWEEP_SEEDS` – comma-separated list of seeds (overrides `AGG_SEED`).
52+
- `AGG_SWEEP_K` – comma-separated list of mixture counts (overrides `AGG_K`).
53+
- `AGG_SWEEP_N` – comma-separated list of observation counts (overrides `AGG_N`).
54+
55+
HMM scaling benchmark (`hmm_scaling_bench.jl`):
56+
57+
- `AS_SEED` (default `1`) – RNG seed. Use `AS_SWEEP_SEEDS` for a list.
58+
- `AS_K` (default `2,4`) – number of states. Use `AS_SWEEP_K` for a list.
59+
- `AS_T` (default `50,200`) – sequence length. Use `AS_SWEEP_T` for a list.
60+
- `AS_TRIALS` (default `5`) – number of timing repetitions per case.
61+
62+
Notes:
63+
- The benchmark enforces the interleaved (time-first) order to reflect optimal scaling.
64+
65+
FHMM order comparison (`fhmm_order_comparison.jl`):
66+
67+
- `AFH_SEED` (default `1`) – RNG seed.
68+
- `AFH_C` (default `2`) – number of chains.
69+
- `AFH_K` (default `4`) – number of states per chain.
70+
- `AFH_T` (default `100`) – length of the sequence.
71+
- `AFH_TRIALS` (default `10`) – timing samples per order.
72+
- `AFH_MODE` (default `frontier`) – `frontier` or `timed`. Interleaved is always timed; the bad order is timed only if its proxy cost ≤ `AFH_COST_THRESH` (or when `AFH_MODE=timed`).
73+
- `AFH_COST_THRESH` (default `1e8`) – threshold on the proxy Σ K^width (compared in log-space) to avoid intractable timings.
74+
- `AFH_ORDERS` (optional) – comma‑separated list of orders to run. Accepted values: `interleaved`, `states_then_y`. Default runs both in that order. Example: `AFH_ORDERS=interleaved` or `AFH_ORDERS=states_then_y`.
75+
76+
Outputs CSV lines with columns
77+
`order,max_frontier,mean_frontier,sum_frontier,log_cost_proxy,min_time_sec,logp`.
78+
Two consistent orders are evaluated:
79+
- `interleaved` (time‑first, tractable)
80+
- `states_then_y` (all z’s, then all y’s; typically intractable for moderate T)
81+
82+
Output: CSV lines with columns
83+
`seed,K,T,trials,min_time_sec,logp,max_frontier,mean_frontier,sum_frontier`.
84+
85+
When adding new scripts, document their environment variables near the top of
86+
the file and list them here for quick reference.

‎JuliaBUGS/experiments/experiment_plan.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Streamlined Auto-Marginalization Experiment Plan
2+
3+
## Core Validation: Essential Evidence
4+
**Goal**: Prove correctness and demonstrate core algorithmic advantages
5+
6+
1) **Correctness Validation**
7+
- HMM: K ∈ {2, 4}, T ∈ {50, 200} vs forward-backward reference
8+
- GMM: K ∈ {2, 4}, N ∈ {100, 1000} vs analytical collapsed likelihood
9+
- Gradient verification via finite differences
10+
- *Status: Partially implemented, needs expansion*
11+
12+
2) **Scaling Demonstration**
13+
- HMM temporal ordering: measure O(T·K²) scaling
14+
- Peak frontier width profiling during evaluation
15+
- *Status: Basic benchmarks exist, needs theoretical overlay*
16+
17+
## Critical Impact Demo: Order Matters
18+
**Goal**: Show dramatic practical importance of variable ordering
19+
20+
3) **Factorial HMM Order Comparison**
21+
- C ∈ {2, 3, 4} chains, T = 100
22+
- Compare policies: interleaved (time‑first) vs min‑fill vs min‑degree (weighted by log K), with randomized tie‑breaks and a few restarts; report best of R restarts per heuristic.
23+
- Replace “worst‑case” (grouped/random) with practical heuristics; avoid pathological explosions.
24+
- Metrics:
25+
- Frontier stats: max/mean/sum width over evaluation order
26+
- Predicted DP cost proxy: Σ_t K^{w_t} (or Σ_t exp(Σ_i log K_i) for heterogeneous K)
27+
- Timing: always time interleaved; time heuristic orders only if predicted cost < threshold (frontier‑only mode otherwise). Verify equal logp on small T.
28+
- Order construction: build discrete primal graph; generate elimination order via heuristic; lift to evaluation order by placing emissions as soon as all discrete parents are placed; topo‑repair; recompute minimal keys.
29+
- *Status: Heuristic plan defined; utils in place; implement min‑fill/min‑degree + frontier‑only reporting*
30+
31+
## Theoretical Generalization: Beyond Chains
32+
**Goal**: Demonstrate algorithmic generality
33+
34+
4) **Tree Structure Validation**
35+
- Binary HMT: DFS vs BFS vs random orders
36+
- Show near-optimal frontier management
37+
- *Status: Not implemented*
38+
39+
## Nonparametric Extension: Exact Finite Truncation
40+
**Goal**: Show method works for nonparametric models with finite support
41+
42+
5) **HDP-HMM with Truncation**
43+
- Stick-breaking with K_max ∈ {5, 10, 20}
44+
- Marginalize assignments exactly under truncation
45+
- Compare against forward-backward with same truncation
46+
- Demonstrate exact gradients w.r.t. hyperparameters
47+
- *Status: Not implemented*
48+
49+
## Implementation Priorities
50+
51+
**Core Validation**
52+
1. Extend existing HMM/GMM correctness tests with more configurations
53+
2. Add theoretical complexity curve overlays to existing benchmarks
54+
3. Implement gradient verification via finite differences
55+
56+
**Order Impact**
57+
4. Implement FHMM with heuristic orders (interleaved, min‑fill, min‑degree)
58+
5. Add frontier‑only mode with predicted cost proxy and timing threshold; time interleaved by default
59+
6. Add randomized restarts for heuristics with weighted scores (log K) and select the best
60+
61+
**Generalization**
62+
7. Add basic tree model (HMT) with heuristic orders (DFS‑like, min‑fill/min‑degree on tree moralization)
63+
8. Implement HDP-HMM with truncation and exact marginalization
64+
65+
**Infrastructure**
66+
- Extend existing experiment harness for structured logging
67+
- Add frontier profiling and complexity proxy generation (Σ K^{w_t})
68+
- Add skip/timeout guards based on predicted cost; record “skipped” in logs

‎JuliaBUGS/experiments/experiments_log.md‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Auto-Marginalization Experiments Log
2+
3+
All commands are assumed to be run from the repo root with the experiments
4+
project: `julia --project=JuliaBUGS/experiments …`.
5+
6+
## 2024-09-24 — HMM marginal logp check
7+
8+
- Script: `JuliaBUGS/experiments/scripts/hmm_marginal_logp.jl`
9+
- Default config (`AM_SEED=1`, `AM_T=50`, `K=2`).
10+
- Output:
11+
- Auto-marg logp: `-44.994096`
12+
- Forward reference: `-44.994096`
13+
- Delta: `-4.97e-14`
14+
- Notes: First deterministic check; script reads env vars `AM_SEED`, `AM_T`.
15+
16+
## 2024-09-24 — HMM correctness sweep
17+
18+
- Script: `JuliaBUGS/experiments/scripts/hmm_correctness_sweep.jl`
19+
- Config: defaults (`AM_SWEEP_SEEDS=1`, `AM_SWEEP_K=2,4`, `AM_SWEEP_T=50,200`).
20+
- Outputs (seed,K,T,logp_autmarg,logp_forward,diff):
21+
- `1,2,50,-79.870504305836,-79.870504305836,5.684e-14`
22+
- `1,2,200,-278.627031832427,-278.627031832427,-4.547e-13`
23+
- `1,4,50,-71.473518232377,-71.473518232377,1.421e-14`
24+
- `1,4,200,-275.848467761660,-275.848467761659,-2.274e-13`
25+
- Notes: Confirms auto-marg matches forward algorithm across plan’s grid.
26+
27+
## 2024-09-24 — GMM correctness sweep
28+
29+
- Script: `JuliaBUGS/experiments/scripts/gmm_correctness_sweep.jl`
30+
- Config: defaults (`AG_SWEEP_SEEDS=1`, `AG_SWEEP_K=2,4`, `AG_SWEEP_N=100,1000`).
31+
- Outputs (seed,K,N,logp_autmarg,logp_closed_form,diff):
32+
- `1,2,100,-204.416329424201,-204.416329424201,-5.684e-14`
33+
- `1,2,1000,-1970.291568636987,-1970.291568637027,4.047e-11`
34+
- `1,4,100,-203.586552750729,-203.586552750729,-2.558e-13`
35+
- `1,4,1000,-1948.173739898702,-1948.173739898751,4.843e-11`
36+
- Notes: Analytic marginal and auto-marg agree up to numerical precision.
37+
38+
## 2024-09-24 — HMM gradient verification (AutoMarg target)
39+
40+
- Script: `JuliaBUGS/experiments/scripts/hmm_gradient_check.jl`
41+
- Config: defaults (`AGC_SEED=1`, `AGC_K=2`, `AGC_T=50`).
42+
- Output (excerpt):
43+
- `logp = -126.799281599876`
44+
- `θ[1]: autodiff=1.438140e+02 fd=1.438140e+02 diff=-2.41e-09`
45+
- `θ[2]: autodiff=-8.204939e+01 fd=-8.204939e+01 diff=3.40e-09`
46+
- `θ[3]: autodiff=8.965588e+00 fd=8.965588e+00 diff=6.22e-10`
47+
- `θ[4]: autodiff=4.711574e+00 fd=4.711574e+00 diff=1.37e-09`
48+
- Notes: Gradients from ForwardDiff match central finite differences to ~1e-9.
49+
50+
## 2024-09-24 — HMM gradient verification (sweep-enabled, default run)
51+
52+
- Script: `JuliaBUGS/experiments/scripts/hmm_gradient_check.jl`
53+
- Config: defaults (`AGC_SEED=1`, `AGC_K=2`, `AGC_T=50`, `AGC_EPS=1e-5`, `AGC_VERBOSE=0`).
54+
- Summary:
55+
- `logp = -80.444277269128`
56+
- `max_abs_diff = 2.422e-09`, `max_rel_diff = 8.995e-10`
57+
- Notes: Script now supports sweeps via `AGC_SWEEP_SEEDS`, `AGC_SWEEP_K`, `AGC_SWEEP_T` and per‑θ verbose output via `AGC_VERBOSE=1`.
58+
59+
## 2024-09-24 — GMM gradient verification
60+
61+
- Script: `JuliaBUGS/experiments/scripts/gmm_gradient_check.jl`
62+
- Config: defaults (`AGG_SEED=1`, `AGG_K=2`, `AGG_N=200`, `AGG_EPS=1e-5`, `AGG_VERBOSE=0`).
63+
- Summary:
64+
- `logp = -468.030155527501`
65+
- `max_abs_diff = 6.644e-08`, `max_rel_diff = 4.846e-08`
66+
- Notes: Validates gradients w.r.t. continuous parameters (`mu`, `log_sigma`) under auto‑marginalization with fixed uniform weights; sweep supported via `AGG_SWEEP_*`.
67+
68+
## 2024-09-24 — HMM scaling benchmark (auto-marginalization)
69+
70+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
71+
- Config: defaults (`AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=2,4`, `AS_SWEEP_T=50,200`, `AS_TRIALS=5`).
72+
- Output (seed,K,T,trials,mean_time_sec,logp,max_frontier,mean_frontier,sum_frontier):
73+
- `1,2,50,5,1.747571e-02,-79.870504305836,1,0.990,99`
74+
- `1,2,200,5,3.401337e-02,-278.627031832427,1,0.998,399`
75+
- `1,4,50,5,1.657132e-02,-71.473518232377,1,0.990,99`
76+
- `1,4,200,5,3.799386e-02,-275.848467761660,1,0.998,399`
77+
- Notes: Mean times grow roughly linearly in T. Frontier metrics reflect interleaved‑like minimal width (max_frontier≈1) under the model’s cached order.
78+
79+
## 2024-09-24 — HMM scaling sweep (interleaved order)
80+
81+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
82+
- Config: `AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=2,4,8`, `AS_SWEEP_T=50,100,200,400,800`, `AS_TRIALS=5`.
83+
- Output (seed,K,T,trials,mean_time_sec,logp,max_frontier,mean_frontier,sum_frontier):
84+
- `1,2,50,5,1.725919e-02,-79.870504305836,1,0.990,99`
85+
- `1,2,100,5,2.026912e-02,-145.761382159410,1,0.995,199`
86+
- `1,2,200,5,2.251455e-02,-278.627031832427,1,0.998,399`
87+
- `1,2,400,5,9.836627e-02,-528.062883321577,1,0.999,799`
88+
- `1,2,800,5,3.282191e-01,-1049.936193752744,1,0.999,1599`
89+
- `1,4,50,5,1.502275e-02,-71.473518232377,1,0.990,99`
90+
- `1,4,100,5,7.704333e-03,-137.896475262787,1,0.995,199`
91+
- `1,4,200,5,2.397348e-02,-275.848467761660,1,0.998,399`
92+
- `1,4,400,5,8.882374e-02,-525.491535975989,1,0.999,799`
93+
- `1,4,800,5,3.479876e-01,-1045.351313078216,1,0.999,1599`
94+
- `1,8,50,5,5.099008e-03,-68.367716912908,1,0.990,99`
95+
- `1,8,100,5,1.158021e-02,-134.566394931630,1,0.995,199`
96+
- `1,8,200,5,3.383068e-02,-268.884504181967,1,0.998,399`
97+
- `1,8,400,5,1.070151e-01,-507.899254715377,1,0.999,799`
98+
- `1,8,800,5,4.041233e-01,-1014.031647189991,1,0.999,1599`
99+
100+
## 2024-09-24 — HMM scaling sweep (min_time_sec, interleaved order)
101+
102+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
103+
- Config: `AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=2,4,8`, `AS_SWEEP_T=50,100,200,400,800`, `AS_TRIALS=5`.
104+
- Output (seed,K,T,trials,min_time_sec,logp,max_frontier,mean_frontier,sum_frontier):
105+
- `1,2,50,5,1.822208e-03,-79.870504305836,1,0.990,99`
106+
- `1,2,100,5,5.764750e-03,-145.761382159410,1,0.995,199`
107+
- `1,2,200,5,2.135263e-02,-278.627031832427,1,0.998,399`
108+
- `1,2,400,5,8.513233e-02,-528.062883321577,1,0.999,799`
109+
- `1,2,800,5,3.454963e-01,-1049.936193752744,1,0.999,1599`
110+
- `1,4,50,5,2.201750e-03,-71.473518232377,1,0.990,99`
111+
- `1,4,100,5,6.811584e-03,-137.896475262787,1,0.995,199`
112+
- `1,4,200,5,2.401713e-02,-275.848467761660,1,0.998,399`
113+
- `1,4,400,5,9.021662e-02,-525.491535975989,1,0.999,799`
114+
- `1,4,800,5,3.536854e-01,-1045.351313078216,1,0.999,1599`
115+
- `1,8,50,5,4.020833e-03,-68.367716912908,1,0.990,99`
116+
- `1,8,100,5,1.082317e-02,-134.566394931630,1,0.995,199`
117+
- `1,8,200,5,3.241446e-02,-268.884504181967,1,0.998,399`
118+
- `1,8,400,5,1.081037e-01,-507.899254715377,1,0.999,799`
119+
- `1,8,800,5,3.867681e-01,-1014.031647189991,1,0.999,1599`
120+
121+
## 2024-09-24 — HMM scaling (K^2 normalization, interleaved order)
122+
123+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
124+
- Config: `AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=8,16,32,64`, `AS_SWEEP_T=400`, `AS_TRIALS=20`.
125+
- Derived metric: `time_over_TK2 = mean_time_sec / (T*K^2)`.
126+
- Results:
127+
- `K=8, T=400, trials=20, mean_time_sec=1.221838e-01, time_over_TK2=4.772805e-06`
128+
- `K=16, T=400, trials=20, mean_time_sec=1.921807e-01, time_over_TK2=1.876765e-06`
129+
- `K=32, T=400, trials=20, mean_time_sec=4.844382e-01, time_over_TK2=1.182710e-06`
130+
- `K=64, T=400, trials=20, mean_time_sec=1.484085e+00, time_over_TK2=9.058136e-07`
131+
132+
## 2024-09-24 — HMM scaling (larger K, min_time_sec)
133+
134+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
135+
- Config: `AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=8,16,32,64,128`, `AS_SWEEP_T=400`, `AS_TRIALS=10`.
136+
- Results (normalized with `time_over_TK2 = min_time_sec/(T*K^2)`):
137+
- `K=8, T=400, min_time_sec=1.040547e-01, time_over_TK2=4.064637e-06`
138+
- `K=16, T=400, min_time_sec=1.634169e-01, time_over_TK2=1.595868e-06`
139+
- `K=32, T=400, min_time_sec=4.772193e-01, time_over_TK2=1.165086e-06`
140+
- `K=64, T=400, min_time_sec=1.436576e+00, time_over_TK2=8.768164e-07`
141+
- `K=128, T=400, min_time_sec=5.412097e+00, time_over_TK2=8.258205e-07`
142+
143+
## 2024-09-24 — HMM scaling (very large K, min_time_sec)
144+
145+
- Script: `JuliaBUGS/experiments/scripts/hmm_scaling_bench.jl`
146+
- Config: `AS_SWEEP_SEEDS=1`, `AS_SWEEP_K=256,512`, `AS_SWEEP_T=400`, `AS_TRIALS=5` (K=512 used 3 trials).
147+
- Results (normalized with `time_over_TK2 = min_time_sec/(T*K^2)`):
148+
- `K=256, T=400, min_time_sec=2.061049e+01, time_over_TK2=7.862278e-07`
149+
- `K=512, T=400, min_time_sec=8.741909e+01, time_over_TK2=8.336934e-07`
150+
151+
## 2025-09-24 — FHMM order comparison (tiny T; consistent orders)
152+
153+
- Script: `JuliaBUGS/experiments/scripts/fhmm_order_comparison.jl`
154+
- Config: `AFH_MODE=timed`, `AFH_C=2`, `AFH_K=2`, `AFH_TRIALS=20`.
155+
- Orders: `interleaved` (good), `states_then_y` (all z’s then all y; bad).
156+
157+
Results (order,max_frontier,mean_frontier,sum_frontier,log_cost_proxy,min_time_sec,logp):
158+
159+
- T=5
160+
- `states_then_y,10,5.200,130,8.397e+00,9.741708e-03,-11.605424781297`
161+
- `interleaved,2,1.880,47,4.554e+00,2.232500e-04,-11.605424781297`
162+
163+
- T=8
164+
- `states_then_y,16,8.200,328,1.256e+01,1.142920e+00,-21.198978101373`
165+
- `interleaved,2,1.925,77,5.043e+00,4.742500e-04,-21.198978101373`
166+
167+
- T=10
168+
- `states_then_y,20,10.200,510,1.533e+01,2.566664e+01,-23.230717049023`
169+
- `interleaved,2,1.940,97,5.273e+00,5.868750e-04,-23.230717049023`
170+
171+
Notes:
172+
- Both orders agree on logp for the same data; differences are purely computational.
173+
- The frontier lens explains tractability: interleaved keeps max_frontier≈C, while
174+
states_then_y grows roughly with T, making Σ K^{width} explode quickly.
175+
- New env var `AFH_ORDERS` allows selecting orders to run, e.g. `AFH_ORDERS=interleaved`
176+
or `AFH_ORDERS=states_then_y`.
177+
178+
179+
## 2025-09-24 — HDP-HMM correctness (finite truncation, sticky optional)
180+
181+
- Script: `JuliaBUGS/experiments/scripts/hdphmm_correctness_fixed.jl`
182+
- Config: `AHDPC_SEEDS=1`, `AHDPC_K=5,10`, `AHDPC_T=100,200`, `AHDPC_ALPHA=5.0`, `AHDPC_GAMMA=1.0`, `AHDPC_KAPPA=0.0`.
183+
- Output (seed,K,T,alpha,gamma,logp_autmarg,logp_forward,diff):
184+
- `1,5,100,5.000,1.000,-132.648742618434,-132.648742618434,0.000e+00`
185+
- `1,5,200,5.000,1.000,-263.448999241194,-263.448999241194,1.137e-13`
186+
- `1,10,100,5.000,1.000,-94.917105497644,-94.917105497644,-7.105e-14`
187+
- `1,10,200,5.000,1.000,-175.671119347952,-175.671119347951,-9.095e-13`
188+
- Notes: Auto-marginalized log p(y|β,π,μ,σ) matches forward reference up to numerical precision. Sticky variant supported via `AHDPC_KAPPA` (default 0.0).
189+
190+
## 2025-09-24 — HDP-HMM gradient verification
191+
192+
- Script: `JuliaBUGS/experiments/scripts/hdphmm_gradient_check.jl`
193+
- Config: `AHDPG_SEED=1`, `AHDPG_K=5`, `AHDPG_T=200`, `AHDPG_ALPHA=5.0`, `AHDPG_GAMMA=1.0`, `AHDPG_EPS=5e-6`.
194+
- Summary:
195+
- `logp = -403.142155296914`
196+
- `max_abs_diff = 4.926e-08`, `max_rel_diff = 1.388e-03`
197+
- Notes: Gradients validated for μ, log σ, stick v, and π rows under Dirichlet(α·β). Sticky κ is used in the correctness script; gradient script uses non‑sticky rows to satisfy BUGS single‑assignment constraints.
198+
199+
## 2025-09-24 — HDP-HMM gradient verification (sticky)
200+
201+
- Script: `JuliaBUGS/experiments/scripts/hdphmm_gradient_check.jl`
202+
- Config: `AHDPG_SEED=1`, `AHDPG_K=5`, `AHDPG_T=120`, `AHDPG_ALPHA=5.0`, `AHDPG_GAMMA=1.0`, `AHDPG_KAPPA=5.0`, `AHDPG_EPS=5e-6`.
203+
- Summary:
204+
- `logp = -236.450382897373`
205+
- `max_abs_diff = 9.347e-08`, `max_rel_diff = 1.402e-03`
206+
- Notes: Implemented sticky prior in-gradient via deterministic `diagshift` primitive to keep single-assignment and AD friendliness; validates end‑to‑end gradients with z marginalized.

0 commit comments

Comments
 (0)