Skip to content

Commit 056ff33

Browse files
committed
Merge branch 'main' into py/dppl-models
2 parents 5fa095a + 2bf0fb4 commit 056ff33

File tree

7 files changed

+151
-36
lines changed

7 files changed

+151
-36
lines changed

Project.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,5 @@ Turing = "fce5fe82-541a-59a6-adf8-730c64b5f9a0"
2323
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
2424

2525
[compat]
26-
DynamicPPL = "0.36"
26+
DynamicPPL = "0.37"
27+
Turing = "0.40"

README.md

Lines changed: 62 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -12,31 +12,75 @@ You can modify the list of AD types in `main.jl`.
1212

1313
## I want to add more models!
1414

15-
You can modify the list of models by adding a new file inside the `models` directory.
15+
You can modify the list of models by:
1616

17-
Inside this file, you do not need to call `using Turing` or any of the AD backends.
18-
However, you will have to make sure to import any other packages that your model uses.
17+
1. **Adding a new call to `@include_model {category_heading} {model_name}` in `main.jl`.**
18+
Both `category_heading` and `model_name` should be strings.
1919

20-
This file should have, as the final line, the creation of the Turing model object using `model = model_f(...)`.
21-
(It is mandatory for the model object to be called `model`.)
20+
`category_heading` is used to determine which table the model appears under on the website.
21+
This should be self-explanatory if you look at the [current website](https://turinglang.org/ADTests).
22+
23+
2. **Adding a new file, `models/{model_name}.jl`.**
24+
25+
The basic structure of this file should look like this, where `model_name` is replaced accordingly:
2226

23-
Then, inside `main.jl`, call `@include_model category_heading model_name`.
27+
```julia
28+
#=
29+
(1) You can add any explanatory comments here if necessary
30+
=#
2431

25-
- `category_heading` is a string that is used to determine which table the model appears under on the website.
26-
- For the automated tests to run properly, `model_name` **must** be consistent between the following:
27-
- The name of the model itself i.e. `@model function model_name(...)`
28-
- The filename i.e. `models/model_name.jl`
29-
- The name of the model in `main.jl` i.e. `@include_model "Category Heading" model_name`
32+
# (2) Imports if necessary
33+
using MyOtherPackage: some_function
3034

31-
Ideally, `model_name` would be self-explanatory, i.e. it would serve to illustrate exactly one feature and the name would indicate this.
32-
However, if necessary, you can add explanatory comments inside the model definition file.
35+
# (3) Define data if necessary
36+
data = ...
3337

34-
You can see the existing files in that directory for examples.
38+
# (4) Define the model
39+
@model function model_name(data, ...)
40+
# Define your model here
41+
...
42+
end
3543

36-
> [!NOTE]
37-
> This setup does admittedly feel a bit complicated.
38-
> Unfortunately I could not find a simpler way to get all the components (Julia, Python, web app) to work together in an automated fashion.
39-
> Hopefully it is a small price to pay for the ability to just add a new model and have it be automatically included on the website.
44+
# (5) Instantiate the model
45+
model = model_name(data, ...)
46+
```
47+
48+
**(1) Description**
49+
50+
Ideally, `model_name` would be self-explanatory, i.e. it would serve to illustrate exactly one feature and the name would indicate this.
51+
However, if necessary, you can add further explanatory comments inside the model definition file.
52+
53+
**(2) Dependencies**
54+
55+
Inside this file, you do not need to call `using Turing` or any of the AD backends.
56+
(This also means you do not need to import anything that Turing re-exports, such as distributions.)
57+
58+
However, you will have to make sure to import any other packages that your model requires.
59+
(If this package is not already present in the project environment, you will also have to add it to `Project.toml`.)
60+
61+
**(3) Data definition**
62+
63+
Each file in `models/` is evaluated within its own module, so you can declare data variables, etc. without worrying about name clashes.
64+
65+
**(4) Model definition**
66+
67+
Models can be defined as usual with `@model function model_name(...)`.
68+
69+
**(5) Model instantiation**
70+
71+
The last line in the file should be the creation of the Turing model object using `model = model_name(...)`.
72+
(It is mandatory for the model object to be called `model`.)
73+
74+
> [!IMPORTANT]
75+
> Note that for CI to run properly, `model_name` **must** be consistent between the following:
76+
>
77+
> - The name of the model itself i.e. `@model function model_name(...)`
78+
> - The filename i.e. `models/model_name.jl`
79+
> - The name of the model in `main.jl` i.e. `@include_model "Category Heading" "model_name"`
80+
81+
(This setup does admittedly feel a bit fragile.
82+
Unfortunately I could not find a simpler way to get all the components (Julia, Python, web app) to work together in an automated fashion.
83+
Hopefully it is a small price to pay for the ability to just add a new model and have it be automatically included on the website.)
4084

4185
## I want to edit the website!
4286

main.jl

Lines changed: 21 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
using DynamicPPL: DynamicPPL, VarInfo
2-
using DynamicPPL.TestUtils.AD: run_ad, ADResult, ADIncorrectException
2+
using DynamicPPL.TestUtils.AD: run_ad, ADResult, ADIncorrectException, WithBackend
33
using ADTypes
4+
using Random: Xoshiro
45

56
import FiniteDifferences: central_fdm
67
import ForwardDiff
@@ -15,7 +16,8 @@ ADTYPES = Dict(
1516
"ForwardDiff" => AutoForwardDiff(),
1617
"ReverseDiff" => AutoReverseDiff(; compile=false),
1718
"ReverseDiffCompiled" => AutoReverseDiff(; compile=true),
18-
"Mooncake" => AutoMooncake(; config=nothing),
19+
"MooncakeReverse" => AutoMooncake(),
20+
"MooncakeForward" => AutoMooncakeForward(),
1921
"EnzymeForward" => AutoEnzyme(; mode=set_runtime_activity(Forward, true)),
2022
"EnzymeReverse" => AutoEnzyme(; mode=set_runtime_activity(Reverse, true)),
2123
"Zygote" => AutoZygote(),
@@ -56,14 +58,12 @@ macro include_model(category::AbstractString, model_name::AbstractString)
5658
if MODELS_TO_LOAD == "__all__" || model_name in split(MODELS_TO_LOAD, ",")
5759
# Declare a module containing the model. In principle esc() shouldn't
5860
# be needed, but see https://github.com/JuliaLang/julia/issues/55677
59-
Expr(:toplevel, esc(:(
60-
module $(gensym())
61-
using .Main: @register
62-
using Turing
63-
include("models/" * $(model_name) * ".jl")
64-
@register $(category) model
65-
end
66-
)))
61+
Expr(:toplevel, esc(:(module $(gensym())
62+
using .Main: @register
63+
using Turing
64+
include("models/" * $(model_name) * ".jl")
65+
@register $(category) model
66+
end)))
6767
else
6868
# Empty expression
6969
:()
@@ -76,6 +76,7 @@ end
7676
# although it's hardly a big deal.
7777
@include_model "Base Julia features" "control_flow"
7878
@include_model "Base Julia features" "multithreaded"
79+
@include_model "Base Julia features" "call_C"
7980
@include_model "Core Turing syntax" "broadcast_macro"
8081
@include_model "Core Turing syntax" "dot_assume"
8182
@include_model "Core Turing syntax" "dot_observe"
@@ -122,6 +123,7 @@ end
122123
@include_model "Effect of model size" "n500"
123124
@include_model "PosteriorDB" "pdb_eight_schools_centered"
124125
@include_model "PosteriorDB" "pdb_eight_schools_noncentered"
126+
@include_model "Miscellaneous features" "metabayesian_MH"
125127

126128
# The entry point to this script itself begins here
127129
if ARGS == ["--list-model-keys"]
@@ -139,12 +141,18 @@ elseif length(ARGS) == 3 && ARGS[1] == "--run"
139141
# https://github.com/TuringLang/ADTests/issues/4
140142
vi = DynamicPPL.unflatten(VarInfo(model), [0.5, -0.5])
141143
params = [-0.5, 0.5]
142-
result = run_ad(model, adtype; varinfo=vi, params=params, benchmark=true)
144+
result = run_ad(model, adtype; varinfo=vi, params=params, test=WithBackend(ADTYPES["FiniteDifferences"]), benchmark=true)
143145
else
144-
result = run_ad(model, adtype; benchmark=true)
146+
result = run_ad(
147+
model,
148+
adtype;
149+
rng=Xoshiro(468),
150+
test=WithBackend(ADTYPES["FiniteDifferences"]),
151+
benchmark=true,
152+
)
145153
end
146154
# If reached here - nothing went wrong
147-
println(result.time_vs_primal)
155+
println(result.grad_time / result.primal_time)
148156
catch e
149157
@show e
150158
if e isa ADIncorrectException

models/broadcast_macro.jl

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
1-
@model function broadcast_macro(
2-
x = [1.5, 2.0],
3-
::Type{TV} = Vector{Float64},
4-
) where {TV}
1+
@model function broadcast_macro(x = [1.5, 2.0], ::Type{TV} = Vector{Float64}) where {TV}
52
a ~ Normal(0, 1)
63
b ~ InverseGamma(2, 3)
74
@. x ~ Normal(a, $(sqrt(b)))

models/call_C.jl

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
@model function call_C(y = 0.0)
2+
x ~ Normal(0, 1)
3+
4+
# Call C library abs function
5+
x_abs = @ccall fabs(x::Cdouble)::Cdouble
6+
7+
y ~ Normal(0, x_abs)
8+
end
9+
10+
model = call_C()

models/metabayesian_MH.jl

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#=
2+
This is a "meta-Bayesian" model, where the generative model includes an inversion of a different generative model.
3+
These types of models are common in cognitive modelling, where systems of interest (e.g. human subjects) are thought to use Bayesian inference to navigate their environment.
4+
Here we use a Metropolis-Hasting sampler implemented with Turing as the inversion of the inner "subjective" model.
5+
=#
6+
using Random: Xoshiro
7+
8+
# Inner model function
9+
@model function inner_model(observation, prior_μ = 0, prior_σ = 1)
10+
# The inner model's prior
11+
mean ~ Normal(prior_μ, prior_σ)
12+
# The inner model's likelihood
13+
observation ~ Normal(mean, 1)
14+
end
15+
16+
# Outer model function
17+
@model function metabayesian_MH(
18+
observation,
19+
action,
20+
inner_sampler = MH(),
21+
inner_n_samples = 20,
22+
)
23+
### Sample parameters for the inner inference and response ###
24+
# The inner model's prior's sufficient statistics
25+
subj_prior_μ ~ Normal(0, 1)
26+
subj_prior_σ = 1.0
27+
# Inverse temperature for actions
28+
β ~ Exponential(1)
29+
30+
### "Perceptual inference": running the inner model ###
31+
# Condition the inner model
32+
inner_m = inner_model(observation, subj_prior_μ, subj_prior_σ)
33+
# Run the inner Bayesian inference
34+
chns = sample(Xoshiro(468), inner_m, inner_sampler, inner_n_samples, progress = false)
35+
# Extract subjective point estimate
36+
subj_mean_expectationₜ = mean(chns[:mean])
37+
38+
39+
### "Response model": picking an action ###
40+
# The action is a Gaussian-noise report of the subjective point estimate
41+
action ~ Normal(subj_mean_expectationₜ, β)
42+
end
43+
44+
model = metabayesian_MH(0.0, 1.0)

web/src/App.svelte

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,17 @@
102102
>Download the raw data (JSON)</a
103103
>
104104
</p>
105+
106+
<p>
107+
<b>Note about Enzyme:</b> Enzyme does not work with DynamicPPL 0.37
108+
/ Turing 0.40 because of
109+
<a
110+
href="https://github.com/EnzymeAD/Enzyme.jl/issues/2429"
111+
target="_blank">this issue</a
112+
>. If you want to use Enzyme with Turing, please use an older
113+
version of Turing / DynamicPPL.
114+
</p>
115+
105116
{#each categorisedData.entries() as [category, modelData]}
106117
<h3>{category}</h3>
107118
<ResultsTable data={modelData} {modelDefinitions} />

0 commit comments

Comments
 (0)