Skip to content

Commit 580b70d

Browse files
vyuduvyudu
committed
split benchmarks
1 parent 9453fdc commit 580b70d

File tree

5 files changed

+222
-123
lines changed

5 files changed

+222
-123
lines changed

‎benchmarks/Jumps/AggregatorBenchmark.jmd‎

Lines changed: 0 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -31,62 +31,6 @@ plot!(plt)
3131
These results seem pretty reasonable - it seems like we're getting the same dimer
3232
concentration curve for each method.
3333

34-
Let's also go ahead and benchmark the performance of JumpProcess solvers on some other large
35-
biochemical reaction network models. We first define a function to plot the results.
36-
37-
```julia
38-
function benchmark_and_bar_plot(models, model_names, end_times, algs)
39-
timedict = Dict(["$modelname" => Float64[] for modelname in model_names])
40-
alg_names = ["$s"[1:end-2] for s in algs]
41-
42-
for (end_time, model, model_name) in zip(end_times, models, model_names)
43-
# benchmarking and saving
44-
benchmarks = Vector{BenchmarkTools.Trial}(undef, length(algs))
45-
46-
# callback for terminating simulations
47-
for (i, alg) in enumerate(algs)
48-
alg_name = alg_names[i]
49-
println("benchmarking $alg_name, $model_name")
50-
dprob = DiscreteProblem(complete(model.rn), model.u0, (0., end_time), model.p)
51-
dprob = remake(dprob,u0=Int64.(dprob.u0))
52-
jprob = JumpProblem(complete(model.rn), dprob, alg; rng)
53-
54-
b = @benchmarkable solve($jprob, SSAStepper(); saveat = $end_time) samples = 3 seconds = 3600
55-
bm = run(b)
56-
push!(timedict["$model_name"], median(bm).time/1e9)
57-
end
58-
end
59-
times = vcat([timedict["$model_name"] for model_name in model_names]...)
60-
println(times)
61-
n = length(model_names); m = length(alg_names)
62-
println(n, m)
63-
groupedbar(repeat(model_names, inner = m), times, group = repeat(alg_names, n), xlabel = "Reaction Network", ylabel = "Average Time", title = "SSA Runtime for standard reaction networks")
64-
end
65-
```
66-
67-
Below we create the other models to benchmark and plot the results.
68-
```julia
69-
BCR_net = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/BCR.net"));
70-
fceri_gamma2 = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/fceri_gamma2.net"));
71-
multistate = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/multistate.net"));
72-
multisite2 = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/multisite2.net"));
73-
models = [egfr_net
74-
#,multistate
75-
#,multisite2
76-
,BCR_net
77-
,fceri_gamma2
78-
];
79-
80-
model_names = ["egfr"
81-
#,"multistate"
82-
#,"multisite2"
83-
,"BCR_net"
84-
,"fceri_gamma2"
85-
]
86-
end_times = [10., .1, 10.]#, 100.]
87-
88-
benchmark_and_bar_plot(models, model_names, end_times, algs)
89-
```
9034

9135
# Performance Benchmark (Sanft 2015)
9236
For our performance benchmark test, we will look at a very simple reaction network.

‎benchmarks/Jumps/BCR_Benchmark.jmd‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: BCR Network Benchmark
3+
author:
4+
---
5+
6+
```julia
7+
using JumpProcesses, Plots, StableRNGs, Random, BenchmarkTools, ReactionNetworkImporters, StatsPlots, Catalyst
8+
```
9+
10+
# Model Benchmark
11+
12+
We define a function to benchmark the model and then plot the results in a benchmark.
13+
```julia
14+
function benchmark_and_bar_plot(model, end_time, algs)
15+
times = Vector{Float64}()
16+
alg_names = ["$s"[1:end-2] for s in algs]
17+
18+
benchmarks = Vector{BenchmarkTools.Trial}(undef, length(algs))
19+
for (i, alg) in enumerate(algs)
20+
alg_name = alg_names[i]
21+
println("benchmarking $alg_name")
22+
dprob = DiscreteProblem(complete(model.rn), model.u0, (0., end_time), model.p)
23+
dprob = remake(dprob,u0 = Int64.(dprob.u0))
24+
jprob = JumpProblem(complete(model.rn), dprob, alg; rng, save_positions = (false, false))
25+
26+
b = @benchmarkable solve($jprob, SSAStepper(); saveat = $end_time) samples = 3 seconds = 3600
27+
bm = run(b)
28+
push!(times, median(bm).time/2e9)
29+
end
30+
31+
bar(alg_names, times, xlabel = "Algorithm", ylabel = "Average Time", title = "SSA Runtime for BCR network", legend = false)
32+
end
33+
```
34+
35+
Now we benchmark the BCR network on the four algorithms.
36+
```julia
37+
tf = 10000.
38+
rng = StableRNG(53124)
39+
algs = [NRM(), CCNRM(), DirectCR(), RSSACR()]
40+
BCR_net = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/BCR.net"));
41+
42+
benchmark_and_bar_plot(BCR_net, tf, algs)
43+
```
44+

‎benchmarks/Jumps/EGFR_Benchmark.jmd‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: EGFR Network Benchmark
3+
author:
4+
---
5+
6+
```julia
7+
using JumpProcesses, Plots, StableRNGs, Random, BenchmarkTools, ReactionNetworkImporters, StatsPlots, Catalyst
8+
```
9+
10+
# Model Benchmark
11+
12+
We define a function to benchmark the model and then plot the results in a benchmark.
13+
```julia
14+
function benchmark_and_bar_plot(model, end_time, algs)
15+
times = Vector{Float64}()
16+
alg_names = ["$s"[1:end-2] for s in algs]
17+
18+
benchmarks = Vector{BenchmarkTools.Trial}(undef, length(algs))
19+
for (i, alg) in enumerate(algs)
20+
alg_name = alg_names[i]
21+
println("benchmarking $alg_name")
22+
dprob = DiscreteProblem(complete(model.rn), model.u0, (0., end_time), model.p)
23+
dprob = remake(dprob,u0 = Int64.(dprob.u0))
24+
jprob = JumpProblem(complete(model.rn), dprob, alg; rng, save_positions = (false, false))
25+
26+
b = @benchmarkable solve($jprob, SSAStepper(); saveat = $end_time) samples = 3 seconds = 3600
27+
bm = run(b)
28+
push!(times, median(bm).time/2e9)
29+
end
30+
31+
bar(alg_names, times, xlabel = "Algorithm", ylabel = "Average Time", title = "SSA Runtime for EGFR network", legend = false)
32+
end
33+
```
34+
35+
Now we benchmark the EGFR network on the four algorithms.
36+
```julia
37+
tf = 10.
38+
rng = StableRNG(53124)
39+
algs = [NRM(), CCNRM(), DirectCR(), RSSACR()]
40+
EGFR_net = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/egfr_net.net"));
41+
42+
benchmark_and_bar_plot(EGFR_net, tf, algs)
43+
```
44+

‎benchmarks/Jumps/Fceri_gamma2_Benchmark.jmd‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: Fc gamma Network Benchmark
3+
author:
4+
---
5+
6+
```julia
7+
using JumpProcesses, Plots, StableRNGs, Random, BenchmarkTools, ReactionNetworkImporters, StatsPlots, Catalyst
8+
```
9+
10+
# Model Benchmark
11+
12+
We define a function to benchmark the model and then plot the results in a benchmark.
13+
```julia
14+
function benchmark_and_bar_plot(model, end_time, algs)
15+
times = Vector{Float64}()
16+
alg_names = ["$s"[1:end-2] for s in algs]
17+
18+
benchmarks = Vector{BenchmarkTools.Trial}(undef, length(algs))
19+
for (i, alg) in enumerate(algs)
20+
alg_name = alg_names[i]
21+
println("benchmarking $alg_name")
22+
dprob = DiscreteProblem(complete(model.rn), model.u0, (0., end_time), model.p)
23+
dprob = remake(dprob,u0 = Int64.(dprob.u0))
24+
jprob = JumpProblem(complete(model.rn), dprob, alg; rng, save_positions = (false, false))
25+
26+
b = @benchmarkable solve($jprob, SSAStepper(); saveat = $end_time) samples = 3 seconds = 3600
27+
bm = run(b)
28+
push!(times, median(bm).time/2e9)
29+
end
30+
31+
bar(alg_names, times, xlabel = "Algorithm", ylabel = "Average Time", title = "SSA Runtime for Fc gamma network", legend = false)
32+
end
33+
```
34+
35+
Now we benchmark the Fc gamma network on the four algorithms.
36+
```julia
37+
tf = 150.
38+
rng = StableRNG(53124)
39+
algs = [NRM(), CCNRM(), DirectCR(), RSSACR()]
40+
fceri_gamma2_net = loadrxnetwork(BNGNetwork(), joinpath(@__DIR__, "Data/fceri_gamma2.net"));
41+
42+
benchmark_and_bar_plot(fceri_gamma2_net, tf, algs)
43+
```
44+

0 commit comments

Comments
 (0)