|
2 | 2 | title: ❌ Number of replications |
3 | 3 | --- |
4 | 4 |
|
5 | | -<!--separate seeds for each replication--> |
| 5 | +{{< include ../../scripts/_reticulate-setup.md >}} |
| 6 | + |
| 7 | +:::: {.pale-blue} |
| 8 | + |
| 9 | +**Learning objectives:** |
| 10 | + |
| 11 | +* TODO |
| 12 | + |
| 13 | +**Pre-reading:** |
| 14 | + |
| 15 | +This page continues on from: [Replications](outputs.qmd). |
| 16 | + |
| 17 | +> *Entity generation → Entity processing → Initialisation bias → Performance measures → Replications → Number of replications* |
| 18 | +
|
| 19 | +**Required packages:** |
| 20 | + |
| 21 | +These should be available from environment setup in the "🧪 Test yourself" section of [Environments](../setup/environment.qmd). |
| 22 | + |
| 23 | +::: {.python-content} |
| 24 | + |
| 25 | +```{python} |
| 26 | +from IPython.display import HTML |
| 27 | +from itables import to_html_datatable |
| 28 | +import numpy as np |
| 29 | +import pandas as pd |
| 30 | +import scipy.stats as st |
| 31 | +import simpy |
| 32 | +from sim_tools.distributions import Exponential |
| 33 | +``` |
| 34 | + |
| 35 | +::: |
| 36 | + |
| 37 | +::: {.r-content} |
| 38 | + |
| 39 | +```{r} |
| 40 | +#| output: false |
| 41 | +library(dplyr) |
| 42 | +library(kableExtra) |
| 43 | +library(knitr) |
| 44 | +library(simmer) |
| 45 | +library(tidyr) |
| 46 | +``` |
| 47 | + |
| 48 | +::: |
| 49 | + |
| 50 | +:::: |
| 51 | + |
| 52 | +## Confidence interval method |
| 53 | + |
| 54 | +The **confidence interval method** can be used to select the number of replications to run. The more replications you run, the narrower your confidence interval becomes, leading to a more precise estimate of the model's mean performance. |
| 55 | + |
| 56 | +First, you select a **desired confidence interval** - for example, 95%. Then, run the model with an **increasing number of replications**, and identify the number required to achieve that precision in the estimate of a given metric - and also, to maintain that precision (as the intervals may converge or expand again later on). |
| 57 | + |
| 58 | +When selecting the number of replications you should **repeat the analysis for all performance measures** and select the highest value as your number of replications. |
| 59 | + |
| 60 | +We'll show two ways of implementing this: **manually** and **automated**. |
| 61 | + |
| 62 | +The model being used is that from the [Replications](replications.qmd) page. |
| 63 | + |
| 64 | +```{python} |
| 65 | +#| echo: false |
| 66 | +{{< include replications_resources/parameters.py >}} |
| 67 | +{{< include replications_resources/patient.py >}} |
| 68 | +{{< include replications_resources/model.py >}} |
| 69 | +{{< include replications_resources/summary_stats.py >}} |
| 70 | +{{< include replications_resources/runner.py >}} |
| 71 | +``` |
| 72 | + |
| 73 | +```{r} |
| 74 | +#| echo: false |
| 75 | +{{< include replications_resources/create_params.R >}} |
| 76 | +{{< include replications_resources/filter_warmup.R >}} |
| 77 | +{{< include replications_resources/calc_arrivals.R >}} |
| 78 | +{{< include replications_resources/calc_mean_wait.R >}} |
| 79 | +{{< include replications_resources/get_run_results.R >}} |
| 80 | +{{< include replications_resources/model.R >}} |
| 81 | +{{< include replications_resources/runner.R >}} |
| 82 | +``` |
| 83 | + |
| 84 | +## Manual inspection |
| 85 | + |
| 86 | +<!--TODO: Import from sim-tools...waiting for Tom's update...--> |
| 87 | + |
| 88 | +## Algorithm |
| 89 | + |
| 90 | +## 🔍 Explore the example models |
| 91 | + |
| 92 | +<div class="h3-tight"></div> |
| 93 | + |
| 94 | +### 🩺 Nurse visit simulation |
| 95 | + |
| 96 | +<!--TODO: Mention sensitivity analysis--> |
| 97 | + |
| 98 | +:::: {.python-content} |
| 99 | + |
| 100 | +{{< include ../../html/pydesrapmms.html >}} |
| 101 | + |
| 102 | +::: {.blue-table} |
| 103 | +| | | |
| 104 | +| - | - | |
| 105 | +| **Key files** | | |
| 106 | +| **What to look for?** | | |
| 107 | +: {tbl-colwidths="[20,80]"} |
| 108 | +::: |
| 109 | + |
| 110 | +:::: |
| 111 | + |
| 112 | +:::: {.r-content} |
| 113 | + |
| 114 | +{{< include ../../html/rdesrapmms.html >}} |
| 115 | + |
| 116 | +::: {.blue-table} |
| 117 | +| | | |
| 118 | +| - | - | |
| 119 | +| **Key files** | | |
| 120 | +| **What to look for?** | | |
| 121 | +: {tbl-colwidths="[20,80]"} |
| 122 | +::: |
| 123 | + |
| 124 | +:::: |
| 125 | + |
| 126 | +### 🧠 Stroke pathway simulation |
| 127 | + |
| 128 | +:::: {.python-content} |
| 129 | + |
| 130 | +{{< include ../../html/pydesrapstroke.html >}} |
| 131 | + |
| 132 | +Not relevant - replicating an existing model described in a paper, so just used the number of replications stated by the authors. |
| 133 | + |
| 134 | +:::: |
| 135 | + |
| 136 | +:::: {.r-content} |
| 137 | + |
| 138 | +{{< include ../../html/rdesrapstroke.html >}} |
| 139 | + |
| 140 | +Not relevant - replicating an existing model described in a paper, so just used the number of replications stated by the authors. |
| 141 | + |
| 142 | +:::: |
| 143 | + |
| 144 | +## 🧪 Test yourself |
| 145 | + |
| 146 | +TODO |
6 | 147 |
|
7 | 148 | <br><br> |
0 commit comments