update README and add real-time report

Author: sbfnk

README.md

@@ -1,65 +1,59 @@
 # Estimating epidemiological quantities from prevalence and antibody estimates in the ONS Community Infection Survey
 
-We use a semi-mechanistic method to estimating incidence from Office for National Statistics (ONS) prevalence and antibody positivity estimates at the national, and subnational levels, as well as across age groups.
+In this work, we use a semi-mechanistic method to estimating epidemiological quantities such as infection incidence, infection growth rates, reproduction numbers and immunological parameters from Office for National Statistics (ONS) prevalence and antibody positivity estimates at the national, and subnational levels, as well as across age groups.
+Our approach assumes that unobserved infections can be represented using an initial intercept and a Gaussian process with a logit link function.
+To estimate population prevalence we convolve the PCR detection curve estimated in [Hellewell _et al._, _BMC Medicine_, 2021](https://doi.org/10.1186/s12916-021-01982-x), with uncertainty assumed to be normal and independent for each day since infection
 
-# Citation
-
-# Summary
-
-## Method
-
-Our approach assumes that unobserved infections can be represented using an initial intercept and a Gaussian process with a logit link function. We use a Matern 3/2 kernal and an approximate Hilbert space Gaussian process formulation to reduce the computational cost. To estimate population prevalence we convolve the PCR detection curve estimated in Hellewell _et al._, _BMC Medicine_, 2021, with uncertainty assumed to be normal and independent for each day since infection, with our estimated infection curve. To map this to estimated prevalence from the ONS we assume a normal observation model with the standard error made up of the ONS estimated standard error and a shared standard error term estimated in the model. 
-
-We model antibody postitivity by fitting an initial proportion of the population that have infection derived antibodies. We then fit a daily model that assumes that some fraction of new infections that are not already antibody postive (here we assume equivalent infection risk for individuals who are antibody positive and negative) become so and that a proportion of those current antibody postive become antibody negative. We include vaccination similarly and assume that some fraction of those vaccination become antibody positive and that this positivity wanes with a daily rate (independent from the waning rate of those antibody positive from infection). This estimate of population level antibody positivity is then averaged across the time windows of the available antibody positivity estimates with again a normal observation model being assumed with the standard error made up of the ONS estimated standard error and a shared standard error term estimated in the model. 
+We model antibody postitivity by fitting an initial proportion of the population that have infection derived antibodies.
+We then fit a daily model that assumes that some fraction of new infections that are not already antibody postive become so and that a proportion of those current antibody postive become antibody negative. 
+We include vaccination similarly and assume that some fraction of those vaccination become antibody positive and that this positivity wanes with a daily rate (independent from the waning rate of those antibody positive from infection). 
+This estimate of population level antibody positivity is then averaged across the time windows of the available antibody positivity estimates with again a normal observation model being assumed with the standard error made up of the ONS estimated standard error and a shared standard error term estimated in the model. 
 
-This model can be described mathematically as follows:
+The model is implemented in `stan` using `cmdstanr`.
 
+# Citation
 
-## Implementation
-
-The model is implemented in `stan` using `cmdstanr` with the maximum treedepth increased to 12 from the default of 10. 
-
-## Limitations
-
- - Assumes that the probability of detection follows the Hellewell et al estimates and that testing of survey participants is happening each day, which is unlikely, but for which there is little public information.
+S. Abbott, S. Funk. _Estimating epidemiological quantities from repeated cross-sectional prevalence measurements_ (2022). https://github.com/epiforecasts/inc2prev
 
- - Assumes that uncertainty in the Hellewell et al estimates is independent normal which is known not to be the case. This limitation is imposed by not implementing the parameteric Hellewell et al model though this could in principle be done. However, this would again assume some level of independence in parameters and so still not return the posterior distribution found by Hellewell at al.
+**Draft paper in progress: [as html](https://epiforecasts.io/inc2prev/paper), [or pdf](https://epiforecasts.io/inc2prev/paper.pdf)**
 
-- Assumes that infections can be well modelled by a Gaussian process with a Matern 3/2 kernal. This may not be the case for a range of reasons such as variation over time is non-stationary, and variation is piecewise constant.
+# Real-time estimates
 
-- Real-time estimates may be unreliable as a zero mean Guassian process has been used. Alternative approaches exist to account for this but each of these imposes a parameteric assumption. Further work is needed on this area.
+The latest estimates are available in a [real-time report](https://epiforecasts.io/inc2prev/report).
 
-- Assumes that antibody waning, from both infection and vaccination, wanes with an exponential rate. 
+The code in this repository can be used to [reproduce the results](https://github.com/epiforecasts/inc2prev/blob/master/scripts/estimate.R),
+and [create the figures](https://github.com/epiforecasts/inc2prev/blob/master/scripts/plot_estimates.R)
+Estimates are also available as [data tables](https://github.com/epiforecasts/inc2prev/blob/master/outputs/) (labelled estimates_{level}.csv).
 
-# Estimates from England
+# Example estimates from England
 
 We ONS estimates for prevalence and antibody positivity in England to estimate infections and transmission parameters. The code to reproduce these results can be found [here](https://github.com/epiforecasts/inc2prev/blob/master/scripts/simple-example.R)
 
-![](figures/readme/prev.png)
+![](figures/example/prev.png)
 
 *Figure 1: ONS prevalence estimates compared to model estimates of ONS prevalence combined with model estimates of population prevalence.*
 
-![](figures/readme/ab.png)
+![](figures/example/ab.png)
 
-*Figure 1: ONS antibody positivity estimates compared to model estimates of ONS antibody positivity combined with model estimates of population antibody positivity.*
+*Figure 2: ONS antibody positivity estimates compared to model estimates of ONS antibody positivity combined with model estimates of population antibody positivity.*
 
-![](figures/readme/infections.png)
+![](figures/example/infections.png)
 
-*Figure 2: Model infection estimates*
+*Figure 3: Daily infection incidence estimates*
 
-![](figures/readme/growth.png)
+![](figures/example/growth.png)
 
-*Figure 3: Model infection growth rate estimates*
+*Figure 4: Infection growth rate estimates*
 
-![](figures/readme/Rt.png)
+![](figures/example/Rt.png)
 
-*Figure 4: Model effective reproduction rate rate estimates*
+*Figure 5: Effective reproduction rate rate estimates*
 
 ## Fit diagnostics
 
-![](figures/readme/pairs.png)
+![](figures/example/pairs.png)
 
-*Figure 5: Pairs plot of a sample of parameter posteriors*
+*Figure 6: Pairs plot of a sample of parameter posteriors*
 
 # Documentation
 
@@ -74,6 +68,31 @@ Folder | Purpose
 [`stan`](stan/) | The backend stan model code and support functions.
 [`.devcontainer`](.devcontainer/) | Contains the projects `Dockerfile` and setup instructions for using the code with `vscode`.
 
+## Obtaining data
+
+The data used in this repository can be obtained using the scripts in
+`data-raw/`. In particular, PCR positivity data by nation, region, age and
+variant can be downloaded using
+
+```{sh}
+Rscript data-raw/update-cis.R
+```
+
+and antibody data can be downloaded using
+
+```{sh}
+Rscript data-raw/update-ab.R
+```
+
+## Reproducing the estimates
+
+Estimates provided here can be generated using the `scripts/estimate.R` script.
+The estimates shown in the plots above were generated using
+
+```{sh}
+Rscript scripts/estimate.R -d 1
+```
+
 ## Dependencies
 
 All dependencies can be installed using the following, 

docs/report.Rmd

@@ -0,0 +1,46 @@
+---
+title: "Real-time epidemiological estimates from ONS Community Infection Survey data"
+author: Sam Abbott, Sebastian Funk
+date: "`r format(Sys.Date(), format = '%B %d, %Y')`"
+output: html_document
+---
+
+```{r setup, echo = FALSE, include = FALSE}
+library(cowplot)
+library(here)
+
+estimates <- c(national = "National",
+               regional = "Regional",
+               age_school = "Age")
+
+quantities <- c(est_prev = "PCR positivity prevalence",
+                infections = "Daily infection incidence",
+                r = "Growth rate",
+                Rt = "Reproduction number")
+
+time_horizon <- c(est_prev = "1year",
+                  infections = "1year",
+                  r = "3months",
+                  Rt = "3months")
+```
+
+Estimates were derived from the modelled daily and weekly data published weekly as part of the Covid-19 Community Infection Survey by the Office of National Statistics. Prevalence and incidence estimates are shown for the last year, and reproduction number for the past three months. Code to reproduce the results and links to data tables with estimates are available at https://github.com/epiforecasts/inc2prev
+
+```{r, results = 'asis'}
+for (level in names(estimates)) {
+  cat("\n\n#", estimates[[level]], "estimates", "{.tabset}", "\n")
+  for (quantity in names(quantities)) {
+    cat("\n\n##", quantities[[quantity]], "\n")
+    file_name <-
+      paste0(level, "_", quantity, "_", time_horizon[[quantity]], ".svg")
+    p <- ggdraw() + draw_image(here::here("figures", "additional", file_name))
+    print(p)
+  }
+}
+```
+
+# License
+
+This project uses data from the Office for National Statistics Community Infection Survey, which is licensed under the [Open Government License v3.0](https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datasets/coronaviruscovid19infectionsurveydata).
+
+