R/stremr: Streamlined Causal Inference for Static, Dynamic and Stochastic Regimes in Longitudinal Data
Analysis of longitudinal data, with continuous or time-to-event (binary) outcome and time-varying confounding. Allows adjustment for all measured time-varying confounding and informative right-censoring. Estimate the expected counterfactual outcome under static, dynamic or stochastic interventions. Includes doubly robust and semi-parametrically efficient Targeted Minimum Loss-Based Estimator (TMLE), along with several other estimators.
Perform data-adaptive estimation of the outcome and treatment models with Super Learnersl3.
Authors: Oleg Sofrygin, Holly Finertie, Mark van der Laan, Romain Neugebauer
Currently available estimators can be roughly categorized into 4 groups:
- Propensity-score / Inverse Probability Weighted (IPW):
- direct (bounded) IPW (
directIPW) - IPW-adjusted Kaplan-Meier
(
survNPMSM) - MSM-IPW for the survival
hazard (
survMSM)
- direct (bounded) IPW (
- Outcome regression:
- longitudinal G-formula (
GCOMP)
- longitudinal G-formula (
- Doubly-robust (DR) approaches:
- TMLE for longitudinal data (
TMLE) - iterative TMLE (
iterTMLE) - cross-validated TMLE (
CVTMLE)
- TMLE for longitudinal data (
- Sequentially doubly-robust (SDR) approaches:
- infinite-dimensional TMLE (
iTMLE) - doubly robust unbiased transformations (
DR_transform) (Rubin and van der Laan, 2006, Luedtke, Sofrygin, van der Laan, and Carone (2017)`)
- infinite-dimensional TMLE (
The exposure, monitoring and censoring variables can be coded as either binary, categorical or continuous. Each can be multivariate (e.g., can use more than one column of dummy indicators for different censoring events). The input data needs to be in long format.
- Possibly right-censored data has to be in long format.
- Each row must contain a subject identifier (
ID) and the integer indicator of the current time (t), e.g., day, week, month, year. - The package assumes that the temporal ordering of covariates in each
row is fixed according to (
ID,t,L,C,A,N,Y), whereL– Time-varying and baseline covariates.C– Indicators of right censoring events at timet; this can be either a single categorical or several binary columns.A– Exposure (treatment) at timet; this can be multivariate (more than one column) and each column can be binary, categorical or continuous.N– Indicator of being monitored at time pointt+1(binary).Y– Outcome (binary 0/1 or continuous between 0 and 1).
- Categorical censoring can be useful for representing all of the censoring events with a single column (variable).
- Separate models are fit for the observed censoring, exposure and monitoring mechanisms. Jointly, these make up what is known as the propensity score.
- Separate outcome regression models can be specified for each time-point.
- Each propensity score model can be stratified (separate model is fit) by time or any other user-specified stratification criteria. Each strata is defined with by a single logical expression that selects specific observations/rows in the observed data (strata).
- By default, all models are fit with the logistic regression.
- Alternatively, model fitting can be performed via any machine
learning (ML) algorithm available in
sl3andgridislR packages. Seexgboostandh2ofor a sample description of possible ML algorithms. - One can select the best model from an ensemble of many learners via model stacking or Super Learning, which finds the optimal convex combination of all models in the ensemble via cross-validation.
- Installing
stremrand Documentation - Issues
- Documentation
- Example with Simulated Data
- Sequential G-Computation (GCOMP) and Targeted Maximum Likelihood Estimation (TMLE) for longitudinal survival data
- Machine Learning
- Details on some implemented estimators
To install the development version (requires the devtools package):
devtools::install_github('romainkp/stremr')For ensemble-learning with Super Learner algorithm we recommend
installing the latest development version of the sl3 R package. It can
be installed as follows:
devtools::install_github('jeremyrcoyle/sl3')For optimal performance, we also recommend installing the latest version
of data.table package:
remove.packages("data.table") # First remove the current version
install.packages("data.table", type = "source",
repos = "http://Rdatatable.github.io/data.table") # Then install devel versionIf you encounter any bugs or have any specific feature requests, please file an issue.
To obtain documentation for specific relevant functions in stremr
package:
?stremr
?importData
?fitPropensity
?getIPWeights
?directIPW
?survNPMSM
?survMSM
?fit_GCOMP
?fit_iTMLELoad the data:
require("magrittr")
#> Loading required package: magrittr
require("data.table")
#> Loading required package: data.table
require("stremr")
#> Loading required package: stremr
data(OdataNoCENS)
datDT <- as.data.table(OdataNoCENS, key=c("ID", "t"))Define some summaries (lags):
datDT[, ("N.tminus1") := shift(get("N"), n = 1L, type = "lag", fill = 1L), by = ID]
datDT[, ("TI.tminus1") := shift(get("TI"), n = 1L, type = "lag", fill = 1L), by = ID]Define counterfactual exposures. In this example we define one
intervention as always treated and another as never treated. Such
intervention can be defined conditionally on other variables (dynamic
intervention). Similarly, one can define the intervention as a
probability that the counterfactual exposure is 1 at each time-point t
(for stochastic interventions).
datDT[, ("TI.set1") := 1L]
datDT[, ("TI.set0") := 0L]Import input data into stremr object DataStorageClass and define
relevant covariates:
OData <- importData(datDT, ID = "ID", t = "t", covars = c("highA1c", "lastNat1", "N.tminus1"), CENS = "C", TRT = "TI", OUTCOME = "Y.tplus1")Once the data has been imported, it is still possible to inspect it and modify it, as shown in this example:
get_data(OData)[, ("TI.set0") := 1L]
get_data(OData)[, ("TI.set0") := 0L]Regressions for modeling the propensity scores for censoring (CENS)
and exposure (TRT). By default, each of these propensity scores is fit
with a common model that pools across all available time points
(smoothing over all time-points).
gform_CENS <- "C ~ highA1c + lastNat1"
gform_TRT <- "TI ~ CVD + highA1c + N.tminus1"Stratification, that is, fitting separate models for different
time-points, is enabled with logical expressions in arguments
stratify_... (see ?fitPropensity). For example, the logical
expression below states that we want to fit the censoring mechanism with
a separate model for time point 16, while pooling with a common model
fit over time-points 0 to 15. Any logical expression can be used to
define such stratified modeling. This can be similarly applied to
modeling the exposure mechanism (stratify_TRT) and the monitoring
mechanism (stratify_MONITOR).
stratify_CENS <- list(C=c("t < 16", "t == 16"))Fit the propensity scores for censoring, exposure and monitoring:
OData <- fitPropensity(OData,
gform_CENS = gform_CENS,
gform_TRT = gform_TRT,
stratify_CENS = stratify_CENS)Estimate survival based on non-parametric/saturated IPW-MSM (IPTW-ADJUSTED KM):
AKME.St.1 <- getIPWeights(OData, intervened_TRT = "TI.set1") %>%
survNPMSM(OData) %$%
estimatesThe result is a data.table that contains the estimates of the
counterfactual survival for each time-point, for the treatment regimen
TI.set1. In this particular case, the column St.NPMSM contains the
survival estimates for IPW-NPMSM and the first row represents the
estimated proportion alive at the end of the first cycle / time-point.
Note that the column St.KM contains the unadjusted/crude estimates of
survival (should be equivalent to standard Kaplan-Meier estimates for
most cases).
head(AKME.St.1[],2)
#> est_name time sum_Y_IPW sum_all_IPAW ht.NPMSM St.NPMSM ht.KM
#> 1: NPMSM 0 1.6610718 38.13840 0.04355379 0.9564462 0.04733728
#> 2: NPMSM 1 0.8070748 48.10323 0.01677797 0.9403990 0.01863354
#> St.KM rule.name
#> 1: 0.9526627 TI.set1
#> 2: 0.9349112 TI.set1Estimate survival with bounded IPW:
IPW.St.1 <- getIPWeights(OData, intervened_TRT = "TI.set1") %>%
directIPW(OData) %$%
estimatesAs before, the result is a data.table with estimates of the
counterfactual survival for each time-point, for the treatment regimen
TI.set1, located in column St.directIPW.
head(IPW.St.1[],2)
#> est_name time sum_Y_IPW sum_IPW St.directIPW rule.name
#> 1: directIPW 0 9.828827 225.6710 0.9564462 TI.set1
#> 2: directIPW 1 14.841714 308.6067 0.9519073 TI.set1Estimate hazard with IPW-MSM then map into survival estimate. Using two regimens and smoothing over two intervals of time-points:
wts.DT.1 <- getIPWeights(OData = OData, intervened_TRT = "TI.set1", rule_name = "TI1")
wts.DT.0 <- getIPWeights(OData = OData, intervened_TRT = "TI.set0", rule_name = "TI0")
options(datatable.allow.cartesian = TRUE)
survMSM_res <- survMSM(list(wts.DT.1, wts.DT.0), OData, tbreaks = c(1:8,12,16)-1,trunc_weights=20)In this particular case the output is a little different, with separate
survival tables for each regimen. The output of survMSM is hence a
list, with one item for each counterfactual treatment regimen considered
during the estimation. The actual estimates of survival are located in
the column(s) St.MSM. Note that survMSM output also contains the
standard error estimates of survival at each time-point in column(s)
SE.MSM. Finally, the output table also contains the subject-specific
estimates of the influence-curve (influence-function) in column(s)
IC.St. These influence function estimates can be used for constructing
the confidence intervals of the counterfactual risk-differences for two
contrasting treatments (see help for get_RDs function for more
information).
head(survMSM_res[["TI0"]][["estimates"]],2)
#> est_name time ht.MSM St.MSM SE.MSM rule.name
#> 1: MSM 0 0.004214338 0.9957857 0.002105970 TI0
#> 2: MSM 1 0.016516126 0.9793391 0.004850157 TI0
#> IC.St
#> 1: 0.004543242,0.004543242,0.004543242,0.004543242,0.004543242,0.004543242,...
#> 2: 0.004468206,0.023952415,0.024134964,0.025897204,0.025897204,0.025897204,...
head(survMSM_res[["TI1"]][["estimates"]],2)
#> est_name time ht.MSM St.MSM SE.MSM rule.name IC.St
#> 1: MSM 0 0.04355379 0.9564462 0.01521910 TI1 0,0,0,0,0,0,...
#> 2: MSM 1 0.01677797 0.9403990 0.01786105 TI1 0,0,0,0,0,0,...Define time-points of interest, regression formulas and software to be used for fitting the sequential outcome models:
tvals <- c(0:10)
Qforms <- rep.int("Qkplus1 ~ CVD + highA1c + N + lastNat1 + TI + TI.tminus1", (max(tvals)+1))To run iterative means substitution estimator (G-Computation), where all
at risk observations are pooled for fitting each outcome regression
(Q-regression):
options(gridisl.verbose = FALSE)
gcomp_est <- fit_GCOMP(OData, tvals = tvals, intervened_TRT = "TI.set1", Qforms = Qforms)
#> Registered S3 methods overwritten by 'gridisl':
#> method from
#> +.ModelStack stremr
#> pander.H2OBinomialMetrics stremr
#> pander.H2OGrid stremr
#> pander.H2ORegressionMetrics stremr
#> print.GLMmodel stremr
#> print.H2Oensemblemodel stremr
#> print.ModelStack stremr
#> summary.GLMmodel stremr
#> summary.H2OBinomialModel stremr
#> summary.H2ORegressionModel stremr
#> summary.H2Oensemblemodel stremr
#> summary.xgb.Booster stremr
#> summary.xgb.cv.synchronous stremrThe output table of fit_GCOMP contains the following information, with
the column St.GCOMP containing the survival estimates for each time
period:
head(gcomp_est$estimates[],2)
#> est_name time St.GCOMP St.TMLE type cum.inc IC.St
#> 1: GCOMP 0 0.9837583 NA pooled 0.01624168 NA,NA,NA,NA,NA,NA,...
#> 2: GCOMP 1 0.9699022 NA pooled 0.03009778 NA,NA,NA,NA,NA,NA,...
#> fW_fit rule.name
#> 1: TI.set1
#> 2: TI.set1To run the longitudinal long format Targeted Minimum-Loss Estimation
(TMLE), stratified by rule-followers for fitting each outcome
regression (Q-regression):
tmle_est <- fit_TMLE(OData, tvals = tvals, intervened_TRT = "TI.set1", Qforms = Qforms)
#> GLM TMLE update cannot be performed since the outcomes (Y) are either all 0 or all 1, setting epsilon to 0
#> Warning: Supplying `...` without names was deprecated in tidyr 1.0.0.
#> ℹ Please specify a name for each selection.
#> ℹ Did you want `IC.St = c(clusterID, EIC_i)`?
#> ℹ The deprecated feature was likely used in the stremr package.
#> Please report the issue at <https://github.com/osofr/stremr/issues>.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was generated.
#> GLM TMLE update cannot be performed since the outcomes (Y) are either all 0 or all 1, setting epsilon to 0The output table of fit_TMLE contains the following information, with
the column St.TMLE containing the survival estimates for each time
period. In addition, the column SE.TMLE contains the standard error
estimates and the column and the column IC.St contains the
subject-specific estimates of the efficient influence curve. The letter
estimates are useful for constructing the confidence intervals of risk
differences for two contrasting treatments (see help for get_RDs
function for more information).
head(tmle_est$estimates[],2)
#> est_name time St.GCOMP St.TMLE type cum.inc SE.TMLE
#> 1: TMLE 0 NA 0.9839271 pooled 0.01607286 0.003449949
#> 2: TMLE 1 NA 0.9707676 pooled 0.02923243 0.004492235
#> IC.St
#> 1: -0.007292922,-0.007292922,-0.010190141,-0.007292922,-0.007292922,-0.007292922,...
#> 2: -0.009469707,-0.009469707,-0.010503891,-0.009469707,-0.009469707,-0.009469707,...
#> fW_fit rule.name
#> 1: TI.set1
#> 2: TI.set1To parallelize estimation over several time-points (tvals) for either
GCOMP or TMLE use argument parallel = TRUE:
require("doParallel")
registerDoParallel(cores = parallel::detectCores())
tmle_est <- fit_TMLE(OData, tvals = tvals, intervened_TRT = "TI.set1", Qforms = Qforms, parallel = TRUE)Nuisance parameters can be modeled with any machine learning algorithm
supported by sl3 R package. For
example, for GLMs use learner Lrnr_glm_fast, for xgboost use learner
Lrnr_xgboost, for h2o GLM learner use Lrnr_h2o_glm, for any other
ML algorithm implemented in h2o use
Lrnr_h2o_grid$new(algorithm = "algo_name"), for glmnet use learner
Lrnr_glmnet. All together, these learners provide access to a wide
variety of ML algorithms. To name a few: GLM, Regularized GLM,
Distributed Random Forest (RF), Extreme Gradient Boosting (GBM) and
Deep Neural Nets.
Model selection can be performed via V-fold cross-validation or random
validation splits and model stacking and Super Learner combination can
be accomplished by using the learner Lrnr_sl and specifying the
meta-learner (e.g., Lrnr_solnp). In the example below we define a
Super Learner ensemble consisting of several learning algorithms.
First, we define sl3 learners for for xgboost, two types of GLMs and
glmnet. Then we will stack these learners into a single learner called
Stack:
library("sl3")
lrn_xgb <- Lrnr_xgboost$new(nrounds = 5)
lrn_glm <- Lrnr_glm_fast$new()
lrn_glm2 <- Lrnr_glm_fast$new(covariates = c("CVD"))
lrn_glmnet <- Lrnr_glmnet$new(nlambda = 5, family = "binomial")
## Stack the above candidates:
lrn_stack <- Stack$new(lrn_xgb, lrn_glm, lrn_glm2, lrn_glmnet)Next, we will define a Super Learner on the above defined stack, by
feeding the stack into the Lrnr_sl object and then specifying the
meta-learner that will find the optimal convex combination of the
learners in a stack (Lrnr_solnp):
lrn_sl <- Lrnr_sl$new(learners = lrn_stack, metalearner = Lrnr_solnp$new())We will now use stremr to estimate the exposure / treatment propensity
model with the above defined Super Learner (lrn_sl):
OData <- fitPropensity(OData,
gform_CENS = gform_CENS,
gform_TRT = gform_TRT,
models_TRT = lrn_sl,
stratify_CENS = stratify_CENS)Currently implemented estimators include:
- Kaplan-Meier Estimator. No adjustment for time-varying confounding or informative right-censoring.
- Inverse Probability Weighted (IPW) Kaplan-Meier (
survNPMSM). Also known as the Adjusted Kaplan Meier (AKME). Also known as the saturated (non-parametric) IPW-MSM estimator of the survival hazard. This estimator inverse weights each observation based on the exposure/censoring model fits (propensity scores). - Bounded Inverse Probability Weighted (B-IPW) Estimator of Survival(‘directIPW’). Estimates the survival directly (without hazard), also based on the exposure/censoring model fit (propensity scores).
- Inverse Probability Weighted Marginal Structural Model (
survMSM) for the hazard function, mapped into survival. Currently only logistic regression is allowed where covariates are time-points and regime/rule indicators. This estimator is also based on the exposure/censoring model fit (propensity scores), but allows additional smoothing over multiple time-points and includes optional weight stabilization. - Longitudinal G-formula (
GCOMP). Also known as the iterative G-Computation formula or Q-learning. Directly estimates the outcome model while adjusting for time-varying confounding. Estimation can be stratified by rule/regime followed or pooled across all rules/regimes. - Longitudinal Targeted Minimum-Loss-based Estimator (
TMLE). Also known as L-TMLE. Doubly robust and semi-parametrically efficient estimator that de-biases each outcome regression fit with a targeting step, using IPW. - Iterative TMLE (
iterTMLE) for longitudinal data. Fits sequential G-Computation and then iteratively performs targeting for all pooled Q’s until convergence. - Infinite-dimensional TMLE (
iTMLE) for longitudinal data. Fits sequential G-Computation and performs additional infinite-dimensional targeting to achieve sequential double robustness.
To cite stremr in publications, please use:
Sofrygin O, Finertie H, van der Laan MJ, Neugebauer R (2017). stremr: Streamlined Estimation for Static, Dynamic and Stochastic Treatment Regimes in Longitudinal Data. R package version x.x.xx
This work was partially supported through several Patient-Centered Outcomes Research Institute (PCORI) Awards (ME-1403-12506, ME-2018C1-10942, DB-2020C2-20318). All statements in this work, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. This work was also supported through an NIH grant (R01 AI074345-07).
The contents of this repository are distributed under the MIT license.
The MIT License (MIT)
Copyright (c) 2015-2017 Oleg Sofrygin
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1. Breiman L. Stacked regressions. Machine learning (1996) 24:49–64.
2. Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics (2005) 61:962–973.
3. van der Laan MJ, Polley EC, Hubbard AE. Super learner. Stat Appl Genet Mol Biol (2007) 6:Article25.
4. van der Laan MJ, Gruber S. Targeted minimum loss based estimation of causal effects of multiple time point interventions. Int J Biostat (2012) 8:
5. Neugebauer R, Schmittdiel JA, van der Laan MJ. Targeted learning in real-world comparative effectiveness research with time-varying interventions. Stat Med (2014) 33:2480–2520.
6. Neugebauer R, An J, Dombrowski SK, Oshiro C, Cassidy-Bushrow A, Gilliam L, Simonson G, Karter AJ, Bergenstal R, Finertie H, et al. Glucose-lowering medication classes and cardiovascular outcomes in patients with type 2 diabetes. JAMA Netw Open (2025) 8:e2536100.
7. Neugebauer R, Schmittdiel JA, van der Laan MJ. A case study of the impact of data-adaptive versus model-based estimation of the propensity scores on causal inferences from three inverse probability weighting estimators. Int J Biostat (2016) 12:131–155.
8. Neugebauer R, Schmittdiel JA, Adams AS, Grant RW, van der Laan MJ. Identification of the joint effect of a dynamic treatment intervention and a stochastic monitoring intervention under the no direct effect assumption. J Causal Inference (2017) 5:
9. Luedtke AR, Sofrygin O, van der Laan MJ, Carone M. Sequential double robustness in right-censored longitudinal models. arXiv preprint arXiv:170502459 (2017) Available at: https://arxiv.org/abs/1705.02459
10. Sofrygin O, Zhu Z, Schmittdiel JA, Adams AS, Grant RW, van der Laan MJ, Neugebauer R. Targeted learning with daily EHR data. Stat Med (2019) 38:3073–3090.
11. Kreif N, Sofrygin O, Schmittdiel JA, Adams AS, Grant RW, Zhu Z, van der Laan MJ, Neugebauer R. Exploiting nonsystematic covariate monitoring to broaden the scope of evidence about the causal effects of adaptive treatment strategies. Biometrics (2021) 77:329–342.