Newbie question re MeasureSurvDCalibration.R

[kiwigander]

Hi all, I'm a student of statistics and have limited skill with R, so please forgive this ?silly request.

What is the syntax for obtaining the D-calibration of a model?

I have a large number of synthetic data sets, each divided into one training and one testing subset. (I am not concerned about cross-validation). I want to check the D-calibration of each testing subset. I have written code that gives two versions of the C-index, integrated Brier score, IAE and ISE. But I have not been able to deduce from the mlr-org/mlr3proba page or from the (rather odd-looking) Help pane in RStudio just how to produce the D-calibration.

[bblodfon]

Hi, here is a minimum working example:

library(mlr3proba)
#> Loading required package: mlr3

task = tsk("lung")
learner = lrn("surv.coxph")
part = partition(task)
p = learner$train(task, part$train)$predict(task, part$test)
p$score(msr("surv.dcalib"))
#> surv.dcalib 
#>    11.39873

^{Created on 2025-08-15 with reprex v2.1.1}

[kiwigander]

Thank you, I have run that code. A few further questions:

• Can the data be modelled by other than a Cox model (for example, if I want to compare how Cox and Weibull models perform on a number of data sets)?
• Can the D-calibration be computed on data sets that are pre-partitioned? I've already got multiple data sets each partitioned into df0.train and df0.test; I've been training my models on the former (via rsfrc, coxph or survreg) and testing them on the latter.

[bblodfon]

1. Of course - see all available survival learners with mlr_learners$keys("surv"). Using mlr3extralearners loads a bunch of those.
2. Yes sure, there are many ways to do this. The easiest would be to run a benchmark and then score() using the dcalib or other metrics. Learners support also the predict_newdata() that can be used on data frames. If you manually do the train and predict steps and you have the survival prediction matrix, you can use .surv_return and then call as_prediction(output-list-from-surv-return) to make a PredictionSurv object (see also as_prediction_surv) and there you can use the score with the metric you want (given that it is not a complicated measure, ie doesn't use training data).