Support censoring (in latent projection) #528
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds support for censored observations (i.e., survival or time-to-event analysis) when using the latent projection. Originally, #2 already mentioned that support for time-to-event models would be a nice feature, and the request came up again recently in https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275. This PR only makes it possible to use the latent projection for time-to-event models. For a more general perspective, see #2.
The solution presented here is not optimal because it would probably be more desirable to allow for an extension of the formula in
init_refmodel()
(similarly to brms'sresp_cens()
term), see https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275/10. Via such an extension of theinit_refmodel()
formula, we could also handle observation weights in a better way.In contrast to the less elegant solution suggested in https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275/10, the solution presented here requires users to add a right-hand side formula as an attribute called
cens_var
to thelatent_ll_oscale
function. The variable named in that formula is then retrieved (internally, whenever calling thelatent_ll_oscale
function) from the original dataset (possibly subsetted to the observations from a given test set),newdata
, or elementdata
fromvarsel()
's argumentd_test
, whichever is applicable in the specific situation where thelatent_ll_oscale
function is called. The content of the retrieved variable is then passed to argumentcens
of thelatent_ll_oscale
function. In other words, the variable mentioned in thecens_var
formula needs to contain the censoring indicators (e.g.,0
= uncensored,1
= censored) which can then be used (by the user) within thelatent_ll_oscale
function. On the Stan Forums, I will update the Weibull (and perhaps also the log-normal) example from https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275 to illustrate this.The solution presented here is still less elegant than the
brms::resp_cens()
-like solution, but in my opinion, it is preferable over the less elegant solution I suggested in https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275/10. Via thecens_var
attribute, we also catch the censored-observations case (when using the latent projection) invalidate_vsel_object_stats()
to throw a warning if a performance statistic other than"elpd"
,"mlpd"
, or"gmpd"
is used.This is only a draft PR. Still to do:
NEWS.md
.DESCRIPTION
file to mention survival models.@avehtari: A few weeks ago, we discussed how we should make this feature available and we decided to keep this feature in a separate branch. Thinking over this again, I would say we could merge this into
master
as soon as the to-do's from above are completed. The reason is that I consider the implementation presented here not as "hacky" as the solution we discussed back then (which was the less elegant solution I suggested in https://discourse.mc-stan.org/t/using-projpred-latent-projection-with-brms-weibull-family-models/39275/10). But I'll leave the decision up to you and @aloctavodia what you want to do with this PR. Unfortunately, I was not able to request a review from @aloctavodia for this PR.