Confused about Prevalence for Case-Control

[AdamMS]

I suspect the 'prevalence' option for case-control is not working the way I expect it to work...
[I am using the dcurves package in R]

I think the following two approaches should yield identical results:

• Fit a simple logistic regression model to the data and feed it into dca() while setting prevalence = population prevalence
• Duplicate the controls in the dataset in order to achieve the desired population prevalence. Then fit the logistic regression model to the augmented dataset and feed it into dca() without setting the prevalence value.

However, the results are clearly not identical.

Can somebody either provide a brief explanation why these approaches should not generate the same result? Or point me to a source where this is explained (I have been unable to find anything addressing this issue)?

I am attaching a short script as a .txt file with a straight-forward example.

MWE_CaseControl_DCA.txt

[VickersA]

Can you send a picture of the two decision curves? That might help diagnose the problem.

[AdamMS]

The image at the end of this email is generated by the MWE I included in my original post, as indeed is the mistaken assumption I made about how the software works. I had a flicker of inspiration over the weekend that showed where the confusion lay, but the image can still serve as a demonstration of what goes wrong if a user (like me) misunderstands the software.

I think my error could be avoided if the documentation for 'prevalence' were more explicit in both the Tutorial and the R help("dca") documentation.

I wrongly thought that the following was a valid workflow:

1. Fit a prediction model to the unweighted case-control data to generate point estimates for patient-level risks. These risks estimates reflect only the case-control prevalence, not the true population prevalence.
2. Supply the risks to dca(), setting prevalence = "population prevalence", in order to calculate Net Benefit at a pre-specified range of threshold probabilities -- wrongly thinking that dca() would adjust the patient-level risk estimates to reflect the change in prevalence.

The correct workflow for the "prevalence" option as implemented in the software is to:

1. Fit a weighted logistic regression model (or other weighted model) to the case-control data to generate point estimates for patient-level risks. These risks estimates must reflect the true population prevalence.
2. Supply the risks to dca(), setting prevalence = "population prevalence", in order to calculate Net Benefit at a pre-specified range of threshold probabilities -- dca() will weight the Net Benefit calculations to reflect the change in prevalence.

In my defense, both of the above could be valid workflows. The algebra to convert patient-level risk estimates to a new prevalence is not difficult. But it is not the workflow implemented by the software. And that is why I think the documentation needs to be more explicit about: (i) what the prevalence statement does, and (ii) the need to feed already-weighted estimates of risk to the dca() function.

[VickersA]

Fair point, we will consider it From: Adam Martin-Schwarze ***@***.***> Reply-To: MSKCC-Epi-Bio/decisioncurveanalysis ***@***.***> Date: Monday, October 23, 2023 at 9:11 AM To: MSKCC-Epi-Bio/decisioncurveanalysis ***@***.***> Cc: "Vickers, Andrew" ***@***.***>, Comment ***@***.***> Subject: [EXTERNAL] Re: [MSKCC-Epi-Bio/decisioncurveanalysis] Confused about Prevalence for Case-Control (Discussion #8) The image at the end of this email is generated by the MWE I included in my original post, as indeed is the mistaken assumption I made about how the software works. I had a flicker of inspiration over the weekend that showed where the confusion The image at the end of this email is generated by the MWE I included in my original post, as indeed is the mistaken assumption I made about how the software works. I had a flicker of inspiration over the weekend that showed where the confusion lay, but the image can still serve as a demonstration of what goes wrong if a user (like me) misunderstands the software. I think my error could be avoided if the documentation for 'prevalence' were more explicit in both the Tutorial and the R help("dca") documentation. I wrongly thought that the following was a valid workflow: 1. Fit a prediction model to the unweighted case-control data to generate point estimates for patient-level risks. These risks estimates reflect only the case-control prevalence, not the true population prevalence. 2. Supply the risks to dca(), setting prevalence = "population prevalence", in order to calculate Net Benefit at a pre-specified range of threshold probabilities -- wrongly thinking that dca() would adjust the patient-level risk estimates to reflect the change in prevalence. The correct workflow for the "prevalence" option as implemented in the software is to: 1. Fit a weighted logistic regression model (or other weighted model) to the case-control data to generate point estimates for patient-level risks. These risks estimates must reflect the true population prevalence. 2. Supply the risks to dca(), setting prevalence = "population prevalence", in order to calculate Net Benefit at a pre-specified range of threshold probabilities -- dca() will weight the Net Benefit calculations to reflect the change in prevalence. In my defense, both of the above could be valid workflows. The algebra to convert patient-level risk estimates to a new prevalence is not difficult. But it is not the workflow implemented by the software. And that is why I think the documentation needs to be more explicit about: (i) what the prevalence statement does, and (ii) the need to feed already-weighted estimates of risk to the dca() function. [Image removed by sender. MWE_Image]<https://urldefense.com/v3/__https:/user-images.githubusercontent.com/4751000/277337531-48be4f0f-81fb-4107-a964-883c2ff2ae33.png__;!!KVWo1iE!Tp5HipGSzxFpNCRXIFQzr5RugtMkQwfIvycvw0hy5g87RA5eQQNmqe6asbFVXyV4GjahRoWfkXZHC6yPdnOPSpw$> — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/MSKCC-Epi-Bio/decisioncurveanalysis/discussions/8*discussioncomment-7358765__;Iw!!KVWo1iE!Tp5HipGSzxFpNCRXIFQzr5RugtMkQwfIvycvw0hy5g87RA5eQQNmqe6asbFVXyV4GjahRoWfkXZHC6yPTRzqWp0$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AVDIPRUNZAMMMNJIA2HRLDDYAZUINAVCNFSM6AAAAAA6HIBQWWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TGNJYG43DK__;!!KVWo1iE!Tp5HipGSzxFpNCRXIFQzr5RugtMkQwfIvycvw0hy5g87RA5eQQNmqe6asbFVXyV4GjahRoWfkXZHC6yPas2e08w$>. You are receiving this because you commented.Message ID: ***@***.***> ===================================================================== Please note that this e-mail and any files transmitted from Memorial Sloan Kettering Cancer Center may be privileged, confidential, and protected from disclosure under applicable law. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this communication or any of its attachments is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and deleting this message, any attachments, and all copies and backups from your computer. Disclaimer ID:MSKCC