Experiment to test diagnostic noncompliance propagation

[korourke5]

in README. Created scenario files from prepare.py. Edited run_iteration.sh and summarize.py for new
AWS S3.

CRCsim PR Template

Description

PR Type

• Feature (intended to merge)
• Experiment (not intended to merge)

Experiment review checklist

• This experiment branch has the exp- prefix
• This experiment uses the appropriate crcsim commit hash (typically, the latest commit in the main branch). This ensures that the experiment uses the latest version of the model.
• This experiment uses the correct baseline parameters.json file
  • Correct screening start and end ages for the experiment
  • Latest values of all calibrated parameters
  • All necessary tests and latest test parameters
  • Latest cost parameters
• My prepare.py script applies incidence rate ratio (IRR) adjustment if appropriate for this experiment - NOT applied
• I have run prepare.py and:
  • Confirmed that all scenarios necessary for this experiment were created
  • Confirmed that no extraneous scenarios are in the scenarios/ directory (eg, uncommitted local changes from a previous experiment)
  • Spot-checked a sample of scenario parameter files to ensure that my scenario creation logic works as expected
• I have updated crcsim/experiment/README.md with detailed information about the experiment's goals, scenarios, and corresponding AWS objects
• I am opening this PR as a Draft PR

Experiment review process

All experiments must follow each step of this review process.

1. Contributor checks all items in experiment review checklist
2. Contributor opens Draft PR and tags a collaborator to review
3. Reviewer conducts a thorough review, including pulling experiment branch, running prepare.py, and spot-checking scenarios
4. Contributor and reviewer address any issues identified during review
5. Reviewer explicitly approves PR
6. Contributor builds image and pushes to ECR, copies experiment files to S3, and runs experiment
7. Contributor analyzes experiment (eg crcsim/experiment/summarize.py)
8. Contributor pushes results summary, so we have documentation of detailed results (eg crcsim/experiment/summary/summarized.xlsx)
9. Contributor updates experiment README with a detailed summary of results
10. Contributor closes this PR with an informative comment (eg, brief summary of results)

Note that experiment PRs are never merged into main! The PR is closed, and the experiment is maintained as a separate branch. That's why we keep all experiment PRs as drafts.

[sandypreiss]

I think I see the issue here. This experiment is still installing crcsim from an older commit. (It looks like you copied the commit hash from exp-fqhc-diagnositic-compliance-comparison.)

Experiments use whatever version of crcsim is installed in the Docker image. That version is defined in pyproject.toml, which creates uv.lock, which is used to install the project in the Dockerfile.

So, even though this experiment branch has the new feature in the local code, it's running the experiment with an older version of crcsim. That means the propagate_diagnostic_noncompliance parameter is never used.

To resolve this:

1. Update the crcsim commit hash in pyproject.toml to the latest commit from main (now that you've merged the noncompliance propagation branch in). You can get that by opening the commit history from the repo landing page. Just to be sure, it's c4e08bffba34c07d18b784c0632de506a36611e8.
2. Update the comment in pyproject.toml to explain which commit you're installing from and why.
3. From crcsim/experiment, run uv lock to update the lockfile.
4. Proceed with the normal steps to run the experiment (including rebuild and push).

I also added a few small style suggestions.

[korourke5]

@sandypreiss Those updates should be applied.

[korourke5]

@sandypreiss I have updated the scenarios here with the hispanic male and female parameters added and removed extra low and low cost scenarios since I don't believe that will impact this analysis.

[sandypreiss]

This is looking good. Have you run it yet? Are we seeing the effect we'd expect from diagnostic noncompliance propagation now?

[korourke5]

I am currently running it. I had a bit of a problem running it last week because of the race specific mean_duration_clin but have rectified it. I hope to have results to send over tomorrow.

[sandypreiss]

This was run successfully. Results need to be pushed and a brief summary added to experiment readme.

[korourke5]

Updated with results and scenario files.