Currently Contest.is_private, Contest.is_organization_private, Contest.limit_join_organizations, and Problem.is_organization_private are computed on save_model. However, if all associated Organization or Profile objects is deleted, these booleans will desync.
The booleans will be recomputed on model save, and the probability of this bug occurring naturally are quite low. We could fix this by writing listeners, or we could wait for the site to update to Django 5 and use https://docs.djangoproject.com/en/5.1/ref/models/fields/#generatedfield.