Label free parameters in modules

[marghe-molaro]

Parameters in each module should be labelled as either free (i..e tuned during calibration to data) or fixed (assumed to be constant during calibration). This would allows us to:

• Get a better handle on the number of free parameters assumed in the model, which would be important in evaluating its predictive validity;
• Allow any developer to identify which parameters can be changed if issues in calibration arise during development;
• Facilitate the development in the future of more robust calibration frameworks;

Parameters which are fixed could be further identified as being either:

• Country-dependent: parameters assumed to be fixed in the current calibration, but which may need to be reviewed if the model were to be generalised to other countries;
• Constant: parameters that can be safely assumed to be constant regardless of context in which they are applied (e.g. tests sensitivity)
  This would partially build on PR ScenarioSwitcher to an ideal healthcare system and ideal healthcare seeking #973

[matt-graham]

Adding to the list of things it might be useful to track for free parameters would be some form of prior distribution representing our beliefs aboutt the plausible values for the parameters, and/or specification of any hard constraints on the parameter (for example positive / non-negative, bounded).

[mnjowe]

@marghe-molaro and @tbhallett . Any ideas on how to get started on this one? I'm interested in this issue

[marghe-molaro]

Hi @mnjowe, my suggestion would be to create a single label for parameters which can take values ["free", "constant", "context_specific"], where:
"free": a free parameter in the module
"constant": a parameter which is universally constant (e.g. test specificity/sensitivity)
"context_specific": a parameter which is informed by available data but is not universally constant, and therefore may very from context to context (e.g. probability of referral for a particular screening)

Each parameter declared in each module would then have to be assigned one of these labels. The safest thing to do would be to speak to the module developers to understand what label each of their parameters should be assigned, although some of them may be inferred from references/context.

Priors could be collected at the same time or in a second phase.

[tbhallett]

Hi @mnjowe, my suggestion would be to create a single label for parameters which can take values ["free", "constant", "context_specific"], where: "free": a free parameter in the module "constant": a parameter which is universally constant (e.g. test specificity/sensitivity) "context_specific": a parameter which is informed by available data but is not universally constant, and therefore may very from context to context (e.g. probability of referral for a particular screening)

Each parameter declared in each module would then have to be assigned one of these labels. The safest thing to do would be to speak to the module developers to understand what label each of their parameters should be assigned, although some of them may be inferred from references/context.

Priors could be collected at the same time or in a second phase.

Thanks very much for taking this on @mnjowe. Very excited to see this develop.

I agree with @marghe-molaro's suggestion. Just to add that I think this entails adapting the Parameter class, as a first PR, with some sensible defaults. A second PR could be module developers adding this definitions to their modules.

I think it also seems like a good time to define a range that the parameter could take: a LOW and a HIGH value, for instance. I think this could become a way to declare a primitive prior, but even without much thought it would enable some sense-checking and help us define some sensible range for sweeping across these parameters.

[mnjowe]

Thanks @marghe-molaro for these helpful suggestions on how to get started, and @tbhallett on the first steps.
A quick question @tbhallett. If this is to take effect, do we want to assign a default value for the label argument or not? The former will not enforce defining a label when modelers define parameters for disease modules, while the latter will.

[marghe-molaro]

Hi @mnjowe, a quick solution for this would be for now to allow for another categorical to the list of labels, namely "unassigned" (so "free", "constant", "context_specific", or "unassigned").

By default all label values can be set to "unassigned", and then we can find a way to overwrite that. It would also allow us to keep tabs on how many parameters are still "unassigned" as values are filled by developers.

[mnjowe]

Great! Let's get started then. I will make a first PR as Tim suggested and then we continue our discussion from there

[marghe-molaro]

Amazing! Really exciting to see this work started!