Move to objective based interface

[rick12000]

Change Log:

API

Radical redesign of the package's entry point:

• ConformalSearcher class becomes ConformalTuner.
• Removal of model abstraction:
  • ConformalSearcher was instantiated with a model object and corresponding X and y data. This limited flexibility, since the model object had to either be a scikit-learn BaseEstimator or a TunableModel (abstract class with fit and predict methods) and the way in which the model object interacted with the data was pre-defined and internal to the package.
  • ConformalTuner is instantiated with an objective_function callable, removing any ownership of the model definition or data structure from the package. The end user defines how the model or other entity of interest returns the objective metric to optimize directly in the objective function.
• Robust search space ranges:
  • ConformalSearcher's search_space input mapped hyperparameter key names to list of values to search, then created exhaustive combinations during search.
  • New search_space maps names to IntRange, FloatRange or CategoricalRange value ranges, allowing for better type safety and user experience.
• Miscellaneous method and input changes:
  • ConformalSearcher's .search() method renamed to .tune().
  • Removal of string configuration for conformal_search_estimator, in favour of more direct access to conformal acquisition objects via searcher: Optional[QuantileConformalSearcher], with default handling to versatile searcher.
  • Smaller changes not listed.

Methodology

• Default optimistic UCB acquisition function is now accompanied by suite of alternative acquisition functions:
  • Scheduled UCB: LowerBoundSampler class.
  • Thompson Sampling: ThompsonSampler class.
  • Optimistic Bayesian Sampling: ThompsonSampler class with enable_optimistic_sampling = True
  • Expected Improvement: ExpectedImprovementSampler class.
• Added support for ensemble surrogate models via QuantileEnsembleEstimator class.
• Added Dynamically Tuned Adaptive Conformal Intervals (DtACI) via DtACI class, for superior local coverage performance.
  • Adaptive Conformal Intervals (ACI) are now reproducible as a special case of DtACI, rather than a standalone class.
• Via dynamic_sampling input in ConformalTuner, added option to either search a fixed number of hyperparameter configurations upfront, or random sample from some population at each search iteration, rather than previous default behaviour of fixed upfront number.
• Removed all usage of LocallyWeightedConformalRegression and dependant logic, with no replacement. Locally Weighted Conformal Regression support is terminated, with Conformalized Quantile Regression remaining as the default framework (this is primarily for maintainability, but CQR generally outperforms LWCR).

Other

• Added GitHub Actions CICD flow for:
  • Linting.
  • Unit testing on all supported python versions.
  • Packaging.
  • Smoke testing on TestPyPI.
  • Uploading to PyPI.
  • Creating release.
• Added documentation site in docs/ directory.