crepes 0.9.0

v0.9.0 (8/10/2025)

Features

• Conformal test martingales have been included in the new module crepes.martingales, which defines the betting functions EpsilonBettingFunction and StepBettingFunction (with and without drift) as well as the martingales SimpleJumper, SleeperStayer, SleeperDrifterand CompositeMartingale, which can be used in combination with any betting function. The new module also includes the method semi_online_p_values for generating (smoothed) p-values from non-conformity scores in the semi-online mode. For more details, please consult the updated documentation and the newly added Jupyter notebook martingales_nb.ipynb.

Fixes

• A minor bug in the documentation has been fixed.

crepes 0.8.0

v0.8.0 (31/03/2025)

Features

• Semi-online inductive conformal classifiers, regressors and predictive systems, which iteratively update the calibration set after making each prediction, have been incorporated. For WrapClassifier, the methods predict_p, predict_set and evaluate, now include the argument online together with the argument warm_start, where the former enables online calibration (disabled by default) and the latter enables extending the original calibration set during online calibration (enabled by default). Similarly, for WrapRegressor, the methods predict_p, predict_int, predict_percentiles, predict_cpds and evaluate include the arguments online and warm_start. Semi-online conformal predictors are also enabled by the methods predict_p_online (for ConformalClassifier, ConformalRegressor, and ConformalPredictiveSystem), predict_set_online (for ConformalClassifier), predict_int_online (for ConformalRegressor and ConformalPredictiveSystem), predict_percentiles_online and predict_cpds_online (for ConformalPredictiveSystem). Moreover, the evaluate method for all three types of conformal predictor now includes an additional argument online, which if set to true enables online calibration.

• The method predict_p for generating (smoothed or non-smoothed) p-values using conformal regressors has been added.

• New methods for conformal predictive systems have been added; predict_p for obtaining p-values, predict_int for obtaining prediction intervals, predict_percentiles for obtaining percentiles, and predict_cpds for obtaining conformal predictive distributions. These outputs can alternatively (as before) be generated by the methods predict (for ConformalPredictiveSystem) and predict_cps (for WrapRegressor).

• Both smoothed and non-smoothed p-values may now be output for conformal predictive systems; smoothing is the default but can be disabled by setting smoothing=False for the methods predict_p and predict.

• The evaluate method of ConformalClassifier, ConformalRegressor, and ConformalPredictiveSystem, as well as of WrapClassifier and WrapRegressor, now includes the metric ks_test, which provides the p-value for the Kolmogorov-Smirnov test of uniformity of predicted p-values. Thanks to @egonmedhatten for the suggestion!

Fixes

• Fitted objects now contain the attribute fitted_ to allow for proper handling in scikit-learn pipelines. Thanks to @lukethomrichardson for suggesting the fix.

crepes 0.7.1

v0.7.1 (21/09/2024)

Features

• The calibrate methods of the classes WrapClassifier and WrapRegressor now take an additional argument seed, for setting the state of the random number generator. This allows for predict_p and predict_set of the former class and predict_int and predict_cps of the latter class as well as evaluate for both classes to become deterministic. The methods predict_p, predict_set, predict_int ,predict_cps and evaluate of these classes also have an argument seed, which can be used to over-ride any setting by the calibrate method. In addition, the corresponding methods of the classes ConformalClassifier and ConformalPredictiveSystem also include the argument seed for the same purpose. (The methods of ConformalRegressor currently contains no stochastic components and there is hence no need for a seed.) Thanks to @egonmedhatten and @tuvelofstrom for suggesting this extension.

• The predict_p methods of the classes ConformalClassifier and WrapClassifier now takes an optional argument smoothing to allow for generating both smoothed and non-smoothed p-values (default: smoothing=True).

• The default value for the parameter smoothing of the predict_set and evaluate methods has been changed to True.

• The class DifficultyEstimator in crepes.extras now includes a parameter f for providing a function to compute the difficulty estimates.

Fixes

• The documentation for the class MondrianCategorizer in crepes.extras has been corrected.

crepes 0.7.0

v0.7.0 (27/06/2024)

Features

• The class MondrianCategorizer was added to crepes.extras, for generating categories to be used by Mondrian conformal classifiers, regressors and predictive systems. See the documentation for the interface to objects of the class through the fit and apply methods.

• The class WrapRegressor has been updated so that the method calibrate allows for specifying an (optional) difficulty estimator and (optional) Mondrian categorizer, which are used both for calibration and making predictions, instead of requiring that difficulty estimates and Mondrian categories are provided separately for these tasks. The methods predict_int, predict_cps and evaluate no longer require sigmas and bins to be provided for normalized and Mondrian conformal regressors and predictive systems. Thanks to @tuvelofstrom for suggestions along these lines.

• The class WrapClassifier has been updated so that the method calibrate allows for specifying an (optional) Mondrian categorizer, which is used both for calibration and making predictions. The methods predict_p, predict_set and evaluate no longer require bins to be provided for Mondrian conformal classifiers.

Fix

• Label vectors represented by pandas.Series are converted to NumPy arrays to avoid indexing issues. Thanks to @valeman for pointing this out.

crepes 0.6.2

v0.6.2 (02/02/2024)

Fixes

• Fixed deprecated code for checking if an array is non-empty in the ConformalPredictiveSystem class. Thanks to @tuvelofstrom for pointing this out.

crepes 0.6.1

v0.6.1 (21/08/2023)

Features

• The function margin for computing non-conformity scores for conformal classifiers has been added to crepes.extras.

Fixes

• Fixed a bug in the DifficultyEstimator class (in crepes.extras), which caused an error when trying to display a non-fitted object. Thanks to @tuvelofstrom for pointing this out.

• Fixed an error in the documentation for the function hinge.

• The Jupyter notebooks crepes_nb_wrap.ipynb and crepes_nb.ipynb have been updated to illustrate the new margin function.

crepes 0.6.0

v0.6.0 (28/06/2023)

Features

• The classes ConformalClassifier and WrapClassifier have been added to crepes, allowing for generation of standard and Mondrian conformal classifiers, which produce p-values and prediction sets. The calibrate method of WrapClassifier allows for easily generating class-conditional conformal classifiers and using out-of-bag calibration. See the documentation for the interface to objects of the class through the calibrate, predict_p and predict_set methods, in addition to the fit, predict and predict_proba methods of the wrapped learner. The method evaluate allows for evaluating the predictive performance using a set of standard metrics.

• The function hinge for computing non-conformity scores for conformal classifiers has been added to crepes.extras.

Fixes

• The class Wrap has changed name to WrapRegressor and the arguments to the calibrate method of this class have been changed to be in line with the calibrate method of WrapClassifier.

• The Jupyter notebooks crepes_nb_wrap.ipynb and crepes_nb.ipynb have been updated and extended

crepes 0.5.1

v0.5.1 (22/06/2023)

Fix

• Fixed a bug in the evaluate method of ConformalPredictiveSystem, which caused an error when using CRPS as a single metric, i.e., when providing metrics=["CRPS"] as input. Thanks to @Zeeshan-Khaliq for pointing this out.

crepes 0.5.0

Feature

• The full cpds matrix is calculated only if requested to be output (return_cpds=True) by the predict method of ConformalPredictiveSystem or if the set of metrics include "CRPS" for the evaluate method. This allows large test and calibration sets to be handled without excessive use of memory in other cases. Thanks to @christopherjluke and @SebastianLeborg for highlighting and discussing the problem.

Fixes

• Default values for mandatory arguments for the methods fit, predict and evaluate methods of ConformalRegressor and ConformalPredictiveSystem, as well as the function binning in crepes.extras, are no longer provided

• y_min and y_max correctly inserted for all percentiles

• The evaluate method for ConformalPredictiveSystem fixed to work correctly even if CRPS not included in metrics, and if all test objects belong to the same Mondrian category

• Incorrect values for percentiles will render an error message

crepes 0.4.0

v0.4.0 (16/05/2023)

Feature

• The class Wrap has been added to crepes, allowing for easily extending the underlying learner with methods for forming, and making predictions with, conformal regressors and predictive systems. See the documentation for the interface to objects of the class through the calibrate, predict_int and predict_cps methods, in addition to the fit and predict methods of the wrapped learner.

Fixes

• A Jupyter notebook crepes_nb_wrap.ipynb has been added to the documentation to illustrate the use of the Wrap class.

• The output result array of a conformal predictive system is converted to a vector if the array contains one column only.

• The documentation has been updated and now includes links to classes and methods.

• crepes.fillings has been renamed to crepes.extras