Merge pull request #504 from scikit-learn-contrib/493-add-mondrian-cp

493 add mondrian cp

Author: vincentblot28

HISTORY.rst

@@ -5,6 +5,7 @@ History
 0.8.x (2024-xx-xx)
 ------------------
 
+* Add Mondrian Conformal Prediction for regression and classification
 * Add `** predict_params` in fit and predict method for Mapie Regression
 * Update the ts-changepoint notebook with the tutorial
 * Change import related to conformity scores into ts-changepoint notebook

doc/Makefile

@@ -52,6 +52,7 @@ clean:
 	-rm -rf examples_classification/
 	-rm -rf examples_multilabel_classification/
 	-rm -rf examples_calibration/
+	-rm -rf examples_mondrian/
 	-rm -rf generated/*
 	-rm -rf modules/generated/*
 

doc/api.rst

@@ -108,3 +108,13 @@ Resampling
 
    subsample.BlockBootstrap
    subsample.Subsample
+
+
+Mondrian
+==========
+
+.. autosummary::
+   :toctree: generated/
+   :template: class.rst
+
+   mondrian.MondrianCP

doc/conf.py

@@ -316,13 +316,15 @@
         "../examples/regression",
         "../examples/classification",
         "../examples/multilabel_classification",
-        "../examples/calibration"
+        "../examples/calibration",
+        "../examples/mondrian",
     ],
     "gallery_dirs": [
         "examples_regression",
         "examples_classification",
         "examples_multilabel_classification",
-        "examples_calibration"
+        "examples_calibration",
+        "examples_mondrian",
     ],
     "doc_module": "mapie",
     "backreferences_dir": os.path.join("generated"),

doc/images/mondrian.png

@@ -49,6 +49,14 @@
    examples_multilabel_classification/1-quickstart/plot_tutorial_multilabel_classification
    notebooks_multilabel_classification
 
+.. toctree::
+   :maxdepth: 2
+   :hidden:
+   :caption: MONDRIAN
+
+   theoretical_description_mondrian
+   examples_mondrian/1-quickstart/plot_main-tutorial-mondrian-regression
+
 .. toctree::
    :maxdepth: 2
    :hidden:

doc/index.rst

@@ -0,0 +1,46 @@
+.. title:: Theoretical Description Mondrian : contents
+
+.. _theoretical_description_mondrian:
+
+#######################
+Theoretical Description
+#######################
+
+Mondrian conformal prediction (MCP) [1] is a method that allows to build prediction sets with a group-conditional
+coverage guarantee.  The coverage guarantee is given by:
+
+.. math::
+    P \{Y_{n+1} \in \hat{C}_{n, \alpha}(X_{n+1}) | G_{n+1} = g\} \geq 1 - \alpha
+
+where :math:`G_{n+1}` is the group of the new test point :math:`X_{n+1}` and :math:`g`
+is a group in the set of groups :math:`\mathcal{G}`.
+
+MCP can be used with any split conformal predictor and can be particularly useful when one have a prior 
+knowledge about existing groups wheter the information is directly included in the features
+of the data or not.
+In a classifcation setting, the groups can be defined as the predicted classes of the data. Doing so,
+one can ensure that, for each predicted class, the coverage guarantee is satisfied.
+
+In order to achieve the group-conditional coverage guarantee, MCP simply classifies the data
+according to the groups and then applies the split conformal predictor to each group separately.
+
+The quantile of each group is defined as:
+
+.. math::
+   \widehat{q}^g =Quantile\left(s_1, ..., s_{n^g} ,\frac{\lceil (n^{(g)} + 1)(1-\alpha)\rceil}{n^{(g)}} \right)
+
+Where :math:`s_1, ..., s_{n^g}` are the conformity scores of the training points in group :math:`g` and :math:`n^{(g)}`
+is the number of training points in group :math:`g`.
+
+The following figure (from [1]) explains the process of Mondrian conformal prediction:
+
+.. image:: images/mondrian.png
+   :width: 600
+   :align: center
+
+References
+----------
+
+[1] Vladimir Vovk, David Lindsay, Ilia Nouretdinov, and Alex Gammerman.
+Mondrian confidence machine.
+Technical report, Royal Holloway University of London, 2003