1
+
1
2
Estimating prediction sets on the Cifar10 dataset
2
3
=================================================
3
4
@@ -9,15 +10,15 @@ What is done in this tutorial ?
9
10
10
11
..
11
12
12
- - Use :class: `` mapie.classification.MapieClassifier` ` to compare the
13
+ - Use :class: `mapie.classification.MapieClassifier ` to compare the
13
14
prediction sets estimated by several conformal methods on the
14
15
Cifar10 dataset.
15
16
16
17
- Train a small CNN to predict the image class
17
18
18
19
..
19
20
20
- - Create a custom class `` TensorflowToMapie ` ` to resolve adherence
21
+ - Create a custom class `TensorflowToMapie ` to resolve adherence
21
22
problems between Tensorflow and Mapie
22
23
23
24
Tutorial preparation
@@ -63,15 +64,15 @@ Tutorial preparation
63
64
1. Data loading
64
65
---------------
65
66
66
- The Cifar10 dataset is downloaded from the `` Tensorflow Datasets ` `
67
+ The Cifar10 dataset is downloaded from the `Tensorflow Datasets `
67
68
library. The training set is then splitted into a training, validation
68
69
and a calibration set which will be used as follow:
69
70
70
71
- **Training set **: used to train our neural network.
71
72
- **Validation set **: used to check that our model is not
72
73
overfitting.
73
74
- **Calibration set **: used to calibrate the conformal scores in
74
- :class: `` mapie.classification.MapieClassifier` `
75
+ :class: `mapie.classification.MapieClassifier `
75
76
76
77
.. code-block :: python
77
78
@@ -318,11 +319,11 @@ a perfect classifier.
318
319
model.compile(loss = loss, optimizer = optimizer, metrics = metrics)
319
320
return model
320
321
321
- Training the algorithm with a custom class called `` TensorflowToMapie ` `
322
+ Training the algorithm with a custom class called `TensorflowToMapie `
322
323
-----------------------------------------------------------------------
323
324
324
- As MAPIE asked that the model has a `` fit `` , `` predict_proba ` `,
325
- `` predict ` ` class attributes and that the information about if whether
325
+ As MAPIE asked that the model has a `fit `, `predict_proba `,
326
+ `predict ` class attributes and that the information about if whether
326
327
or not the model is fitted.
327
328
328
329
.. code-block :: python
@@ -495,7 +496,7 @@ or not the model is fitted.
495
496
------------------------------------
496
497
497
498
We will now estimate the prediction sets with the five conformal methods
498
- implemented in :class: `` mapie.classification.MapieClassifier` ` for a
499
+ implemented in :class: `mapie.classification.MapieClassifier ` for a
499
500
range of confidence levels between 0 and 1.
500
501
501
502
.. code-block :: python
@@ -1040,7 +1041,7 @@ smaller than that of the naive method.
1040
1041
1041
1042
.. code-block :: python
1042
1043
1043
- def plot_confusion_matix (method : str , y_ps : Dict[str , np.ndarray], label_names : List) -> None :
1044
+ def plot_confusion_matrix (method : str , y_ps : Dict[str , np.ndarray], label_names : List) -> None :
1044
1045
"""
1045
1046
Plot the confusion matrix for a specific method.
1046
1047
@@ -1075,7 +1076,7 @@ smaller than that of the naive method.
1075
1076
1076
1077
.. code-block :: python
1077
1078
1078
- plot_confusion_matix (" cumulated_score" , y_ps_90, label_names)
1079
+ plot_confusion_matrix (" cumulated_score" , y_ps_90, label_names)
1079
1080
1080
1081
1081
1082
@@ -1091,7 +1092,7 @@ label is quite often within the prediction set while the deer is not
1091
1092
1092
1093
.. code-block :: python
1093
1094
1094
- plot_confusion_matix (" naive" , y_ps_90, label_names)
1095
+ plot_confusion_matrix (" naive" , y_ps_90, label_names)
1095
1096
1096
1097
1097
1098
@@ -1100,7 +1101,7 @@ label is quite often within the prediction set while the deer is not
1100
1101
1101
1102
.. code-block :: python
1102
1103
1103
- plot_confusion_matix (" score" , y_ps_90, label_names)
1104
+ plot_confusion_matrix (" score" , y_ps_90, label_names)
1104
1105
1105
1106
1106
1107
0 commit comments