You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-01-30-conformal-prediction.md
+10-4Lines changed: 10 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -89,9 +89,7 @@ For an updated use case, see [8]. This method allows us to quantify and communic
89
89
90
90
The approach is flexible, model-agnostic, and works across various scenarios. It uses calibration data, non-conformity measures, and prediction intervals to ensure reliable predictions, with specified confidence levels (e.g., 95%). However, challenges include ensuring proper calibration across different datasets and accounting for the assumption of interchangeability in the data.
91
91
92
-
Conformal prediction is particularly useful in high-stakes fields, offering a clearer understanding of prediction limits and helping make more informed decisions.
93
-
94
-
Let’s look at two example frameworks...
92
+
Let’s look at two exemplar frameworks...
95
93
96
94
### Classification Example
97
95
@@ -105,7 +103,7 @@ this indicates that the model does not have enough certainty to differentiate be
105
103
106
104
### Regression Example
107
105
108
-
In regression, conformal prediction generates prediction intervals. For example, if the model estimates that the price of a stock will be:
106
+
In regression, conformal prediction generates intrinsic prediction intervals. For example, if the model estimates that the price of a stock will be:
109
107
110
108
$$
111
109
\hat{Y} = 100
@@ -119,6 +117,12 @@ $$
119
117
120
118
this means that the true price will fall within this interval with a probability of $$1 - \alpha$$.
121
119
120
+
Conformal prediction is particularly useful in high-stakes fields, offering a clearer understanding of prediction limits and helping make more informed decisions.
121
+
122
+
AI is transforming **healthcare** by providing non-invasive methods to assess critical health metrics, such as BMI, blood glucose levels, and other factors linked to heart disease and diabetes. However, traditional AI models often produce single-value predictions that may not fully capture the complexity of individual health risks. CP addresses this limitation by incorporating uncertainty bands, offering doctors a clearer range of potential outcomes. This approach enhances diagnostic accuracy, builds patient trust, and ensures fairer decision-making across diverse populations. Recently, CP has enabled more reliable obesity risk assessments [10], outperforming conventional methods. By analyzing age, height, weight, ethnicity, and education level, the model estimates waist circumference with 95% accuracy—without physical measurement. This advancement could revolutionize obesity care by replacing BMI with a more precise predictor of health risks. As researchers refine the model to include lifestyle factors like diet and exercise, AI-driven healthcare is becoming not just more innovative, but also safer and more equitable.
123
+
124
+
As of today, this is not the time to reinvent the wheel unless we are willing to dive into a very low-level approach, even if we understand the mathematical intricacies of CP. To get started, we can mention some **libraries** and **tools** (ordered by preference based on what I was able to research in less than half a day) to experiment with everything related to this framework, especially in time series and forecasting. Examples include: CatBoost, XGBoost, LightGBM, Prophet, AutoTS, NBEATs, ARIMA, the Theta Model, AutoTS and the Holt-Winters method.
125
+
122
126
# Conclusion
123
127
124
128
Conformal prediction provides an accurate approach to measuring uncertainty in machine learning models, offering a confidence range based on past experience. This approach ensures that the real value has a high probability of falling within this range, making the prediction robust and reliable. While it can be computationally expensive in some cases, its variants allow it to be applied in practical situations.
@@ -142,3 +146,5 @@ Conformal prediction provides an accurate approach to measuring uncertainty in m
142
146
[8][Conformal Prediction: An introduction to measuring uncertainty](https://www.bbvaaifactory.com/conformal-prediction-an-introduction-to-measuring-uncertainty/)
0 commit comments