Hello @gmartinonQM, I have some good news! I have implemented the split conformal variant in my fork here: https://github.com/RudrakshTuwani/MAPIE?organization=RudrakshTuwani&organization=RudrakshTuwani

Changes in files:

mapie/regression.py - Added class MapieCovShiftRegressor.
mapie/dre.py - Added classes DensityRatioEstimator and ProbClassificationDRE.
mapie/utils.py - Added a function empirical_quantile for calculating weighted empirical quantile.

The replication of paper results is at examples/regression/4-covariate-shift/paper_replication.ipynb. There are a couple of differences but they are mostly due to different model defaults between Python and R. In general, we see that MapieCovShiftRegressor is able to adapt to covariate shift with oracle as well as estimated density ratios.

There's still work to be done. Right now I have just limited the functionality of MapieCovShiftRegressor to cv="prefit" and method="base". I can try working on adapting the class to other scenarios. It will likely be non-trivial due to density ratio estimation and I think I would need to spend some time thinking about how to do it in a way that doesn't violate exchangeability. If you feel the current limitations are not significant, I can also spend time on writing tests and documentation.

Let me know what you think! Thank you :)

[ENHANCEMENT] Covariate shift conformal #72

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions