adds psi selection docs (#326)

* first draft of psi selection docs

* Correct bug in PSI and first draft for documentation

* add test on shuffled dataframe and documentation

* Add info to index and fix unexpected identation

* fix unexpected identation

* fix unexpected identation

* remove some bullets

* add blank

* add blank

* add blank

* set to one row

* test bullets

* adjust doc examples

* test equations

* Move the math def of PSI from docstring to documentation

* correct typo

* modifies api selection index to split tables and add psi class

* reorganises order of selectors in index

* minor edits to psi class docstrings

* edits user guide index

* updates drop psi user guide

* first update of code examples

* final edits to docs

* fix style

* address some TODO from the PR

* Fine tune examples

* Add plot distribution for case 1

* Add code and test for cdf plot

* Add image distribution case 4

* add plot distribution PSI case 5

* add image distribution shift case 3

* Add code and comment for the PSI distribution shift

* extend tests to make explicit diff between categ and list of categ

* remove TODO

* remove - to see if it passes doc test

* remove line 68 to see if it passes doc test

* minor wording changes

* Changes related to last review

Co-authored-by: Gilles Verbockhaven <[email protected]>
Co-authored-by: gverbock <[email protected]>

Author: 3 people

README.md

@@ -117,6 +117,7 @@ transforming parameters from the data and then transform it.
  * SelectByTargetMeanPerformance
  * RecursiveFeatureElimination
  * RecursiveFeatureAddition
+ * DropHighPSIFeatures
 
 ### Preprocessing
  * MatchVariables

docs/api_doc/selection/DropHighPSIFeatures.rst

@@ -0,0 +1,6 @@
+DropHighPSIFeatures
+===================
+
+
+.. autoclass:: feature_engine.selection.DropHighPSIFeatures
+    :members:

docs/api_doc/selection/index.rst

@@ -6,10 +6,18 @@ Feature Selection
 
 Feature-engine's feature selection transformers are used to drop subsets of variables,
 or in other words, to select subsets of variables. Feature-engine hosts selection
-algorithms that are in general, not available in other libraries. These algorithms have
+algorithms that are, in general, not available in other libraries. These algorithms have
 been gathered from data science competitions or used in the industry.
 
-**Summary of Feature-engine's selectors main characteristics**
+Feature-engine's transformers select features based on 2 strategies. They either select
+features by looking at the features intrinsic characteristics, like distributions or their
+relationship with other features. Or they select features based on their impact on the
+machine learning model performance.
+
+In the following tables you find the algorithms that belong to either category.
+
+Selection based on feature characteristics
+------------------------------------------
 
 ============================================ ======================= ============= ====================================================================================
     Transformer                                Categorical variables   Allows NA	    Description
@@ -19,13 +27,23 @@ been gathered from data science competitions or used in the industry.
 :class:`DropDuplicateFeatures()`                √	                      √             Drops features that are duplicated
 :class:`DropCorrelatedFeatures()`               ×	                      √	            Drops features that are correlated
 :class:`SmartCorrelatedSelection()`	            ×	                      √	            From a correlated feature group drops the less useful features
+:class:`DropHighPSIFeatures()`	                ×	                      √	            Drops features with high Population Stability Index
+============================================ ======================= ============= ====================================================================================
+
+Selection based on model performance
+------------------------------------
+
+============================================ ======================= ============= ====================================================================================
+    Transformer                                Categorical variables   Allows NA	    Description
+============================================ ======================= ============= ====================================================================================
 :class:`SelectByShuffling()`	                ×	                      ×	            Selects features if shuffling their values causes a drop in model performance
 :class:`SelectBySingleFeaturePerformance()`	    ×	                      ×	            Removes observations with missing data from the dataset
 :class:`SelectByTargetMeanPerformance()`        √                         ×             Using the target mean as performance proxy, selects high performing features
 :class:`RecursiveFeatureElimination()`          ×                         ×             Removes features recursively by evaluating model performance
 :class:`RecursiveFeatureAddition()`             ×                         ×             Adds features recursively by evaluating model performance
 ============================================ ======================= ============= ====================================================================================
 
+
 .. toctree::
    :maxdepth: 2
    :hidden:
@@ -35,21 +53,21 @@ been gathered from data science competitions or used in the industry.
    DropDuplicateFeatures
    DropCorrelatedFeatures
    SmartCorrelatedSelection
+   DropHighPSIFeatures
    SelectByShuffling
    SelectBySingleFeaturePerformance
    SelectByTargetMeanPerformance
    RecursiveFeatureElimination
    RecursiveFeatureAddition
 
-
 Other Feature Selection Libraries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+---------------------------------
 
 For additional feature selection algorithms visit the following open-source libraries:
 
 * `Scikit-learn selection <https://scikit-learn.org/stable/modules/feature_selection.html>`_
 * `MLXtend selection <http://rasbt.github.io/mlxtend/api_subpackages/mlxtend.feature_selection/>`_
 
-Scikit-learn hosts multiple filter and embedded methods, that select features based on
+Scikit-learn hosts multiple filter and embedded methods that select features based on
 statistical tests or machine learning model derived importance. MLXtend hosts greedy
 (wrapper) feature selection methods.

docs/images/PSI_distribution_case1.png

@@ -166,12 +166,14 @@ Feature Selection:
 - :doc:`api_doc/selection/DropDuplicateFeatures`: drops duplicated variables from a dataframe
 - :doc:`api_doc/selection/DropCorrelatedFeatures`: drops correlated variables from a dataframe
 - :doc:`api_doc/selection/SmartCorrelatedSelection`: selects best features from correlated groups
+- :doc:`api_doc/selection/DropHighPSIFeatures`: selects features based on the Population Stability Index (PSI)
 - :doc:`api_doc/selection/SelectByShuffling`: selects features by evaluating model performance after feature shuffling
 - :doc:`api_doc/selection/SelectBySingleFeaturePerformance`: selects features based on their performance on univariate estimators
 - :doc:`api_doc/selection/SelectByTargetMeanPerformance`: selects features based on target mean encoding performance
 - :doc:`api_doc/selection/RecursiveFeatureElimination`: selects features recursively, by evaluating model performance
 - :doc:`api_doc/selection/RecursiveFeatureAddition`: selects features recursively, by evaluating model performance
 
+
 Preprocessing:
 --------------