Psi selector (#311)

* feat: High level structure of the code

* feat: clear code

* move psi code to the right place, simplify psi code and set attribute min value for bin

* feat: adjust to the structure

* add possibility to switch dataframes

* feat: clean code and add tests

* feat: Add comments and doc strings

* bug: fix wrong referencing

* bug: Move binning to feature level and drop na

* correction after run flake8

* correction after run flake8

* pre-commit issue

* run tox

* shorten line with 90 characters

* code layout: remove line with more than 88 character

* adjusments realted to PR

* change doc string of class

* add file

* remove old file

* feat: remove df in init and allow split accoring to all dimensions

* black drop_psi_features

* Black correction

* remove todo

* remove used variables

* feat: ensure the dataframe spit can be done on different types

* add test on quantile calculation for continuous variable

* doc: Adjust docstring

* Correction after 2nd review

* bug fix: correct test for psi_ and move place split_col is removed from features

* Raise error when missing are in the split columns and add PSI formula to docs

* Raise error when missing are in the split columns and add PSI formula to docs

* clean code based on suggestions from PR

* clean code based on suggestions from PR

* adjust the text in the ValueErrors

* adjust docs

* solve mypy type issue

* solve mypy type issue

* solve mypy type issue

* re-words error messages, re-orders params in init

* rewords param docstrings, renames param

* re-words doscstring

* reorganizes code and changes variable names

* rewords dosctrings changes param name

* rewords init

* last changes to main class

* changes series to df

* final changes

* fix bug in split_dataframe

* add cut_off as attribute when it is not defined

* Add series of tests

* add DropPSI to init file

* fix type error on variables

* add tests of column types

* modified init docstrings

* adds space to error message

* removes returns from fit docstring

* revert to previous stance of cut_off assigment

* add info on categorical cut to init docstring

* modify cut-off check

* adds test estimator

* modify test import

* expanded fit attr test

* remove transform from test

* reorders init param tests, modifies split col test

* Changes required to pass check_estimator_selectors

* reorders tests, adds todos for missing tests

* runs isort for sorting imports

* runs black on files

* changes

* add changes from PR

* clarify drift feature in test

* add assertion to cut_off in split_with_different_types

* add test of dataframes in split_using_cut_off

* Compare dataframe is test

* add tests on transform

* add to do on test with missing values

* clean tests

* minor changes in docstrings

* modify wording in error message

* modify wording in error message 2

* changes cut_off got cut_off_

* final changes to psi script

* adds tests and asserts

* Adjust doc string for split_distinct and add unit test

Co-authored-by: Gilles Verbockhaven <[email protected]>
Co-authored-by: Soledad Galli <[email protected]>

Author: 3 people

feature_engine/selection/__init__.py

@@ -5,6 +5,7 @@
 from .drop_correlated_features import DropCorrelatedFeatures
 from .drop_duplicate_features import DropDuplicateFeatures
 from .drop_features import DropFeatures
+from .drop_psi_features import DropHighPSIFeatures
 from .recursive_feature_addition import RecursiveFeatureAddition
 from .recursive_feature_elimination import RecursiveFeatureElimination
 from .shuffle_features import SelectByShuffling
@@ -17,6 +18,7 @@
     "DropConstantFeatures",
     "DropDuplicateFeatures",
     "DropCorrelatedFeatures",
+    "DropHighPSIFeatures",
     "SmartCorrelatedSelection",
     "SelectByShuffling",
     "SelectBySingleFeaturePerformance",