[ENH] Preprocess: implement Select Relevant Feature's percentile

[matejklemen]

Issue

Fixes #3226.

Description of changes

Implements the "Percentile" option for Select Relevant Features preprocessor.
The option enables the selection of all features that fall >= the entered percentile. The rank of the first feature to be taken is computed using nearest-rank method. There are 2 "special" cases: 0th percentile is defined as all the features and 100th percentile is defined as the best feature.

I've also renamed "Strategy" to "Number of features" in Select Random Features and Select Relevant Features as was suggested in the original issue.

Includes

• Code changes
• Tests
• Documentation

[janezd]

__call__ shouldn't change the state of the object (unless when caching, for instance). In this case, imagine you have

fss = SelectBestFeatures(k=0.5)
data1 = <some data set with 10 attributes>
data2 = <some data set with 100 attributes>
fss(data1)  # this sets fss.k to 5!
fss(data2)  # and so this selects just 5 instead of 50 attributes

One option is to introduce

            effective_k = min(n_attrs - idx_attr + 1, n_attrs)
        else:
            effective_k = self.k

and replace further occurrences of self.k in this method with effective_k.

[matejklemen]

Good catch, I didn't think about that.

I've fixed it (done the same thing for selection of random features as well, since it had the same problem) and added a test for this.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@2056481). Click here to learn what that means.
The diff coverage is 80%.

@@            Coverage Diff            @@
##             master    #3588   +/-   ##
=========================================
  Coverage          ?   83.98%           
=========================================
  Files             ?      370           
  Lines             ?    66981           
  Branches          ?        0           
=========================================
  Hits              ?    56253           
  Misses            ?    10728           
  Partials          ?        0

[codecov]

Codecov Report

Merging #3588 into master will increase coverage by 0.04%.
The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master    #3588      +/-   ##
==========================================
+ Coverage   84.02%   84.06%   +0.04%     
==========================================
  Files         370      370              
  Lines       67232    67252      +20     
==========================================
+ Hits        56489    56538      +49     
+ Misses      10743    10714      -29

[janezd]

I added another commit to switch from percentiles to proportion, so that k (in the class) and spin boxes (in the widget) have parallel meanings, and that higher numbers always mean more attributes.

[janezd]

I changed percentiles to proportions, so that k (in the class) and combos (in widget) have parallel meaning (more is always more).