`D0CRT`: Merge `screening` and `screening_threshold` into one parameter.

[jpaillard]

Currently, dcrt takes 2 arguments for screening: screening=True, screening_threshold=1e-1.
I suggest replacing with 1 argument screening: float = None that corresponds to the screening_threshold, and if None, no screening is performed.

[lionelkusch]

The screening is partly done without the boolean. See the position of the following line 150:

hidimstat/src/hidimstat/dcrt.py

Lines 145 to 154 in ba3b953

	non_selection = np.where(
	np.abs(coef_X_full)
	<= np.percentile(np.abs(coef_X_full), 100 - screening_threshold)
	)[0]
	# optimisation to reduce the number of elements different to zeros
	coef_X_full[non_selection] = 0.0
	# select the variables for the screening
	if screening:
	selection_set = np.setdiff1d(np.arange(n_features), non_selection)

[bthirion]

Agreed !

[lionelkusch]

The solution proposed by Joseph can't be done directly without moving line 150.

[jpaillard]

In theory, nothing prevents moving line 150 since if screening_threshold=None, non_selection is empty.

[lionelkusch]

In the actual code, screening_threshold can't be None because there is a subtraction with 1 in 147.

[jpaillard]

in the suggested change I condition this step on screening_threshold not being None

[lionelkusch]

After thinking, the function of screening and screening_threshold are different:

• screening_threshold is a screen over the coefficient of the Lasso
• screening is a screen over the features (based on the coefficient)

We can merge the two functions but the aim is quite different:

• simplified the computation for Lasso
• reduce the number of features

Following the discussion in PR #370, removing the screen procedure of Lasso can be equivalent to univariate method for selection of features as proposed in PR #317 or by scikitlearn in this example.

This is a different problem. I don't know if it's interesting to keep this screening process in the library or just provide an example can be enough.
What do you think?

[bthirion]

Indeed, screening_threshold is sued to sparsity a linear model, while screening is a lasso-based features selection procedure.
However, I thing that the design is too complex: everybody would agree that a variable with a null coefficient in a linear model is not important. On the other hand, I don't really see the point of creating a sparse linear model, if we don't use that for some kind of screening. At the end of the day, I don't see any point in separating the two things.

[lionelkusch]

In this case, it's fine to merge the two.