[DOC] Section 1 of user guide/definition of concepts

docs/src/concepts.rst

@@ -3,4 +3,116 @@
 
 ======================
 Definition of concepts
-======================
+======================
+
+Variable Importance
+-------------------
+
+Global Variable Importance (VI) aims to assign a measure of
+relevance to each feature :math:`X^j` with respect to a target  :math:`y` in the
+data-generating process. In Machine Learning, it can be seen as a measure
+of how much a variable contributes to the predictive power of a model. We 
+can then define "important" variables as those whose absence degrades 
+the model's performance :footcite:p:`Covert2020`.
+
+So if ``VI`` is a variable importance method, ``X`` a variable matrix and ``y`` 
+the target variable, the importance of all the variables
+can be estimated as follows:
+
+.. code-block::
+
+    # instantiate the object
+    vi = VI()
+    # fit the models in the method
+    vi.fit(X, y)
+    # compute the importance and the pvalues
+    importance = vi.importance(X, y)
+    # get importance for each feature
+    importance = vi.importances_
+
+It allow us to rank the variables from more to less important.                            
+
+Here, ``VI`` can be a variable importance method implemented in HiDimStat,
+such as :class:`hidimstat.LOCO` (other methods will support the same API 
+soon).
+
+Variable Selection
+-------------------------------
+
+(Controlled) Variable selection is then the next step that entails filtering out the 
+significant features in a way that provides statistical guarantees, 
+e.g. type-I error or False Discovery Rate (FDR).
+
+For example, if we want to select the variables with a p-value lower than 
+a threshold ``p``, we can do:
+
+.. code-block::
+
+    # selection of the importance and pvalues
+    vi.selection(threshold_pvalue=p)
+
+This step is important to make insighful discoveries. Even if variable 
+importance provides a ranking, due to the estimation step, we need
+statistical control to do reliable selection.
+
+
+
+
+Types of VI methods
+-------------------
+
+There are two main types of VI methods implemented in HiDimStat:
+
+1. Marginal methods: these methods provide importance to all the features 
+that are related to the output, even if it is caused by spurius correlation. They 
+are related with testing if :math:`X^j\perp\!\!\!\!\perp Y`.
+Example of such methods is LOCI.
+
+2. Conditional methods: these methods assign importance only to features that
+provide exclusive information beyond what is already captured by the others, 
+i.e., they contribute unique knowledge. They are related with Conditional 
+Independence Testing, which consist in testing if 
+:math:`X^j\perp\!\!\!\!\perp Y\mid X^{-j}`. Examples of such methods are
+:class:`hidimstat.LOCO` and :class:`hidimstat.CFI`.
+
+
+Generally, conditional methods address the issue of false positives that often
+arise with marginal methods, which may assign importance to variables just 
+because they are correlated with truly important ones. By focusing on unique 
+contributions, conditional methods help preserve parsimony, yielding a smaller 
+and more meaningful subset of important features. However, in certain cases, the 
+distinction between marginal and conditional methods can be more subtle. See 
+:ref:`sphx_glr_generated_gallery_examples_plot_conditional_vs_marginal_xor_data.py` 
+
+
+High-dimension and correlation
+-----------------------------------
+
+In high-dimensional and highly correlated settings, estimation becomes 
+particularly challenging, as it is difficult to clearly distinguish important 
+features from unimportant ones. For such problems, a preliminary filtering step 
+can be applied to avoid having duplicate or redundant input features, or 
+alternatively, one can consider grouping them :footcite:p:`Chamma_AAAI2024` . 
+Grouping consists of treating together features that represent the same 
+underlying concept. This approach extends naturally to many methods, 
+for example :class:`hidimstat.CFI`.
+
+
+
+Statistical Inference
+---------------------
+
+Given the variability inherent in estimation, it is necessary to apply 
+statistical control to the discoveries made. Simply selecting the most important 
+features without such control is not valid. Different forms of guarantees can 
+be employed, such as controlling the type-I error or the False Discovery Rate. 
+This step is directly related to the task of Variable Selection.
+
+
+
+
+
+References
+----------
+
+.. footbibliography::

docs/tools/references.bib

@@ -135,6 +135,16 @@ @article{chevalier_statistical_2020
   year    = {2020}
 }
 
+@inproceedings{Covert2020,
+  title     = {Understanding {{Global Feature Contributions With Additive Importance Measures}}},
+  booktitle = {Advances in {{Neural Information Processing Systems}}},
+  author    = {Covert, Ian and Lundberg, Scott M and Lee, Su-In},
+  year      = {2020},
+  volume    = {33},
+  pages     = {17212--17223},
+  publisher = {Curran Associates, Inc.}
+}
+
 @article{eshel2003yule,
   author  = {Eshel, Gidon},
   journal = {Internet resource},
@@ -348,3 +358,4 @@ @article{zhang2014confidence
   volume    = {76},
   year      = {2014}
 }
+