9.10.0

Author: mathias-von-ottenbreit

API_REFERENCE_FOR_REGRESSION.md

@@ -239,6 +239,19 @@ A numpy matrix with predictor values.
 A numpy matrix with predictor values.
 
 
+## Method: calculate_local_contribution_from_selected_terms(X:npt.ArrayLike, predictor_indexes:List[int])
+
+***Returns a numpy vector containing the contribution to the linear predictor from an user specified combination of interacting predictors for each observation in X. This makes it easier to interpret interactions (or main effects if just one predictor is specified), for example by plotting predictor values against the term contribution.***
+
+### Parameters
+
+#### X
+A numpy matrix with predictor values.
+
+#### predictor_indexes
+A list of integers specifying the indexes of predictors in X to use. For example, [1, 3] means the second and fourth predictors in X.
+
+
 ## Method: calculate_terms(X:npt.ArrayLike)
 
 ***Returns a numpy matrix containing values of model terms calculated on X.***

README.md

@@ -2,7 +2,7 @@
 Automatic Piecewise Linear Regression.
 
 # About
-Build predictive and interpretable parametric regression or classification machine learning models in Python based on the Automatic Piecewise Linear Regression (APLR) methodology developed by Mathias von Ottenbreit. APLR is often able to compete with tree-based methods on predictiveness, but unlike tree-based methods APLR is interpretable. See the ***documentation*** folder for more information. Links to published article: [https://link.springer.com/article/10.1007/s00180-024-01475-4](https://link.springer.com/article/10.1007/s00180-024-01475-4) and [https://rdcu.be/dz7bF](https://rdcu.be/dz7bF).
+Build predictive and interpretable parametric regression or classification machine learning models in Python based on the Automatic Piecewise Linear Regression (APLR) methodology developed by Mathias von Ottenbreit. APLR is often able to compete with tree-based methods on predictiveness, but unlike tree-based methods APLR is interpretable. Please see the [documentation](https://github.com/ottenbreit-data-science/aplr/tree/main/documentation) for more information. Links to published article: [https://link.springer.com/article/10.1007/s00180-024-01475-4](https://link.springer.com/article/10.1007/s00180-024-01475-4) and [https://rdcu.be/dz7bF](https://rdcu.be/dz7bF). More functionality has been added to APLR since the article was published.
 
 # How to install
 ***pip install aplr***
@@ -11,10 +11,10 @@ Build predictive and interpretable parametric regression or classification machi
 Currently available for Windows and most Linux distributions.
 
 # How to use
-Please see the two example Python scripts in the ***examples*** folder. They cover common use cases, but not all of the functionality in this package.
+Please see the two example Python scripts [here](https://github.com/ottenbreit-data-science/aplr/tree/main/examples). They cover common use cases, but not all of the functionality in this package.
 
 # Sponsorship
 Please consider sponsoring Ottenbreit Data Science by clicking on the Sponsor button. Sufficient funding will enable maintenance of APLR and further development.
 
 # API reference
-Please see ***API_REFERENCE_FOR_REGRESSION.md*** and ***API_REFERENCE_FOR_CLASSIFICATION.md***.
+Please see the [api reference for regression](https://github.com/ottenbreit-data-science/aplr/blob/main/API_REFERENCE_FOR_REGRESSION.md) and [api reference for classification](https://github.com/ottenbreit-data-science/aplr/blob/main/API_REFERENCE_FOR_CLASSIFICATION.md).

aplr/aplr.py

@@ -237,6 +237,13 @@ def calculate_local_feature_contribution(self, X: npt.ArrayLike) -> npt.ArrayLik
     def calculate_local_term_contribution(self, X: npt.ArrayLike) -> npt.ArrayLike:
         return self.APLRRegressor.calculate_local_term_contribution(X)
 
+    def calculate_local_contribution_from_selected_terms(
+        self, X: npt.ArrayLike, predictor_indexes: List[int]
+    ) -> npt.ArrayLike:
+        return self.APLRRegressor.calculate_local_contribution_from_selected_terms(
+            X, predictor_indexes
+        )
+
     def calculate_terms(self, X: npt.ArrayLike) -> npt.ArrayLike:
         return self.APLRRegressor.calculate_terms(X)
 

cpp/APLRRegressor.h

@@ -262,6 +262,7 @@ class APLRRegressor
     VectorXd calculate_term_importance(const MatrixXd &X, const VectorXd &sample_weight = VectorXd(0));
     MatrixXd calculate_local_feature_contribution(const MatrixXd &X);
     MatrixXd calculate_local_term_contribution(const MatrixXd &X);
+    VectorXd calculate_local_contribution_from_selected_terms(const MatrixXd &X, const std::vector<size_t> &predictor_indexes);
     MatrixXd calculate_terms(const MatrixXd &X);
     std::vector<std::string> get_term_names();
     VectorXd get_term_coefficients();
@@ -2277,6 +2278,29 @@ MatrixXd APLRRegressor::calculate_local_term_contribution(const MatrixXd &X)
     return output;
 }
 
+VectorXd APLRRegressor::calculate_local_contribution_from_selected_terms(const MatrixXd &X, const std::vector<size_t> &predictor_indexes)
+{
+    validate_that_model_can_be_used(X);
+
+    VectorXd contribution_from_selected_terms{VectorXd::Constant(X.rows(), 0.0)};
+
+    std::vector<size_t> term_indexes_used;
+    term_indexes_used.reserve(terms.size());
+    for (size_t i = 0; i < terms.size(); ++i)
+    {
+        if (terms[i].term_uses_just_these_predictors(predictor_indexes))
+            term_indexes_used.push_back(i);
+    }
+    term_indexes_used.shrink_to_fit();
+
+    for (auto &term_index_used : term_indexes_used)
+    {
+        contribution_from_selected_terms += terms[term_index_used].calculate_contribution_to_linear_predictor(X);
+    }
+
+    return contribution_from_selected_terms;
+}
+
 MatrixXd APLRRegressor::calculate_terms(const MatrixXd &X)
 {
     validate_that_model_can_be_used(X);
@@ -2464,11 +2488,6 @@ std::map<double, double> APLRRegressor::get_coefficient_shape_function(size_t pr
     return coefficient_shape_function;
 }
 
-double APLRRegressor::get_cv_error()
-{
-    return cv_error;
-}
-
 std::vector<size_t> APLRRegressor::compute_relevant_term_indexes(size_t predictor_index)
 {
     std::vector<size_t> relevant_term_indexes;
@@ -2493,4 +2512,9 @@ std::vector<size_t> APLRRegressor::compute_relevant_term_indexes(size_t predicto
     }
     relevant_term_indexes.shrink_to_fit();
     return relevant_term_indexes;
+}
+
+double APLRRegressor::get_cv_error()
+{
+    return cv_error;
 }

cpp/functions.h

@@ -67,6 +67,12 @@ std::set<int> get_unique_integers(const VectorXi &int_vector)
     return unique_integers;
 }
 
+std::set<size_t> get_unique_integers(const std::vector<size_t> &size_t_vector)
+{
+    std::set<size_t> unique_integers{size_t_vector.begin(), size_t_vector.end()};
+    return unique_integers;
+}
+
 double set_error_to_infinity_if_invalid(double error)
 {
     bool error_is_invalid{!std::isfinite(error)};

cpp/pythonbinding.cpp

@@ -58,6 +58,7 @@ PYBIND11_MODULE(aplr_cpp, m)
         .def("calculate_term_importance", &APLRRegressor::calculate_term_importance, py::arg("X"), py::arg("sample_weight") = VectorXd(0))
         .def("calculate_local_feature_contribution", &APLRRegressor::calculate_local_feature_contribution, py::arg("X"))
         .def("calculate_local_term_contribution", &APLRRegressor::calculate_local_term_contribution, py::arg("X"))
+        .def("calculate_local_contribution_from_selected_terms", &APLRRegressor::calculate_local_contribution_from_selected_terms, py::arg("X"), py::arg("predictor_indexes"))
         .def("calculate_terms", &APLRRegressor::calculate_terms, py::arg("X"))
         .def("get_term_names", &APLRRegressor::get_term_names)
         .def("get_term_coefficients", &APLRRegressor::get_term_coefficients)

cpp/term.h

@@ -70,6 +70,7 @@ class Term
     bool coefficient_adheres_to_monotonic_constraint();
     InteractionConstraintsTest test_interaction_constraints(const std::vector<size_t> &legal_interaction_combination);
     std::vector<size_t> get_unique_base_terms_used_in_this_term();
+    bool term_uses_just_these_predictors(const std::vector<size_t> &predictor_indexes);
 
 public:
     std::string name;
@@ -773,6 +774,20 @@ double Term::get_estimated_term_importance()
     return estimated_term_importance;
 }
 
+bool Term::term_uses_just_these_predictors(const std::vector<size_t> &predictor_indexes)
+{
+    std::vector<size_t> predictor_indexes_used_by_this_term;
+    predictor_indexes_used_by_this_term.push_back(base_term);
+    for (auto &given_term : given_terms)
+    {
+        predictor_indexes_used_by_this_term.push_back(given_term.base_term);
+    }
+    std::set<size_t> unique_predictor_indexes_used_by_this_term{get_unique_integers(predictor_indexes_used_by_this_term)};
+    std::set<size_t> unique_predictor_indexes{get_unique_integers(predictor_indexes)};
+    bool only_predictor_indexes_are_used{unique_predictor_indexes_used_by_this_term == unique_predictor_indexes};
+    return only_predictor_indexes_are_used;
+}
+
 std::vector<size_t> create_term_indexes(std::vector<Term> &terms)
 {
     std::vector<size_t> term_indexes;

cpp/tests.cpp

@@ -1597,6 +1597,7 @@ class Tests
 
         VectorXd predictions{model.predict(X_test)};
         MatrixXd li{model.calculate_local_feature_contribution(X_test)};
+        VectorXd li_for_particular_terms{model.calculate_local_contribution_from_selected_terms(X_train, {5, 1})};
 
         // Saving results
         save_as_csv_file("data/output.csv", predictions);
@@ -1607,8 +1608,12 @@ class Tests
         std::map<double, double> coefficient_shape_function = model.get_coefficient_shape_function(1);
         bool coefficient_shape_function_has_correct_length{coefficient_shape_function.size() == 27};
         bool coefficient_shape_function_value_test{is_approximately_equal(coefficient_shape_function.begin()->second, 0.04175, 0.00001)};
+        bool li_for_particular_terms_has_correct_size{li_for_particular_terms.rows() == X_train.rows()};
+        bool li_for_particular_terms_mean_is_correct{is_approximately_equal(li_for_particular_terms.mean(), 0.30321952178814915)};
         tests.push_back(coefficient_shape_function_has_correct_length);
         tests.push_back(coefficient_shape_function_value_test);
+        tests.push_back(li_for_particular_terms_has_correct_size);
+        tests.push_back(li_for_particular_terms_mean_is_correct);
     }
 
     void test_aplr_classifier_multi_class_other_params()

documentation/APLR 9.9.0.pdf renamed to documentation/APLR 9.10.0.pdf

@@ -102,7 +102,7 @@
     }
 )
 
-# Coefficient shape for the third predictor. Will be empty if the third predictor is not used as a main effect in the model
+# Coefficient shape for the third predictor. Will be empty if the third predictor is not used as a main effect in the model.
 coefficient_shape = best_model.get_coefficient_shape_function(predictor_index=2)
 coefficient_shape = pd.DataFrame(
     {
@@ -111,6 +111,13 @@
     }
 )
 
+# Local (observation specific) contribution to the linear predictor from selected interacting predictors. 
+# In this example this concerns two-way interaction terms in the model where the second and the third predictors in X interact.
+# The local contribution will be zero for all observations if there are no such terms in the model. 
+# The local contribution can help interpreting interactions (or main effects if only one predictor index is specified).
+# In this example, the local contribution can be plotted against the predictor values for a visual interpretation.
+contribution_from_selected_terms = best_model.calculate_local_contribution_from_selected_terms(X=data_train[predictors],predictor_indexes=[1,2])
+
 
 # PREDICTING AND TESTING ON THE TEST SET
 data_test[predicted] = best_model.predict(data_test[predictors].values)