added get_coefficient_shape_function, different option regarding monotonicity

Author: mathias-von-ottenbreit

API_REFERENCE_FOR_CLASSIFICATION.md

@@ -1,6 +1,6 @@
 # APLRClassifier
 
-## class aplr.APLRClassifier(m:int=9000, v:float=0.1, random_state:int=0, n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, verbosity:int=0, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, boosting_steps_before_pruning_is_done:int = 0, boosting_steps_before_interactions_are_allowed: int = 0)
+## class aplr.APLRClassifier(m:int=9000, v:float=0.1, random_state:int=0, n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, verbosity:int=0, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, boosting_steps_before_pruning_is_done:int = 0, boosting_steps_before_interactions_are_allowed: int = 0, monotonic_constraints_ignore_interactions: bool = False)
 
 ### Constructor parameters
 
@@ -46,6 +46,9 @@ Specifies how many boosting steps to wait before pruning the model. If 0 (defaul
 #### boosting_steps_before_interactions_are_allowed (default = 0)
 Specifies how many boosting steps to wait before searching for interactions. If for example 800, then the algorithm will be forced to only fit main effects in the first 800 boosting steps, after which it is allowed to search for interactions (given that other hyperparameters that control interactions also allow this). The motivation for fitting main effects first may be 1) to get a cleaner looking model that puts more emphasis on main effects and 2) to speed up the algorithm since looking for interactions is computationally more demanding.
 
+#### monotonic_constraints_ignore_interactions (default = False)
+See ***monotonic_constraints*** in the ***fit*** method.
+
 
 ## Method: fit(X:npt.ArrayLike, y:List[str], sample_weight:npt.ArrayLike = np.empty(0), X_names:List[str]=[], validation_set_indexes:List[int]=[], prioritized_predictors_indexes:List[int]=[], monotonic_constraints:List[int]=[], interaction_constraints:List[List[int]]=[])
 
@@ -72,7 +75,7 @@ An optional list of integers specifying the indexes of observations to be used f
 An optional list of integers specifying the indexes of predictors (columns) in ***X*** that should be prioritized. Terms of the prioritized predictors will enter the model as long as they reduce the training error and do not contain too few effective observations. They will also be updated more often.
 
 #### monotonic_constraints
-An optional list of integers specifying monotonic constraints on model terms. For example, if there are three predictors in ***X***, then monotonic_constraints = [1,0,-1] means that 1) the first predictor in ***X*** cannot be used in interaction terms as a secondary effect and all terms using the first predictor in ***X*** as a main effect must have positive regression coefficients, 2) there are no monotonic constraints on terms using the second predictor in ***X***, and 3) the third predictor in ***X*** cannot be used in interaction terms as a secondary effect and all terms using the third predictor in ***X*** as a main effect must have negative regression coefficients.
+An optional list of integers specifying monotonic constraints on model terms. For example, if there are three predictors in ***X***, then monotonic_constraints = [1,0,-1] means that 1) all terms using the first predictor in ***X*** as a main effect must have positive regression coefficients, 2) there are no monotonic constraints on terms using the second predictor in ***X***, and 3) all terms using the third predictor in ***X*** as a main effect must have negative regression coefficients. In the above example, if ***monotonic_constraints_ignore_interactions*** is ***False*** (default) then the first and the third predictors in ***X*** cannot be used in interaction terms as secondary effects. The latter guarantees monotonicity but can degrade predictiveness especially if a large proportion of predictors have monotonic constraints (in this case significantly fewer interaction terms can be formed).
 
 #### interaction_constraints
 An optional list containing lists of integers. Specifies interaction constraints on model terms. For example, interaction_constraints = [[0,1], [1,2,3]] means that 1) the first and second predictors may interact with each other, and that 2) the second, third and fourth predictors may interact with each other. There are no interaction constraints on predictors not mentioned in interaction_constraints.

API_REFERENCE_FOR_REGRESSION.md

@@ -1,6 +1,6 @@
 # APLRRegressor
 
-## class aplr.APLRRegressor(m:int=1000, v:float=0.1, random_state:int=0, loss_function:str="mse", link_function:str="identity", n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, verbosity:int=0, dispersion_parameter:float=1.5, validation_tuning_metric:str="default", quantile:float=0.5, calculate_custom_validation_error_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_loss_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_transform_linear_predictor_to_predictions_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_differentiate_predictions_wrt_linear_predictor_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None, boosting_steps_before_pruning_is_done: int = 0, boosting_steps_before_interactions_are_allowed: int = 0)
+## class aplr.APLRRegressor(m:int=1000, v:float=0.1, random_state:int=0, loss_function:str="mse", link_function:str="identity", n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, verbosity:int=0, dispersion_parameter:float=1.5, validation_tuning_metric:str="default", quantile:float=0.5, calculate_custom_validation_error_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_loss_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_transform_linear_predictor_to_predictions_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_differentiate_predictions_wrt_linear_predictor_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None, boosting_steps_before_pruning_is_done: int = 0, boosting_steps_before_interactions_are_allowed: int = 0, monotonic_constraints_ignore_interactions: bool = False)
 
 ### Constructor parameters
 
@@ -108,6 +108,9 @@ Specifies how many boosting steps to wait before pruning the model. If 0 (defaul
 #### boosting_steps_before_interactions_are_allowed (default = 0)
 Specifies how many boosting steps to wait before searching for interactions. If for example 800, then the algorithm will be forced to only fit main effects in the first 800 boosting steps, after which it is allowed to search for interactions (given that other hyperparameters that control interactions also allow this). The motivation for fitting main effects first may be 1) to get a cleaner looking model that puts more emphasis on main effects and 2) to speed up the algorithm since looking for interactions is computationally more demanding.
 
+#### monotonic_constraints_ignore_interactions (default = False)
+See ***monotonic_constraints*** in the ***fit*** method.
+
 ## Method: fit(X:npt.ArrayLike, y:npt.ArrayLike, sample_weight:npt.ArrayLike = np.empty(0), X_names:List[str]=[], validation_set_indexes:List[int]=[], prioritized_predictors_indexes:List[int]=[], monotonic_constraints:List[int]=[], group:npt.ArrayLike = np.empty(0), interaction_constraints:List[List[int]]=[], other_data: npt.ArrayLike = np.empty([0, 0]))
 
 ***This method fits the model to data.***
@@ -133,7 +136,7 @@ An optional list of integers specifying the indexes of observations to be used f
 An optional list of integers specifying the indexes of predictors (columns) in ***X*** that should be prioritized. Terms of the prioritized predictors will enter the model as long as they reduce the training error and do not contain too few effective observations. They will also be updated more often.
 
 #### monotonic_constraints
-An optional list of integers specifying monotonic constraints on model terms. For example, if there are three predictors in ***X***, then monotonic_constraints = [1,0,-1] means that 1) the first predictor in ***X*** cannot be used in interaction terms as a secondary effect and all terms using the first predictor in ***X*** as a main effect must have positive regression coefficients, 2) there are no monotonic constraints on terms using the second predictor in ***X***, and 3) the third predictor in ***X*** cannot be used in interaction terms as a secondary effect and all terms using the third predictor in ***X*** as a main effect must have negative regression coefficients.
+An optional list of integers specifying monotonic constraints on model terms. For example, if there are three predictors in ***X***, then monotonic_constraints = [1,0,-1] means that 1) all terms using the first predictor in ***X*** as a main effect must have positive regression coefficients, 2) there are no monotonic constraints on terms using the second predictor in ***X***, and 3) all terms using the third predictor in ***X*** as a main effect must have negative regression coefficients. In the above example, if ***monotonic_constraints_ignore_interactions*** is ***False*** (default) then the first and the third predictors in ***X*** cannot be used in interaction terms as secondary effects. The latter guarantees monotonicity but can degrade predictiveness especially if a large proportion of predictors have monotonic constraints (in this case significantly fewer interaction terms can be formed).
 
 #### group
 A numpy vector of integers that is used when ***loss_function*** is "group_mse". For example, ***group*** may represent year (could be useful in a time series model).
@@ -249,4 +252,13 @@ The index of the term selected. So ***0*** is the first term, ***1*** is the sec
 
 ## Method: get_validation_indexes()
 
-***Returns a list of integers containing the indexes of the training data observations used for validation and not training.***
+***Returns a list of integers containing the indexes of the training data observations used for validation and not training.***
+
+## Method: get_coefficient_shape_function(predictor_index:int)
+
+***For the predictor in X specified by predictor_index, get_coefficient_shape_function returns a dictionary with keys equal to predictor values and values equal to coefficient. For each predictor value, the coefficient is the sum of coefficients for relevant terms using only the predictor (interactions with other predictors are ignored). This function makes it easier to interpret APLR models as one can quickly see how the main effects work across relevant values of the predictor. If the predictor is only used as a linear effect in the model then the predictor value is set to 0 even though the coefficient is valid for all values of the predictor.***
+
+### Parameters
+
+#### predictor_index
+The index of the predictor. So if ***predictor_index*** is ***1*** then the second predictor in ***X*** is used.

aplr/aplr.py

@@ -1,6 +1,6 @@
 import numpy as np
 import numpy.typing as npt
-from typing import List, Callable, Optional
+from typing import List, Callable, Optional, Dict
 import aplr_cpp
 
 
@@ -62,6 +62,7 @@ def __init__(
         ] = None,
         boosting_steps_before_pruning_is_done: int = 0,
         boosting_steps_before_interactions_are_allowed: int = 0,
+        monotonic_constraints_ignore_interactions: bool = False,
     ):
         self.m = m
         self.v = v
@@ -99,6 +100,9 @@ def __init__(
         self.boosting_steps_before_interactions_are_allowed = (
             boosting_steps_before_interactions_are_allowed
         )
+        self.monotonic_constraints_ignore_interactions = (
+            monotonic_constraints_ignore_interactions
+        )
 
         # Creating aplr_cpp and setting parameters
         self.APLRRegressor = aplr_cpp.APLRRegressor()
@@ -146,6 +150,9 @@ def __set_params_cpp(self):
         self.APLRRegressor.boosting_steps_before_interactions_are_allowed = (
             self.boosting_steps_before_interactions_are_allowed
         )
+        self.APLRRegressor.monotonic_constraints_ignore_interactions = (
+            self.monotonic_constraints_ignore_interactions
+        )
 
     def fit(
         self,
@@ -227,6 +234,11 @@ def get_validation_tuning_metric(self) -> str:
     def get_validation_indexes(self) -> List[int]:
         return self.APLRRegressor.get_validation_indexes()
 
+    def get_coefficient_shape_function(
+        self, predictor_index: int
+    ) -> Dict[float, float]:
+        return self.APLRRegressor.get_coefficient_shape_function(predictor_index)
+
     # For sklearn
     def get_params(self, deep=True):
         return {
@@ -254,6 +266,7 @@ def get_params(self, deep=True):
             "calculate_custom_differentiate_predictions_wrt_linear_predictor_function": self.calculate_custom_differentiate_predictions_wrt_linear_predictor_function,
             "boosting_steps_before_pruning_is_done": self.boosting_steps_before_pruning_is_done,
             "boosting_steps_before_interactions_are_allowed": self.boosting_steps_before_interactions_are_allowed,
+            "monotonic_constraints_ignore_interactions": self.monotonic_constraints_ignore_interactions,
         }
 
     # For sklearn
@@ -281,6 +294,7 @@ def __init__(
         max_eligible_terms: int = 5,
         boosting_steps_before_pruning_is_done: int = 0,
         boosting_steps_before_interactions_are_allowed: int = 0,
+        monotonic_constraints_ignore_interactions: bool = False,
     ):
         self.m = m
         self.v = v
@@ -300,6 +314,9 @@ def __init__(
         self.boosting_steps_before_interactions_are_allowed = (
             boosting_steps_before_interactions_are_allowed
         )
+        self.monotonic_constraints_ignore_interactions = (
+            monotonic_constraints_ignore_interactions
+        )
 
         # Creating aplr_cpp and setting parameters
         self.APLRClassifier = aplr_cpp.APLRClassifier()
@@ -327,6 +344,9 @@ def __set_params_cpp(self):
         self.APLRClassifier.boosting_steps_before_interactions_are_allowed = (
             self.boosting_steps_before_interactions_are_allowed
         )
+        self.APLRClassifier.monotonic_constraints_ignore_interactions = (
+            self.monotonic_constraints_ignore_interactions
+        )
 
     def fit(
         self,
@@ -401,6 +421,7 @@ def get_params(self, deep=True):
             "max_eligible_terms": self.max_eligible_terms,
             "boosting_steps_before_pruning_is_done": self.boosting_steps_before_pruning_is_done,
             "boosting_steps_before_interactions_are_allowed": self.boosting_steps_before_interactions_are_allowed,
+            "monotonic_constraints_ignore_interactions": self.monotonic_constraints_ignore_interactions,
         }
 
     # For sklearn