added the possibility to specify a custom link_function

Author: mathias-von-ottenbreit

API_REFERENCE_FOR_REGRESSION.md

@@ -1,6 +1,6 @@
 # APLRRegressor
 
-## class aplr.APLRRegressor(m:int=1000, v:float=0.1, random_state:int=0, loss_function:str="mse", link_function:str="identity", n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, verbosity:int=0, dispersion_parameter:float=1.5, validation_tuning_metric:str="default", quantile:float=0.5, calculate_custom_validation_error_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_loss_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None)
+## class aplr.APLRRegressor(m:int=1000, v:float=0.1, random_state:int=0, loss_function:str="mse", link_function:str="identity", n_jobs:int=0, validation_ratio:float=0.2, bins:int=300, max_interaction_level:int=1, max_interactions:int=100000, min_observations_in_split:int=20, ineligible_boosting_steps_added:int=10, max_eligible_terms:int=5, verbosity:int=0, dispersion_parameter:float=1.5, validation_tuning_metric:str="default", quantile:float=0.5, calculate_custom_validation_error_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_loss_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None, calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_transform_linear_predictor_to_predictions_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None, calculate_custom_differentiate_predictions_wrt_linear_predictor_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None)
 
 ### Constructor parameters
 
@@ -17,7 +17,7 @@ Used to randomly split training observations into training and validation if ***
 Determines the loss function used. Allowed values are "mse", "binomial", "poisson", "gamma", "tweedie", "group_mse", "mae", "quantile", "negative_binomial", "cauchy", "weibull" and "custom_function". This is used together with ***link_function***. When ***loss_function*** is "group_mse" then the "group" argument in the ***fit*** method must be provided. In the latter case APLR will try to minimize group MSE when training the model. The ***loss_function*** "quantile" is used together with the ***quantile*** constructor parameter. When ***loss_function*** is "custom_function" then the constructor parameters ***calculate_custom_loss_function*** and ***calculate_custom_negative_gradient_function***, both described below, must be provided.
 
 #### link_function (default = "identity")
-Determines how the linear predictor is transformed to predictions. Allowed values are "identity", "logit" and "log". For an ordinary regression model use ***loss_function*** "mse" and ***link_function*** "identity". For logistic regression use ***loss_function*** "binomial" and ***link_function*** "logit". For a multiplicative model use the "log" ***link_function***. The "log" ***link_function*** often works best with a "poisson", "gamma", "tweedie", "negative_binomial" or "weibull" ***loss_function***, depending on the data. The ***loss_function*** "poisson", "gamma", "tweedie", "negative_binomial" or "weibull" should only be used with the "log" ***link_function***. Inappropriate combinations of ***loss_function*** and ***link_function*** may result in a warning message when fitting the model and/or a poor model fit. Please note that values other than "identity" typically require a significantly higher ***m*** (or ***v***) in order to converge.
+Determines how the linear predictor is transformed to predictions. Allowed values are "identity", "logit", "log" and "custom_function". For an ordinary regression model use ***loss_function*** "mse" and ***link_function*** "identity". For logistic regression use ***loss_function*** "binomial" and ***link_function*** "logit". For a multiplicative model use the "log" ***link_function***. The "log" ***link_function*** often works best with a "poisson", "gamma", "tweedie", "negative_binomial" or "weibull" ***loss_function***, depending on the data. The ***loss_function*** "poisson", "gamma", "tweedie", "negative_binomial" or "weibull" should only be used with the "log" ***link_function***. Inappropriate combinations of ***loss_function*** and ***link_function*** may result in a warning message when fitting the model and/or a poor model fit. Please note that values other than "identity" typically require a significantly higher ***m*** (or ***v***) in order to converge. When ***link_function*** is "custom_function" then the constructor parameters ***calculate_custom_transform_linear_predictor_to_predictions_function*** and ***calculate_custom_differentiate_predictions_wrt_linear_predictor_function***, both described below, must be provided.
 
 #### n_jobs (default = 0)
 Multi-threading parameter. If ***0*** then uses all available cores for multi-threading. Any other positive integer specifies the number of cores to use (***1*** means single-threading).
@@ -82,6 +82,26 @@ def custom_negative_gradient_function(y, predictions, group):
     return residuals
 ```
 
+#### calculate_custom_transform_linear_predictor_to_predictions_function (default = None)
+A Python function that transforms the linear predictor to predictions if ***link_function*** is "custom_function". Example:
+
+```
+def calculate_custom_transform_linear_predictor_to_predictions(linear_predictor):
+    #This particular example is prone to numerical problems (requires small and non-negative response values).
+    predictions=np.exp(linear_predictor)
+    return predictions
+```
+
+#### calculate_custom_differentiate_predictions_wrt_linear_predictor_function (default = None)
+A Python function that does a first order differentiation of the predictions with respect to the linear predictor. Example:
+
+```
+def calculate_custom_differentiate_predictions_wrt_linear_predictor(linear_predictor):
+    #This particular example is prone to numerical problems (requires small and non-negative response values).
+    differentiated_predictions=np.exp(linear_predictor)
+    return differentiated_predictions
+```
+
 ## Method: fit(X:npt.ArrayLike, y:npt.ArrayLike, sample_weight:npt.ArrayLike = np.empty(0), X_names:List[str]=[], validation_set_indexes:List[int]=[], prioritized_predictors_indexes:List[int]=[], monotonic_constraints:List[int]=[], group:npt.ArrayLike = np.empty(0), interaction_constraints:List[int]=[])
 
 ***This method fits the model to data.***

aplr/aplr.py

@@ -11,7 +11,10 @@ def __init__(self, m:int=1000, v:float=0.1, random_state:int=0, loss_function:st
                  dispersion_parameter:float=1.5, validation_tuning_metric:str="default", quantile:float=0.5,
                  calculate_custom_validation_error_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None,
                  calculate_custom_loss_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], float]]=None,
-                 calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None):
+                 calculate_custom_negative_gradient_function:Optional[Callable[[npt.ArrayLike, npt.ArrayLike, npt.ArrayLike], npt.ArrayLike]]=None,
+                 calculate_custom_transform_linear_predictor_to_predictions_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None,
+                 calculate_custom_differentiate_predictions_wrt_linear_predictor_function:Optional[Callable[[npt.ArrayLike], npt.ArrayLike]]=None
+                 ):
         self.m=m
         self.v=v
         self.random_state=random_state
@@ -32,6 +35,8 @@ def __init__(self, m:int=1000, v:float=0.1, random_state:int=0, loss_function:st
         self.calculate_custom_validation_error_function=calculate_custom_validation_error_function
         self.calculate_custom_loss_function=calculate_custom_loss_function
         self.calculate_custom_negative_gradient_function=calculate_custom_negative_gradient_function
+        self.calculate_custom_transform_linear_predictor_to_predictions_function=calculate_custom_transform_linear_predictor_to_predictions_function
+        self.calculate_custom_differentiate_predictions_wrt_linear_predictor_function=calculate_custom_differentiate_predictions_wrt_linear_predictor_function
 
         #Creating aplr_cpp and setting parameters
         self.APLRRegressor=aplr_cpp.APLRRegressor()
@@ -59,12 +64,16 @@ def __set_params_cpp(self):
         self.APLRRegressor.calculate_custom_validation_error_function=self.calculate_custom_validation_error_function
         self.APLRRegressor.calculate_custom_loss_function=self.calculate_custom_loss_function
         self.APLRRegressor.calculate_custom_negative_gradient_function=self.calculate_custom_negative_gradient_function
+        self.APLRRegressor.calculate_custom_transform_linear_predictor_to_predictions_function=self.calculate_custom_transform_linear_predictor_to_predictions_function
+        self.APLRRegressor.calculate_custom_differentiate_predictions_wrt_linear_predictor_function=self.calculate_custom_differentiate_predictions_wrt_linear_predictor_function
 
     def fit(self, X:npt.ArrayLike, y:npt.ArrayLike, sample_weight:npt.ArrayLike = np.empty(0), X_names:List[str]=[], validation_set_indexes:List[int]=[], prioritized_predictors_indexes:List[int]=[], monotonic_constraints:List[int]=[], group:npt.ArrayLike = np.empty(0), interaction_constraints:List[int]=[]):
         self.__set_params_cpp()
         self.APLRRegressor.fit(X,y,sample_weight,X_names,validation_set_indexes,prioritized_predictors_indexes,monotonic_constraints,group,interaction_constraints)
 
     def predict(self, X:npt.ArrayLike, cap_predictions_to_minmax_in_training:bool=True)->npt.ArrayLike:
+        if self.link_function=="custom_function":
+            self.APLRRegressor.calculate_custom_transform_linear_predictor_to_predictions_function=self.calculate_custom_transform_linear_predictor_to_predictions_function        
         return self.APLRRegressor.predict(X, cap_predictions_to_minmax_in_training)
 
     def set_term_names(self, X_names:List[str]):
@@ -131,7 +140,9 @@ def get_params(self, deep=True):
             "quantile":self.quantile,
             "calculate_custom_validation_error_function":self.calculate_custom_validation_error_function,
             "calculate_custom_loss_function":self.calculate_custom_loss_function,
-            "calculate_custom_negative_gradient_function":self.calculate_custom_negative_gradient_function
+            "calculate_custom_negative_gradient_function":self.calculate_custom_negative_gradient_function,
+            "calculate_custom_transform_linear_predictor_to_predictions_function":self.calculate_custom_transform_linear_predictor_to_predictions_function,
+            "calculate_custom_differentiate_predictions_wrt_linear_predictor_function":self.calculate_custom_differentiate_predictions_wrt_linear_predictor_function
         }
 
     #For sklearn