Delete KernelExplainerWrapper and remove importing LogitLink and IdentityLink (#1603)

Author: SomnathSI

autopilot/model-explainability/explaining_customer_churn_model.ipynb

@@ -8,7 +8,7 @@
     "\n",
     "Kernel `Python 3 (Data Science)` works well with this notebook.\n",
     "\n",
-    "_This notebook was created and tested on an ml.m5.large notebook instance._\n",
+    "_This notebook was created and tested on an ml.m5.xlarge notebook instance._\n",
     "\n",
     "## Table of Contents\n",
     "\n",
@@ -101,9 +101,8 @@
    "source": [
     "import shap\n",
     "\n",
-    "from kernel_explainer_wrapper import KernelExplainerWrapper\n",
+    "from shap import KernelExplainer\n",
     "from shap import sample\n",
-    "from shap.common import LogitLink, IdentityLink\n",
     "from scipy.special import expit\n",
     "\n",
     "# Initialize plugin to make plots interactive.\n",
@@ -235,7 +234,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "churn_data = pd.read_csv('./Data sets/churn.txt')\n",
+    "churn_data = pd.read_csv('../Data sets/churn.txt')\n",
     "data_without_target = churn_data.drop(columns=['Churn?'])\n",
     "\n",
     "background_data = sample(data_without_target, 50)"
@@ -252,7 +251,10 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Next, we create the `KernelExplainer`. Note that since it's a black box explainer, `KernelExplainer` only requires a handle to the predict (or predict_proba) function and does not require any other information about the model. For classification it is recommended to derive feature importance scores in the log-odds space since additivity is a more natural assumption there thus we use `LogitLink`. For regression `IdentityLink` should be used."
+    "Next, we create the `KernelExplainer`. Note that since it's a black box explainer, `KernelExplainer` only requires a handle to the\n",
+    "predict (or predict_proba) function and does not require any other information about the model. For classification it is recommended to\n",
+    "derive feature importance scores in the log-odds space since additivity is a more natural assumption there thus we use `logit`. For\n",
+    "regression `identity` should be used."
    ]
   },
   {
@@ -263,17 +265,16 @@
    "source": [
     "# Derive link function \n",
     "problem_type = automl_job.describe_auto_ml_job(job_name=automl_job_name)['ResolvedAttributes']['ProblemType'] \n",
-    "link_fn = IdentityLink if problem_type == 'Regression' else LogitLink \n",
+    "link = \"identity\" if problem_type == 'Regression' else \"logit\"\n",
     "\n",
-    "# the handle to predict_proba is passed to KernelExplainerWrapper since KernelSHAP requires the class probability\n",
-    "explainer = KernelExplainerWrapper(automl_estimator.predict_proba, background_data, link=link_fn())"
+    "# the handle to predict_proba is passed to KernelExplainer since KernelSHAP requires the class probability\n",
+    "explainer = KernelExplainer(automl_estimator.predict_proba, background_data, link=link)"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Currently, `shap.KernelExplainer` only supports numeric data. A version of SHAP that supports text will become available soon. A workaround is provided by our wrapper `KernelExplainerWrapper`. Once a new version of SHAP is released, `shap.KernelExplainer` should be used instead of `KernelExplainerWrapper`.\n",
     "\n",
     "By analyzing the background data `KernelExplainer` provides us with `explainer.expected_value` which is the model prediction with all features missing. Considering a customer for which we have no data at all (i.e. all features are missing) this should theoretically be the model prediction."
    ]
@@ -326,7 +327,7 @@
    "outputs": [],
    "source": [
     "# Since shap_values are provided in the log-odds space, we convert them back to the probability space by using LogitLink\n",
-    "shap.force_plot(explainer.expected_value, shap_values, x, link=link_fn())"
+    "shap.force_plot(explainer.expected_value, shap_values, x, link=link)"
    ]
   },
   {
@@ -348,7 +349,7 @@
    "source": [
     "with ManagedEndpoint(ep_name) as mep:\n",
     "    shap_values = explainer.shap_values(x, nsamples='auto', l1_reg='num_features(5)')\n",
-    "shap.force_plot(explainer.expected_value, shap_values, x, link=link_fn())"
+    "shap.force_plot(explainer.expected_value, shap_values, x, link=link)"
    ]
   },
   {
@@ -396,7 +397,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "shap.force_plot(explainer.expected_value, shap_values, X, link=link_fn())"
+    "shap.force_plot(explainer.expected_value, shap_values, X, link=link)"
    ]
   },
   {