Skip to content

Commit a6d28e3

Browse files
florian-huberflorian-huber
committed
text edits
1 parent 9c3325d commit a6d28e3

File tree

1 file changed

+40
-44
lines changed

1 file changed

+40
-44
lines changed

‎notebooks/19_machine_learning_techniques.ipynb‎

Lines changed: 40 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -79,43 +79,42 @@
7979
"\n",
8080
"Precision and recall are metrics that provide more insight into the accuracy of positive predictions and the classifier's ability to recover all relevant instances, respectively.\n",
8181
"\n",
82-
"- **Precision (Positive Predictive Value)**: The ratio of correctly predicted positive observations to the total predicted positives.\n",
82+
"**Precision (Positive Predictive Value)**: \n",
83+
"The ratio of correctly predicted positive observations to the total predicted positives.\n",
8384
"\n",
84-
" $$\n",
85-
" Precision = \\frac{TP}{TP + FP}\n",
86-
" $$\n",
87-
" \n",
88-
" High precision indicates a low false positive rate, which is crucial in scenarios where the cost of a false positive is high, such as in email spam classification.\n",
85+
"$$\n",
86+
"Precision = \\frac{TP}{TP + FP}\n",
87+
"$$\n",
88+
"\n",
89+
"High precision indicates a low false positive rate, which is crucial in scenarios where the cost of a false positive is high, such as in email spam classification.\n",
8990
"\n",
90-
"- **Recall (Sensitivity, True Positive Rate)**: The ratio of correctly predicted positive observations to all observations in actual class.\n",
91+
"**Recall (Sensitivity, True Positive Rate)**: \n",
92+
"The ratio of correctly predicted positive observations to all observations in actual class.\n",
9193
"\n",
92-
" $$\n",
93-
" Recall = \\frac{TP}{TP + FN}\n",
94-
" $$\n",
94+
"$$\n",
95+
"Recall = \\frac{TP}{TP + FN}\n",
96+
"$$\n",
9597
"\n",
96-
" High recall is vital in medical scenarios or fraud detection, where failing to detect an anomaly can have severe consequences.\n",
98+
"High recall is vital in medical scenarios or fraud detection, where failing to detect an anomaly can have severe consequences.\n",
9799
"\n",
98100
"#### 3. F1 Score\n",
99101
"\n",
100102
"The F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. It is particularly useful when the class distribution is uneven (biased data).\n",
101103
"\n",
102-
"- Formula:\n",
103104
"\n",
104-
" $$\n",
105-
" F_1 = 2 \\cdot \\frac{Precision \\cdot Recall}{Precision + Recall}\n",
106-
" $$\n",
105+
"$$\n",
106+
"F_1 = 2 \\cdot \\frac{Precision \\cdot Recall}{Precision + Recall}\n",
107+
"$$\n",
107108
" \n",
108-
" The F1 score is an excellent measure to use if you need to seek a balance between Precision and Recall and there is an uneven class distribution (as in the case of your fraudulent vs. non-fraudulent calls scenario).\n",
109+
"The F1 score is an excellent measure to use if you need to seek a balance between Precision and Recall and there is an uneven class distribution (as in the case of your fraudulent vs. non-fraudulent calls scenario).\n",
109110
"\n",
110111
"#### 4. Accuracy\n",
111112
"\n",
112113
"As previously discussed, accuracy is the ratio of correctly predicted observations to the total observations and can be misleading in the presence of an imbalanced dataset.\n",
113114
"\n",
114-
"- Formula:\n",
115-
"\n",
116-
" $$\n",
117-
" Accuracy = \\frac{TP + TN}{TP + TN + FP + FN}\n",
118-
" $$\n",
115+
"$$\n",
116+
"Accuracy = \\frac{TP + TN}{TP + TN + FP + FN}\n",
117+
"$$\n",
119118
"\n",
120119
"\n",
121120
"### Detailed Model Evaluation Metrics (Regression)\n",
@@ -125,25 +124,21 @@
125124
"\n",
126125
"MAE measures the average magnitude of the errors in a set of predictions, without considering their direction. It’s the average over the test sample of the absolute differences between prediction and actual observation where all individual differences have equal weight.\n",
127126
"\n",
128-
"- Formula:\n",
127+
"$$\n",
128+
"\\text{MAE} = \\frac{1}{n} \\sum_{i=1}^n |y_i - \\hat{y}_i|\n",
129+
"$$\n",
129130
"\n",
130-
" $$\n",
131-
" \\text{MAE} = \\frac{1}{n} \\sum_{i=1}^n |y_i - \\hat{y}_i|\n",
132-
" $$\n",
133-
"\n",
134-
" where $y_i$ are the actual values and $\\hat{y}_i$ are the predicted values.\n",
131+
"where $y_i$ are the actual values and $\\hat{y}_i$ are the predicted values.\n",
135132
"\n",
136133
"#### 2. Mean Squared Error (MSE)\n",
137134
"\n",
138135
"MSE is like MAE but squares the difference before summing them all instead of using the absolute value. This has the effect of heavily penalizing larger errors.\n",
139136
"\n",
140-
"- Formula:\n",
141-
"\n",
142-
" $$\n",
143-
" \\text{MSE} = \\frac{1}{n} \\sum_{i=1}^n (y_i - \\hat{y}_i)^2\n",
144-
" $$\n",
137+
"$$\n",
138+
"\\text{MSE} = \\frac{1}{n} \\sum_{i=1}^n (y_i - \\hat{y}_i)^2\n",
139+
"$$\n",
145140
"\n",
146-
" MSE is more sensitive to outliers than MAE and tends to emphasize larger differences.\n",
141+
"MSE is more sensitive to outliers than MAE and tends to emphasize larger differences.\n",
147142
"\n",
148143
"\n",
149144
"There are many more metrics available, such as `RMSE` or `R-squared`, but we will not cover them in this course.\n",
@@ -184,7 +179,6 @@
184179
" X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n",
185180
" ```\n",
186181
"\n",
187-
"\n",
188182
"**2. Train-Validation-Test Split**\n",
189183
"When tuning hyperparameters or making decisions about model architecture, it’s crucial to have a third split: the validation set.\n",
190184
"\n",
@@ -193,19 +187,20 @@
193187
"\n",
194188
"### Cross-Validation: Enhancing Model Validation\n",
195189
"\n",
196-
"**Cross-validation** is a robust method for estimating the effectiveness of your model which is especially useful when dealing with limited data.\n",
190+
"Cross-validation is a robust method for estimating the effectiveness of your model, which is especially useful when dealing with limited data.\n",
197191
"\n",
198-
"- **K-Fold Cross-Validation**: The data set is divided into 'k' smaller sets. The model is trained on 'k-1' of these folds, with the remaining part used as the test fold. This process is repeated 'k' times with each of the 'k' folds used exactly once as the test set.\n",
192+
"A **k-Fold Cross-Validation** means that the dataset is divided into 'k' smaller sets. The model is trained on 'k-1' of these folds, with the remaining part used as the test fold. This process is repeated 'k' times, with each of the 'k' folds used exactly once as the test set.\n",
199193
"\n",
200-
" ```python\n",
201-
" from sklearn.model_selection import cross_val_score, KFold\n",
202-
" kf = KFold(n_splits=5, random_state=42, shuffle=True)\n",
203-
" scores = cross_val_score(model, X, y, cv=kf)\n",
204-
" average_score = scores.mean()\n",
205-
" ```\n",
194+
"Using Scikit-Learn this can be realized like this:\n",
195+
"```python\n",
196+
"from sklearn.model_selection import cross_val_score, KFold\n",
206197
"\n",
198+
"kf = KFold(n_splits=5, random_state=42, shuffle=True)\n",
199+
"scores = cross_val_score(model, X, y, cv=kf)\n",
200+
"average_score = scores.mean()\n",
201+
"```\n",
207202
"\n",
208-
"- **Stratified K-Fold**: A variation of k-fold which is used when one has imbalanced classes. It ensures that each fold of the dataset has the same proportion of examples in each class as the complete set.\n",
203+
"A variation of a k-fold is the **stratified K-Fold** that is used for highly imbalanced classes. It ensures that each fold of the dataset has the same proportion of examples in each class as the complete set.\n",
209204
"\n",
210205
"\n",
211206
"```{figure} ../images/fig_cross_validation_training.png\n",
@@ -217,6 +212,7 @@
217212
"### Advanced Techniques for Unbalanced Data\n",
218213
"\n",
219214
"When dealing with imbalanced datasets, traditional training and testing strategies may not suffice, as they might lead to models biased towards the majority class.\n",
215+
"Common strategies to work with highly imbalanced data are:\n",
220216
"\n",
221217
"- **Oversampling the Minority Class**: Increasing the number of instances in the minority class by duplicating them to prevent the model from being biased toward the majority class.\n",
222218
"- **Undersampling the Majority Class**: Reducing the number of instances in the majority class to balance the dataset.\n",
@@ -230,7 +226,7 @@
230226
"\n",
231227
"\n",
232228
"### Beyond this course:\n",
233-
"Here, we did the sampling, i.e. the selection of data points for either training or testing, fully randomly. In many real world cases, however, the situation can be more complex. For instance, we might have many medical observations in a dataset with sometimes multiple entries for the same patient. In such a case we would have to split the data by patient and not fully randomly (see for instance here: {cite}`tougui2021impact`). In other cases, we will have strong biases and we would often like to compensate for this. For example to avoid that -purely by chance- a certain population or class is not well represented in a certain data split. The process to compensate for this is called **stratification**."
229+
"Here, we did the sampling, that is the selection of data points, for either training or testing, fully randomly. In many real-world cases, however, the situation can be more complex. For instance, we might have many medical observations in a dataset with sometimes multiple entries for the same patient. In such a case we would have to split the data by patient and not fully randomly (see, for instance, here: {cite}`tougui2021impact`). In other cases, we will have strong biases and we would often like to compensate for this. For example, to avoid that -purely by chance- a certain population or class is not well represented in a certain data split. The process to compensate for this is called **stratification**."
234230
]
235231
},
236232
{

0 commit comments

Comments
 (0)