final text and code comments up to 1.3

Author: avaamini

lab3/solutions/Lab3_Part_1_Introduction_to_CAPSA.ipynb

@@ -172,6 +172,7 @@
         }
       ],
       "source": [
+        "# Get the data for the cubic function, injected with noise and missing-ness\n",
         "def gen_data(x_min, x_max, n, train=True):\n",
         "    x = np.random.triangular(x_min, 2, x_max, size=(n, 1))\n",
         "\n",
@@ -180,10 +181,11 @@
         "\n",
         "    return x, y\n",
         "\n",
-        "x, y = gen_data(-4, 4, 2000)\n",
-        "x_val, y_val = gen_data(-6, 6, 500)\n",
-        "plt.scatter(x_val,y_val, s=1.5, label='test data')\n",
-        "plt.scatter(x,y, s=1.5, label='train data')\n",
+        "# Plot the dataset and visualize the train and test datapoints\n",
+        "x_train, y_train = gen_data(-4, 4, 2000) # train data\n",
+        "x_test, y_test = gen_data(-6, 6, 500) # test data\n",
+        "plt.scatter(x_train, y_train, s=1.5, label='train data')\n",
+        "plt.scatter(x_test, y_test, s=1.5, label='test data')\n",
         "\n",
         "plt.legend()\n",
         "plt.show()"
@@ -209,8 +211,9 @@
         "id": "mXMOYRHnv8tF"
       },
       "source": [
-        "### 1.2 Vanilla regression\n",
-        "Let's define a small model that can predict `y` given `x`: this is a classical regression task!"
+        "## 1.2 Regression on cubic dataset\n",
+        "\n",
+        "Next we will define a small dense neural network model that can predict `y` given `x`: this is a classical regression task! We will build the model and use the [`model.fit()`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit) function to train the model -- normally, without any risk-awareness -- using the train dataset that we visualized above."
       ]
     },
     {
@@ -221,7 +224,10 @@
       },
       "outputs": [],
       "source": [
-        "def create_standard_classifier():\n",
+        "### Define and train a dense NN model for the regression task###\n",
+        "\n",
+        "'''Function to define a small dense NN'''\n",
+        "def create_dense_NN():\n",
         "  return tf.keras.Sequential(\n",
         "          [\n",
         "              tf.keras.Input(shape=(1,)),\n",
@@ -231,7 +237,17 @@
         "          ]\n",
         "  )\n",
         "\n",
-        "standard_classifier = create_standard_classifier()"
+        "dense_NN = create_dense_NN()\n",
+        "\n",
+        "# Build the model for regression, defining the loss function and optimizer\n",
+        "dense_NN.compile(\n",
+        "  optimizer=tf.keras.optimizers.Adam(learning_rate=2e-3),\n",
+        "  loss=tf.keras.losses.MeanSquaredError(), # MSE loss for the regression task\n",
+        ")\n",
+        "\n",
+        "# TODO: Train the model for 10 epochs. Use model.fit().\n",
+        "loss_history = dense_NN.fit(x_train, y_train, epochs=10) \n",
+        "# loss_history = # TODO"
       ]
     },
     {
@@ -240,54 +256,9 @@
         "id": "ovwYBUG3wTDv"
       },
       "source": [
-        "Let's first train this model normally, without any wrapping. Which areas would you expect the model to do well in? Which areas should it do worse in?"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "colab": {
-          "base_uri": "https://localhost:8080/"
-        },
-        "id": "oPNxsGBRwaNA",
-        "outputId": "0598cef9-350c-4785-a7a9-51ed3b54fd4b"
-      },
-      "outputs": [
-        {
-          "name": "stdout",
-          "output_type": "stream",
-          "text": [
-            "Epoch 1/10\n",
-            "63/63 [==============================] - 1s 2ms/step - loss: 5.5708\n",
-            "Epoch 2/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 4.3687\n",
-            "Epoch 3/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 3.9064\n",
-            "Epoch 4/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 3.1653\n",
-            "Epoch 5/10\n",
-            "63/63 [==============================] - 0s 4ms/step - loss: 2.1027\n",
-            "Epoch 6/10\n",
-            "63/63 [==============================] - 0s 3ms/step - loss: 1.6488\n",
-            "Epoch 7/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 1.3093\n",
-            "Epoch 8/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 1.1078\n",
-            "Epoch 9/10\n",
-            "63/63 [==============================] - 0s 2ms/step - loss: 0.9919\n",
-            "Epoch 10/10\n",
-            "63/63 [==============================] - 0s 3ms/step - loss: 0.8937\n"
-          ]
-        }
-      ],
-      "source": [
-        "standard_classifier.compile(\n",
-        "  optimizer=tf.keras.optimizers.Adam(learning_rate=2e-3),\n",
-        "  loss=tf.keras.losses.MeanSquaredError(),\n",
-        ")\n",
+        "Now, we are ready to evaluate our neural network. We use the test data to assess performance on the regression task, and visualize the predicted values against the true values.\n",
         "\n",
-        "history = standard_classifier.fit(x, y, epochs=10)\n"
+        "Given your observation of the data in the previous plot, where do you expect the model to perform well? Let's test the model and see:"
       ]
     },
     {
@@ -326,11 +297,29 @@
         }
       ],
       "source": [
-        "plt.scatter(x_val, y_val, s=0.5, label='truth')\n",
-        "plt.scatter(x_val, standard_classifier(x_val), s=0.5, label='predictions')\n",
+        "# Pass the test data through the network and predict the y values\n",
+        "y_predicted = dense_NN(x_test)\n",
+        "\n",
+        "# Visualize the true (x, y) pairs for the test data vs. the predicted values\n",
+        "plt.scatter(x_test, y_test, s=0.5, label='truth')\n",
+        "plt.scatter(x_test, y_predicted, s=0.5, label='predictions')\n",
         "plt.legend()"
       ]
     },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "\n",
+        "Write short (~1 sentence) answers to the questions below to complete the `TODO`s:\n",
+        "\n",
+        "#### **TODO: Analyzing the performance of standard regression model**\n",
+        "1. Where does the model perform well? How does this relate to aleatoric and epistemic uncertainty?\n",
+        "2. Where does the model perform poorly? How does this relate to aleatoric and epistemic uncertainty?"
+      ],
+      "metadata": {
+        "id": "7Vktjwfu0ReH"
+      }
+    },
     {
       "cell_type": "markdown",
       "metadata": {