You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
" <img src=\"https://i.ibb.co/xfJbPmL/github.png\" height=\"70px\" style=\"padding-bottom:5px;\" />View Source on GitHub</a></td>\n",
17
17
"</table>\n",
18
18
"\n",
@@ -27,8 +27,8 @@
27
27
},
28
28
"outputs": [],
29
29
"source": [
30
-
"# Copyright 2023 MIT Introduction to Deep Learning. All Rights Reserved.\n",
31
-
"#\n",
30
+
"# Copyright 2024 MIT Introduction to Deep Learning. All Rights Reserved.\n",
31
+
"#\n",
32
32
"# Licensed under the MIT License. You may not use this file except in compliance\n",
33
33
"# with the License. Use and/or modification of this code outside of MIT Introduction\n",
34
34
"# to Deep Learning must reference:\n",
@@ -53,7 +53,7 @@
53
53
"\n",
54
54
"## 0.1 Install TensorFlow\n",
55
55
"\n",
56
-
"TensorFlow is a software library extensively used in machine learning. Here we'll learn how computations are represented and how to define a simple neural network in TensorFlow. For all the labs in Introduction to Deep Learning 2023, we'll be using the latest version of TensorFlow, TensorFlow 2, which affords great flexibility and the ability to imperatively execute operations, just like in Python. You'll notice that TensorFlow 2 is quite similar to Python in its syntax and imperative execution. Let's install TensorFlow and a couple of dependencies.\n"
56
+
"TensorFlow is a software library extensively used in machine learning. Here we'll learn how computations are represented and how to define a simple neural network in TensorFlow. For all the labs in Introduction to Deep Learning 2024, we'll be using the latest version of TensorFlow, TensorFlow 2, which affords great flexibility and the ability to imperatively execute operations, just like in Python. You'll notice that TensorFlow 2 is quite similar to Python in its syntax and imperative execution. Let's install TensorFlow and a couple of dependencies.\n"
57
57
]
58
58
},
59
59
{
@@ -77,25 +77,29 @@
77
77
},
78
78
{
79
79
"cell_type": "markdown",
80
-
"metadata": {},
80
+
"metadata": {
81
+
"id": "nrWxnP8Cn0En"
82
+
},
81
83
"source": [
82
-
"## 0.2 Set Up Comet\n",
84
+
"## 0.2 Set Up Comet ML\n",
83
85
"\n",
84
-
"When training models, it can be useful to visualize information about the model with plots. We can do this manually, but here, we'll show you how to do this using a tool called Comet, which generates loss and GPU usage curves for you.\n",
86
+
"When training models, it can be useful to visualize information about the model with plots. We can do this manually, but here, we'll show you how to do this using a tool called [Comet ML](https://www.comet.com/docs/v2/), which generates loss and GPU usage curves for you. As you'll see, Comet also enables easy saving of your experiments to a central interface.\n",
85
87
"\n",
86
88
"First, sign up for a Comet account [at this link](https://www.comet.com/signup?utm_source=mit_dl&utm_medium=partner&utm_content=github\n",
87
-
") (you can use your Google or Github account). Running this cell will prompt you to enter your API Key (which you can find by pressing the '?' in the top right corner and then 'Quickstart Guide' - it is on the right hand side of the page)."
89
+
") (you can use your Google or Github account). Running this cell will prompt you to enter your API Key, which you can find either in the first 'Get Started with Comet' page or by pressing the '?' in the top right corner and then 'Quickstart Guide' -- it is on the right hand side of the page."
"# Use tf.zeros to initialize a 4-d Tensor of zeros with size 10 x 256 x 256 x 3.\n",
196
+
"# Use tf.zeros to initialize a 4-d Tensor of zeros with size 10 x 256 x 256 x 3.\n",
193
197
"# You can think of this as 10 images where each image is RGB 256 x 256.\n",
194
198
"images = tf.zeros([10, 256, 256, 3]) # TODO\n",
195
199
"# images = # TODO\n",
@@ -339,7 +343,7 @@
339
343
"## 1.3 Neural networks in TensorFlow\n",
340
344
"We can also define neural networks in TensorFlow. TensorFlow uses a high-level API called [Keras](https://www.tensorflow.org/guide/keras) that provides a powerful, intuitive framework for building and training deep learning models.\n",
341
345
"\n",
342
-
"Let's first consider the example of a simple perceptron defined by just one dense layer: $ y = \\sigma(Wx + b)$, where $W$ represents a matrix of weights, $b$ is a bias, $x$ is the input, $\\sigma$ is the sigmoid activation function, and $y$ is the output. We can also visualize this operation using a graph:\n",
346
+
"Let's first consider the example of a simple perceptron defined by just one dense layer: $ y = \\sigma(Wx + b)$, where $W$ represents a matrix of weights, $b$ is a bias, $x$ is the input, $\\sigma$ is the sigmoid activation function, and $y$ is the output. We can also visualize this operation using a graph:\n",
"Conveniently, TensorFlow has defined a number of ```Layers``` that are commonly used in neural networks, for example a [```Dense```](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense?version=stable). Now, instead of using a single ```Layer``` to define our simple neural network, we'll use the [`Sequential`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Sequential) model from Keras and a single [`Dense` ](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/Dense) layer to define our network. With the `Sequential` API, you can readily create neural networks by stacking together layers like building blocks."
407
+
"Conveniently, TensorFlow has defined a number of ```Layers``` that are commonly used in neural networks, for example a [```Dense```](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense?version=stable). Now, instead of using a single ```Layer``` to define our simple neural network, we'll use the [`Sequential`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Sequential) model from Keras and a single [`Dense` ](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/Dense) layer to define our network. With the `Sequential` API, you can readily create neural networks by stacking together layers like building blocks."
404
408
]
405
409
},
406
410
{
@@ -420,12 +424,12 @@
420
424
"# Define the number of outputs\n",
421
425
"n_output_nodes = 3\n",
422
426
"\n",
423
-
"# First define the model\n",
427
+
"# First define the model\n",
424
428
"model = Sequential()\n",
425
429
"\n",
426
430
"'''TODO: Define a dense (fully connected) layer to compute z'''\n",
427
431
"# Remember: dense layers are defined by the parameters W and b!\n",
428
-
"# You can read more about the initialization of W and b in the TF documentation :)\n",
432
+
"# You can read more about the initialization of W and b in the TF documentation :)\n",
"is one of the most important parts of TensorFlow and is the backbone of training with\n",
615
-
"[backpropagation](https://en.wikipedia.org/wiki/Backpropagation). We will use the TensorFlow GradientTape [`tf.GradientTape`](https://www.tensorflow.org/api_docs/python/tf/GradientTape?version=stable) to trace operations for computing gradients later.\n",
618
+
"is one of the most important parts of TensorFlow and is the backbone of training with\n",
619
+
"[backpropagation](https://en.wikipedia.org/wiki/Backpropagation). We will use the TensorFlow GradientTape [`tf.GradientTape`](https://www.tensorflow.org/api_docs/python/tf/GradientTape?version=stable) to trace operations for computing gradients later.\n",
616
620
"\n",
617
621
"When a forward pass is made through the network, all forward-pass operations get recorded to a \"tape\"; then, to compute the gradient, the tape is played backwards. By default, the tape is discarded after it is played backwards; this means that a particular `tf.GradientTape` can only\n",
618
-
"compute one gradient, and subsequent calls throw a runtime error. However, we can compute multiple gradients over the same computation by creating a ```persistent``` gradient tape.\n",
622
+
"compute one gradient, and subsequent calls throw a runtime error. However, we can compute multiple gradients over the same computation by creating a ```persistent``` gradient tape.\n",
619
623
"\n",
620
624
"First, we will look at how we can compute gradients using GradientTape and access them for computation. We define the simple function $ y = x^2$ and compute the gradient:"
621
625
]
@@ -678,7 +682,7 @@
678
682
"# Define the target value\n",
679
683
"x_f = 4\n",
680
684
"\n",
681
-
"# We will run SGD for a number of iterations. At each iteration, we compute the loss,\n",
685
+
"# We will run SGD for a number of iterations. At each iteration, we compute the loss,\n",
682
686
"# compute the derivative of the loss with respect to x, and perform the SGD update.\n",
683
687
"for i in range(500):\n",
684
688
" with tf.GradientTape() as tape:\n",
@@ -687,7 +691,7 @@
687
691
" # loss = # TODO\n",
688
692
"\n",
689
693
" # Here's where we're going to use Comet! We'll log our loss values into the experiment like so:\n",
" grad = tape.gradient(loss, x) # compute the derivative of the loss with respect to x\n",
@@ -710,7 +714,9 @@
710
714
"id": "pC7czCwk3ceH"
711
715
},
712
716
"source": [
713
-
"`GradientTape` provides an extremely flexible framework for automatic differentiation. In order to back propagate errors through a neural network, we track forward passes on the Tape, use this information to determine the gradients, and then use these gradients for optimization using SGD."
717
+
"`GradientTape` provides an extremely flexible framework for automatic differentiation. In order to back propagate errors through a neural network, we track forward passes on the Tape, use this information to determine the gradients, and then use these gradients for optimization using SGD.\n",
718
+
"\n",
719
+
"In the above code block, you also saw how you can directly log metrics, like loss values, to Comet. These are then retained within your profile's web interface, where you are able to visualize the results of your different training runs!"
0 commit comments