1
1
2
- .. DO NOT EDIT.
3
- .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
4
- .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
5
- .. "beginner/Pretraining_Vgg_from_scratch.py"
6
- .. LINE NUMBERS ARE GIVEN BELOW.
7
-
8
- .. only :: html
9
-
10
- .. note ::
11
- :class: sphx-glr-download-link-note
12
-
13
- Click :ref: `here <sphx_glr_download_beginner_Pretraining_Vgg_from_scratch.py >`
14
- to download the full example code
15
-
16
- .. rst-class :: sphx-glr-example-title
17
-
18
- .. _sphx_glr_beginner_Pretraining_Vgg_from_scratch.py :
19
-
20
-
21
2
Pre-training VGG from scratch
22
3
============================
23
4
@@ -73,7 +54,7 @@ VGG within the training time suggested in the paper.
73
54
Setup
74
55
--------
75
56
76
- .. note :: if you are running this in Google Colab, install ``albumentations`` by running:
57
+ .. note :: If you are running this in Google Colab, install ``albumentations`` by running:
77
58
78
59
.. code-block :: python
79
60
82
63
83
64
First, let's import the required dependencies:
84
65
85
- .. GENERATED FROM PYTHON SOURCE LINES 67-92
86
66
87
67
.. code-block :: default
88
68
@@ -110,22 +90,6 @@ First, let's import the required dependencies:
110
90
111
91
device = 'cuda' if torch.cuda.is_available() else 'cpu'
112
92
113
-
114
-
115
-
116
-
117
-
118
- .. rst-class :: sphx-glr-script-out
119
-
120
- .. code-block :: none
121
-
122
- albumentations are already installed
123
-
124
-
125
-
126
-
127
- .. GENERATED FROM PYTHON SOURCE LINES 93-100
128
-
129
93
VGG Configuration
130
94
-----------------
131
95
@@ -134,7 +98,7 @@ We use the CIFAR100 dataset. The authors of the VGG paper scale images ``isotrop
134
98
which means increasing the size of an image while maintaining its proportions,
135
99
preventing distortion and maintaining the consistency of the object.
136
100
137
- .. GENERATED FROM PYTHON SOURCE LINES 100-140
101
+
138
102
139
103
.. code-block :: default
140
104
@@ -185,7 +149,7 @@ preventing distortion and maintaining the consistency of the object.
185
149
186
150
187
151
188
- .. GENERATED FROM PYTHON SOURCE LINES 141-147
152
+
189
153
190
154
.. note :: In the code above, we have defined the batch size as 32,
191
155
which is recommended for Google Colab. However, if you are
@@ -194,7 +158,7 @@ preventing distortion and maintaining the consistency of the object.
194
158
size according to your preference and hardware capabilities.
195
159
196
160
197
- .. GENERATED FROM PYTHON SOURCE LINES 149-174
161
+
198
162
199
163
Defining the dataset
200
164
--------------------
@@ -222,7 +186,7 @@ To apply preprocessing, we need to override the CIFAR100 class that we have impo
222
186
``torchvision.datasets `` with a custom class:
223
187
224
188
225
- .. GENERATED FROM PYTHON SOURCE LINES 174-227
189
+
226
190
227
191
.. code-block :: default
228
192
@@ -286,7 +250,7 @@ To apply preprocessing, we need to override the CIFAR100 class that we have impo
286
250
287
251
288
252
289
- .. GENERATED FROM PYTHON SOURCE LINES 228-238
253
+
290
254
291
255
Define Model
292
256
------------
@@ -299,7 +263,7 @@ We will use two main components to define the model:
299
263
* ``Config_channels ``: This refers to the number of output channels for each layer.
300
264
* ``Config_kernels ``: This refers to the kernel size (or filter size) for each layer.
301
265
302
- .. GENERATED FROM PYTHON SOURCE LINES 238-266
266
+
303
267
304
268
.. code-block :: default
305
269
@@ -338,12 +302,10 @@ We will use two main components to define the model:
338
302
339
303
340
304
341
- .. GENERATED FROM PYTHON SOURCE LINES 267-269
342
305
343
- Next, we define a model class that generates a model with a choice of six versions.
344
306
307
+ Next, we define a model class that generates a model with a choice of six versions.
345
308
346
- .. GENERATED FROM PYTHON SOURCE LINES 269-363
347
309
348
310
.. code-block :: default
349
311
@@ -448,7 +410,7 @@ Next, we define a model class that generates a model with a choice of six versio
448
410
449
411
450
412
451
- .. GENERATED FROM PYTHON SOURCE LINES 364-377
413
+
452
414
453
415
Initializing Model Weights
454
416
----------------------------
@@ -464,7 +426,7 @@ to initialize the model weights. Specifically, we will apply Xavier
464
426
initialization to the first few layers and the last few layers, while using
465
427
random initialization for the remaining layers.
466
428
467
- .. GENERATED FROM PYTHON SOURCE LINES 377-399
429
+
468
430
469
431
.. code-block :: default
470
432
@@ -497,15 +459,15 @@ random initialization for the remaining layers.
497
459
498
460
499
461
500
- .. GENERATED FROM PYTHON SOURCE LINES 400-405
462
+
501
463
502
464
Training the Model
503
465
------------------
504
466
505
467
First, let's define top-k error.
506
468
507
469
508
- .. GENERATED FROM PYTHON SOURCE LINES 405-422
470
+
509
471
510
472
.. code-block :: default
511
473
@@ -533,13 +495,13 @@ First, let's define top-k error.
533
495
534
496
535
497
536
- .. GENERATED FROM PYTHON SOURCE LINES 423-426
498
+
537
499
538
500
Next, we initiate the model and loss function, optimizer and schedulers. In the VGG model,
539
501
they use a softmax output, Momentum Optimizer, and scheduling based on accuracy.
540
502
541
503
542
- .. GENERATED FROM PYTHON SOURCE LINES 426-434
504
+
543
505
544
506
.. code-block :: default
545
507
@@ -630,12 +592,12 @@ they use a softmax output, Momentum Optimizer, and scheduling based on accuracy.
630
592
631
593
632
594
633
- .. GENERATED FROM PYTHON SOURCE LINES 435-437
595
+
634
596
635
597
As mentioned above, we are using the ``CIFAR100 `` dataset and set gradient
636
598
clipping to 1.0 to prevent gradient exploding.
637
599
638
- .. GENERATED FROM PYTHON SOURCE LINES 437-570
600
+
639
601
640
602
.. code-block :: default
641
603
@@ -3432,14 +3394,14 @@ clipping to 1.0 to prevent gradient exploding.
3432
3394
3433
3395
3434
3396
3435
- .. GENERATED FROM PYTHON SOURCE LINES 571-575
3397
+
3436
3398
3437
3399
(Optional) Additional Exercise: ImageNet
3438
3400
--------------------------------------------
3439
3401
3440
3402
You can apply the same model that we have trained above with another popular dataset called ImageNet:
3441
3403
3442
- .. GENERATED FROM PYTHON SOURCE LINES 575-644
3404
+
3443
3405
3444
3406
.. code-block :: default
3445
3407
@@ -3519,7 +3481,7 @@ You can apply the same model that we have trained above with another popular dat
3519
3481
3520
3482
3521
3483
3522
- .. GENERATED FROM PYTHON SOURCE LINES 645-660
3484
+
3523
3485
3524
3486
Conclusion
3525
3487
----------
0 commit comments