modls

Author: Programmer-RD-AI

going_modular/05_pytorch_going_modular_cell_mode.ipynb

@@ -0,0 +1,15 @@
+# 05. PyTorch Going Modular
+
+The main goal of section [05. PyTorch Going Modular](https://www.learnpytorch.io/05_pytorch_going_modular/) is to: **turn useful notebook code cells into reusable Python scripts (`.py` files)**.
+
+This directory contains all the necessary materials for doing so.
+
+They breakdown as follows:
+* `going_modular/` - directory of Python helper scripts for running PyTorch code (generated by `05_pytorch_going_modular_script_mode.ipynb`).
+* `models/` - trained PyTorch models that come as a result of running notebook 05. Going Modular Part 1 and Part 2.
+* [`05_pytorch_going_modular_cell_mode.ipynb`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/05_pytorch_going_modular_cell_mode.ipynb) - Part 1/2 notebooks for teaching the materials for section 05. This notebook takes the most useful code from notebook 04 and streamlines it.
+* [`05_pytorch_going_modular_script_mode.ipynb`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/05_pytorch_going_modular_script_mode.ipynb) - Part 2/2 notebooks for teaching the materials for section 05. This notebooks turns the most useful code cells from Part 1 into the Python scripts contained in `going_modular/`.
+
+For this section, we're going to see how the Part 1 notebook (cell mode) turns into the Part 2 notebook (script mode).
+
+Doing this will result in us having a directory with the same structure as the `going_modular/` directory above.

going_modular/05_pytorch_going_modular_script_mode.ipynb

@@ -0,0 +1,13 @@
+# Going Modular Scripts
+
+The Python scripts in this directory were generated using the notebook [05. Going Modular Part 2 (script mode)](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/05_pytorch_going_modular_script_mode.ipynb).
+
+They breakdown as follows: 
+* `data_setup.py` - a file to prepare and download data if needed.
+* `engine.py` - a file containing various training functions.
+* `model_builder.py` - a file to create a PyTorch TinyVGG model.
+* `train.py` - a file to leverage all other files and train a target PyTorch model.
+* `utils.py` - a file dedicated to helpful utility functions.
+* **Extra:** `predictions.py` - a file for making predictions with a trained PyTorch model and input image (the main function, `pred_and_plot_image()` was originally created in [06. PyTorch Transfer Learning section 6](https://www.learnpytorch.io/06_pytorch_transfer_learning/#6-make-predictions-on-images-from-the-test-set)).
+
+For an explanation of how this was done, refer to section [05. PyTorch Going Modular of the learnpytorch.io book](https://www.learnpytorch.io/05_pytorch_going_modular/).

going_modular/README.md

@@ -0,0 +1,65 @@
+"""
+Contains functionality for creating PyTorch DataLoaders for 
+image classification data.
+"""
+import os
+
+from torchvision import datasets, transforms
+from torch.utils.data import DataLoader
+
+NUM_WORKERS = os.cpu_count()
+
+def create_dataloaders(
+    train_dir: str, 
+    test_dir: str, 
+    transform: transforms.Compose, 
+    batch_size: int, 
+    num_workers: int=NUM_WORKERS
+):
+  """Creates training and testing DataLoaders.
+
+  Takes in a training directory and testing directory path and turns
+  them into PyTorch Datasets and then into PyTorch DataLoaders.
+
+  Args:
+    train_dir: Path to training directory.
+    test_dir: Path to testing directory.
+    transform: torchvision transforms to perform on training and testing data.
+    batch_size: Number of samples per batch in each of the DataLoaders.
+    num_workers: An integer for number of workers per DataLoader.
+
+  Returns:
+    A tuple of (train_dataloader, test_dataloader, class_names).
+    Where class_names is a list of the target classes.
+    Example usage:
+      train_dataloader, test_dataloader, class_names = \
+        = create_dataloaders(train_dir=path/to/train_dir,
+                             test_dir=path/to/test_dir,
+                             transform=some_transform,
+                             batch_size=32,
+                             num_workers=4)
+  """
+  # Use ImageFolder to create dataset(s)
+  train_data = datasets.ImageFolder(train_dir, transform=transform)
+  test_data = datasets.ImageFolder(test_dir, transform=transform)
+
+  # Get class names
+  class_names = train_data.classes
+
+  # Turn images into data loaders
+  train_dataloader = DataLoader(
+      train_data,
+      batch_size=batch_size,
+      shuffle=True,
+      num_workers=num_workers,
+      pin_memory=True,
+  )
+  test_dataloader = DataLoader(
+      test_data,
+      batch_size=batch_size,
+      shuffle=False,
+      num_workers=num_workers,
+      pin_memory=True,
+  )
+
+  return train_dataloader, test_dataloader, class_names

going_modular/going_modular/README.md

@@ -0,0 +1,195 @@
+"""
+Contains functions for training and testing a PyTorch model.
+"""
+import torch
+
+from tqdm.auto import tqdm
+from typing import Dict, List, Tuple
+
+def train_step(model: torch.nn.Module, 
+               dataloader: torch.utils.data.DataLoader, 
+               loss_fn: torch.nn.Module, 
+               optimizer: torch.optim.Optimizer,
+               device: torch.device) -> Tuple[float, float]:
+    """Trains a PyTorch model for a single epoch.
+
+    Turns a target PyTorch model to training mode and then
+    runs through all of the required training steps (forward
+    pass, loss calculation, optimizer step).
+
+    Args:
+    model: A PyTorch model to be trained.
+    dataloader: A DataLoader instance for the model to be trained on.
+    loss_fn: A PyTorch loss function to minimize.
+    optimizer: A PyTorch optimizer to help minimize the loss function.
+    device: A target device to compute on (e.g. "cuda" or "cpu").
+
+    Returns:
+    A tuple of training loss and training accuracy metrics.
+    In the form (train_loss, train_accuracy). For example:
+
+    (0.1112, 0.8743)
+    """
+    # Put model in train mode
+    model.train()
+
+    # Setup train loss and train accuracy values
+    train_loss, train_acc = 0, 0
+
+    # Loop through data loader data batches
+    for batch, (X, y) in enumerate(dataloader):
+        # Send data to target device
+        X, y = X.to(device), y.to(device)
+
+        # 1. Forward pass
+        y_pred = model(X)
+
+        # 2. Calculate  and accumulate loss
+        loss = loss_fn(y_pred, y)
+        train_loss += loss.item() 
+
+        # 3. Optimizer zero grad
+        optimizer.zero_grad()
+
+        # 4. Loss backward
+        loss.backward()
+
+        # 5. Optimizer step
+        optimizer.step()
+
+        # Calculate and accumulate accuracy metric across all batches
+        y_pred_class = torch.argmax(torch.softmax(y_pred, dim=1), dim=1)
+        train_acc += (y_pred_class == y).sum().item()/len(y_pred)
+
+    # Adjust metrics to get average loss and accuracy per batch 
+    train_loss = train_loss / len(dataloader)
+    train_acc = train_acc / len(dataloader)
+    return train_loss, train_acc
+
+def test_step(model: torch.nn.Module, 
+              dataloader: torch.utils.data.DataLoader, 
+              loss_fn: torch.nn.Module,
+              device: torch.device) -> Tuple[float, float]:
+    """Tests a PyTorch model for a single epoch.
+
+    Turns a target PyTorch model to "eval" mode and then performs
+    a forward pass on a testing dataset.
+
+    Args:
+    model: A PyTorch model to be tested.
+    dataloader: A DataLoader instance for the model to be tested on.
+    loss_fn: A PyTorch loss function to calculate loss on the test data.
+    device: A target device to compute on (e.g. "cuda" or "cpu").
+
+    Returns:
+    A tuple of testing loss and testing accuracy metrics.
+    In the form (test_loss, test_accuracy). For example:
+
+    (0.0223, 0.8985)
+    """
+    # Put model in eval mode
+    model.eval() 
+
+    # Setup test loss and test accuracy values
+    test_loss, test_acc = 0, 0
+
+    # Turn on inference context manager
+    with torch.inference_mode():
+        # Loop through DataLoader batches
+        for batch, (X, y) in enumerate(dataloader):
+            # Send data to target device
+            X, y = X.to(device), y.to(device)
+
+            # 1. Forward pass
+            test_pred_logits = model(X)
+
+            # 2. Calculate and accumulate loss
+            loss = loss_fn(test_pred_logits, y)
+            test_loss += loss.item()
+
+            # Calculate and accumulate accuracy
+            test_pred_labels = test_pred_logits.argmax(dim=1)
+            test_acc += ((test_pred_labels == y).sum().item()/len(test_pred_labels))
+
+    # Adjust metrics to get average loss and accuracy per batch 
+    test_loss = test_loss / len(dataloader)
+    test_acc = test_acc / len(dataloader)
+    return test_loss, test_acc
+
+def train(model: torch.nn.Module, 
+          train_dataloader: torch.utils.data.DataLoader, 
+          test_dataloader: torch.utils.data.DataLoader, 
+          optimizer: torch.optim.Optimizer,
+          loss_fn: torch.nn.Module,
+          epochs: int,
+          device: torch.device) -> Dict[str, List]:
+    """Trains and tests a PyTorch model.
+
+    Passes a target PyTorch models through train_step() and test_step()
+    functions for a number of epochs, training and testing the model
+    in the same epoch loop.
+
+    Calculates, prints and stores evaluation metrics throughout.
+
+    Args:
+    model: A PyTorch model to be trained and tested.
+    train_dataloader: A DataLoader instance for the model to be trained on.
+    test_dataloader: A DataLoader instance for the model to be tested on.
+    optimizer: A PyTorch optimizer to help minimize the loss function.
+    loss_fn: A PyTorch loss function to calculate loss on both datasets.
+    epochs: An integer indicating how many epochs to train for.
+    device: A target device to compute on (e.g. "cuda" or "cpu").
+
+    Returns:
+    A dictionary of training and testing loss as well as training and
+    testing accuracy metrics. Each metric has a value in a list for 
+    each epoch.
+    In the form: {train_loss: [...],
+              train_acc: [...],
+              test_loss: [...],
+              test_acc: [...]} 
+    For example if training for epochs=2: 
+             {train_loss: [2.0616, 1.0537],
+              train_acc: [0.3945, 0.3945],
+              test_loss: [1.2641, 1.5706],
+              test_acc: [0.3400, 0.2973]} 
+    """
+    # Create empty results dictionary
+    results = {"train_loss": [],
+               "train_acc": [],
+               "test_loss": [],
+               "test_acc": []
+    }
+    
+    # Make sure model on target device
+    model.to(device)
+
+    # Loop through training and testing steps for a number of epochs
+    for epoch in tqdm(range(epochs)):
+        train_loss, train_acc = train_step(model=model,
+                                          dataloader=train_dataloader,
+                                          loss_fn=loss_fn,
+                                          optimizer=optimizer,
+                                          device=device)
+        test_loss, test_acc = test_step(model=model,
+          dataloader=test_dataloader,
+          loss_fn=loss_fn,
+          device=device)
+
+        # Print out what's happening
+        print(
+          f"Epoch: {epoch+1} | "
+          f"train_loss: {train_loss:.4f} | "
+          f"train_acc: {train_acc:.4f} | "
+          f"test_loss: {test_loss:.4f} | "
+          f"test_acc: {test_acc:.4f}"
+        )
+
+        # Update results dictionary
+        results["train_loss"].append(train_loss)
+        results["train_acc"].append(train_acc)
+        results["test_loss"].append(test_loss)
+        results["test_acc"].append(test_acc)
+
+    # Return the filled results at the end of the epochs
+    return results

going_modular/going_modular/__pycache__/data_setup.cpython-311.pyc

@@ -0,0 +1,56 @@
+"""
+Contains PyTorch model code to instantiate a TinyVGG model.
+"""
+import torch
+from torch import nn 
+
+class TinyVGG(nn.Module):
+    """Creates the TinyVGG architecture.
+
+    Replicates the TinyVGG architecture from the CNN explainer website in PyTorch.
+    See the original architecture here: https://poloclub.github.io/cnn-explainer/
+
+    Args:
+    input_shape: An integer indicating number of input channels.
+    hidden_units: An integer indicating number of hidden units between layers.
+    output_shape: An integer indicating number of output units.
+    """
+    def __init__(self, input_shape: int, hidden_units: int, output_shape: int) -> None:
+        super().__init__()
+        self.conv_block_1 = nn.Sequential(
+          nn.Conv2d(in_channels=input_shape, 
+                    out_channels=hidden_units, 
+                    kernel_size=3, 
+                    stride=1, 
+                    padding=0),  
+          nn.ReLU(),
+          nn.Conv2d(in_channels=hidden_units, 
+                    out_channels=hidden_units,
+                    kernel_size=3,
+                    stride=1,
+                    padding=0),
+          nn.ReLU(),
+          nn.MaxPool2d(kernel_size=2,
+                        stride=2)
+        )
+        self.conv_block_2 = nn.Sequential(
+          nn.Conv2d(hidden_units, hidden_units, kernel_size=3, padding=0),
+          nn.ReLU(),
+          nn.Conv2d(hidden_units, hidden_units, kernel_size=3, padding=0),
+          nn.ReLU(),
+          nn.MaxPool2d(2)
+        )
+        self.classifier = nn.Sequential(
+          nn.Flatten(),
+          # Where did this in_features shape come from? 
+          # It's because each layer of our network compresses and changes the shape of our inputs data.
+          nn.Linear(in_features=hidden_units*13*13,
+                    out_features=output_shape)
+        )
+    
+    def forward(self, x: torch.Tensor):
+        x = self.conv_block_1(x)
+        x = self.conv_block_2(x)
+        x = self.classifier(x)
+        return x
+        # return self.classifier(self.block_2(self.block_1(x))) # <- leverage the benefits of operator fusion