Video 78: Loss not decreasing in linear model

[pthomas92]

I'm following video 78 for classification of the circles data set. When checking if the model has the capacity to learn despite not being able to classify circles correctly, for some reason mine is not updating. Code below:

Generate the data, using y = mx + c

import torch
from torch import nn

weight = 0.7
bias = 0.3
start = 0
end = 1
step = 0.01

X_regression = torch.arange(start, end, step).unsqueeze(dim = 1)
y_regression = weight * X_regression + bias

train_split = int(0.8 * len(X_regression))
X_train_regression, y_train_regression = X_regression[:train_split], y_regression[:train_split]
X_test_regression, y_test_regression = X_regression[train_split:], y_regression[train_split:]

plot_predictions(train_data = X_train_regression, train_labels = y_train_regression,
                 test_data = X_test_regression, test_labels = y_test_regression)

Create the model, and move it and the data to the gpu:

device = 'cuda' if torch.cuda.is_available() else 'cpu'

model_2 = nn.Sequential(
    nn.Linear(in_features = 1,
              out_features = 10),
    nn.Linear(in_features = 10,
              out_features = 10),
    nn.Linear(in_features = 10,
              out_features = 1)
).to(device)

loss_fn = nn.L1Loss()
optimiser = torch.optim.SGD(lr = 0.01,
                        params = model_2.parameters())

X_train_regression, y_train_regression = X_train_regression.to(device), y_train_regression.to(device)
X_test_regression, y_test_regression = X_test_regression.to(device), y_test_regression.to(device)

Training the model

torch.cuda.manual_seed(42)
torch.manual_seed(42)

epochs = 1000

for epoch in range(epochs):
  
  model_2.train()
  y_pred = model_2(X_train_regression)
  loss = loss_fn(y_pred, y_train_regression)

  optimiser.zero_grad()
  loss.backward()
  optimiser.step

  model_2.eval()
  with torch.inference_mode():
    test_pred = model_2(X_test_regression)
    test_loss = loss_fn(test_pred, y_test_regression)

  if epoch % (epochs/10) == 0:
    print(f'>Epoch: {epoch} | Loss: {loss} | Test Loss: {test_loss}')

Which gives me the following output:

>Epoch: 0 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 100 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 200 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 300 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 400 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 500 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 600 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 700 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 800 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936
>Epoch: 900 | Loss: 0.7598584890365601 | Test Loss: 0.953961968421936

My code is correct as far as I can see, so I don't know how to fix this. Is it a Google Colab issue?

Thanks

[Perian-Yan]

Hi,
your model is actually not updating parameters. There are parentheses after optimiser.step -> optimiser.step()

[pthomas92]

Oh I can't believe I missed that! Guess I'd been staring at the screen for too long. Thanks a lot, it's fixed now.