Getting train_loss & test_loss as nan when executing traing & testing loop

[Arun-Kumar-Rathod]

Hello there I am currently here in the video (or say here in the ZTM solutions notebook)

I've gone through the code several times by looking for errors, comparing it with the solution notebook but I am not able to figure out

Why am Igetting nan in my train_loss , test_loss and no change in my accuracy ???

Link to my colab notebook

Please have a look at the code & its output below. Help if you know how to navigate around this problem. Thanks!

My code:

# import tqdm for progress bar
from tqdm.auto import tqdm

# set the seed and start the timer
torch.manual_seed(42)
train_time_start_on_cpu = timer()

# set the no. of epochs (smaller no. of faster running)
epochs = 3

# Create training & test loop
for epoch in tqdm(range(epochs)):
  print(f"Epoch: {epoch}\n-----")
  ### Training
  train_loss = 0

  # Add a loop to loop through the training batches
  for batch, (X, y) in enumerate(train_dataloader):
    model_0.train()
    # 1. forward pass
    y_pred = model_0(X)

    # 2. Calculate loss (per batch)
    loss = loss_fn(y_pred, y)
    train_loss += loss # Accumulate training loss

    # 3. Optimizer zero grad
    optimizer.zero_grad

    # 4. Loss backward backprop
    loss.backward()

    # 5. Optimizer step
    optimizer.step()

    # print out whats happening
    if batch % 400 == 0:
      print(f"Looked at {batch * len(X)} / {len(train_dataloader.dataset)} samples")

  # Divide total train loss by length of train dataloader
  train_loss /= len(train_dataloader)

  ### Testing loop
  test_loss, test_acc = 0.0, 0.0
  model_0.eval()
  with torch.inference_mode():
    for X, y in test_dataloader:
      # 1. Forward pass
      test_pred = model_0(X)

      # 2. Calculate the loss(accumatively)
      test_loss += loss_fn(test_pred, y)

      # 3. Accuracy
      test_acc += accuracy_fn(y_true=y, y_pred=test_pred.argmax(dim=1))

    # Calculate the test loss
    test_loss /= len(test_dataloader)

    # calculate the test  average per batch
    test_acc /= len(test_dataloader)

  # print out whats happening
  print(f"\nTrain loss:{train_loss}| Test loss: {test_loss} , Test accuracy: {test_acc:.2f}%")

# Calculate the training time
train_time_end_on_cpu = timer()
total_train_time = print_train_time(start = train_time_start_on_cpu,
                                    end = train_time_end_on_cpu,
                                    device=str(next(model_0.parameters()).device))

Output:

Epoch: 0
-----
Looked at 0 / 60000 samples
Looked at 12800 / 60000 samples
Looked at 25600 / 60000 samples
Looked at 38400 / 60000 samples
Looked at 51200 / 60000 samples

Train loss:nan| Test loss: nan , Test accuracy: 9.98%
Epoch: 1
-----
Looked at 0 / 60000 samples
Looked at 12800 / 60000 samples
Looked at 25600 / 60000 samples
Looked at 38400 / 60000 samples
Looked at 51200 / 60000 samples

Train loss:nan| Test loss: nan , Test accuracy: 9.98%
Epoch: 2
-----
Looked at 0 / 60000 samples
Looked at 12800 / 60000 samples
Looked at 25600 / 60000 samples
Looked at 38400 / 60000 samples
Looked at 51200 / 60000 samples

Train loss:nan| Test loss: nan , Test accuracy: 9.98%
train time on device cpu:34.610 seconds

[Arun-Kumar-Rathod]

Solved:
I forgot to add parantheses after optimizer.zero_grad ()😂