Pytorch Workflow revisited not registering parameters correctly

[lobachevscki]

Hi!

Im in the 'Putting it all Together' section of the workflow fundamental https://www.learnpytorch.io/01_pytorch_workflow/#6-putting-it-all-together

Im not using Google Colab but my own installation of PyCharm Community and Anaconda. It works perfectly.

Now the problem is that creating the class doesnt seem to register the parameters as expected.

My code is the same than in the lessons, here it is:

import torch
from torch import nn
import matplotlib.pyplot as plt
device = 'cuda' if torch.cuda.is_available() else 'cpu'

weight = 0.7
bias = 0.23

start = 0
end = 1

step = 0.01

X = torch.arange(start, end, step).unsqueeze(dim = 1)
y = weight * X + bias

train_split = int(0.8*len(X))
X_train, y_train = X[:train_split], y[:train_split]
X_test, y_test = X[train_split:], y[train_split:]

class LinearRegressionModelV2(nn.Module):
    def __int__(self):
        super().__init__()
        self.linear_layer = nn.Linear(in_features=1, out_features=1)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.linear_layer(x)

But when i try to print the state_dict by simply it prints an empty dictionary. This is a test code for you to see the result:

torch.manual_seed(42)
model_1 = LinearRegressionModelV2()

print(model_1)
print(model_1.state_dict())
print('---------------------')
print(nn.Linear(in_features=1, out_features=1).state_dict())

Gives this result:

LinearRegressionModelV2()
OrderedDict()
---------------------
OrderedDict([('weight', tensor([[0.7645]])), ('bias', tensor([0.8300]))])

So if i create an instance of my newly created linear class it doesnt register any parameters it seems, for comparison if i just simply instance directly from nn.Linear the parameters are there (as seen above).

Moreover if write this

print(next(nn.Linear(in_features=1, out_features=1).parameters()).device)

It prints the device but if try to do that with the instance of the class i created it prompts an error.

This blocks me completely for moving forward with the lesson.

So, what i understood from reading in the internet is that my class is not inheriting the parameters from the nn.Linear as we should expect, which is in any case what is being reflected in the behavior described above. Moreover, in the previous lessons everything worked perfectly and the difference is precisely that I literally defined the parameters with the nn.Parameter method. So the parameters are definitely missing but i simply dont have the knowledge to know how to fix it.

Any suggestions here?

Thanks

[mrdbourke]

Hi @lobachevscki ,

I read your code and thought this is strange... because your code looks fine.

Then I went through it line by line and found a silent error.

A small typo in your __init__() method.

Your code:

class LinearRegressionModelV2(nn.Module):
    def __int__(self): # "int" not "init"

Corrected code:

class LinearRegressionModelV2(nn.Module):
    def __init__(self):

A small error but it is causing exactly the error you're talking about.

Full example:

import torch
from torch import nn
import matplotlib.pyplot as plt
device = 'cuda' if torch.cuda.is_available() else 'cpu'

weight = 0.7
bias = 0.23

start = 0
end = 1

step = 0.01

X = torch.arange(start, end, step).unsqueeze(dim = 1)
y = weight * X + bias

train_split = int(0.8*len(X))
X_train, y_train = X[:train_split], y[:train_split]
X_test, y_test = X[train_split:], y[train_split:]

class LinearRegressionModelV2(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear_layer = nn.Linear(in_features=1, out_features=1)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.linear_layer(x)

And then:

torch.manual_seed(42)
model_1 = LinearRegressionModelV2()

print(model_1)
print(model_1.state_dict())
print('---------------------')
print(nn.Linear(in_features=1, out_features=1).state_dict())

See output:

LinearRegressionModelV2(
  (linear_layer): Linear(in_features=1, out_features=1, bias=True)
)
OrderedDict([('linear_layer.weight', tensor([[0.7645]])), ('linear_layer.bias', tensor([0.8300]))])
---------------------
OrderedDict([('weight', tensor([[-0.2343]])), ('bias', tensor([0.9186]))])

---

As a side note, if you've got access to ChatGPT, it's quite good at spotting these kind of silent but small errors:

[lobachevscki]

Oh, i guess i learned that i need to be extra careful then. I dont have access to chaptGpt but this is definitely a case where it would have save a lot of time.

Thanks!

[shisirkha]

@lobachevscki You can use Bing AI that is powered by chatgpt-4 i think it is better than ChatGPT and safer and harder to jailbreak