parameters not having gradient when calling training_step() outside of trainer #14437
Unanswered
malfonsoarquimea
asked this question in
Lightning Trainer API: Trainer, LightningModule, LightningDataModule
Replies: 1 comment
-
I believe it's totally expected because gradients will be available only after backward call. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi! I am writing pytest tests for a model that I coded and I am now writing the tests for the training_step() function to make sure that everything is working as expected.
My test is simple, I initialize the model and then call
model.training_step(batch,0)
, wherebatch
is a list with data with the same structure than the one that will be provided by the dataloader.Everything seems to work and the training_step returns a loss as expected, but then when I try to get the gradients of the model with
layer.weight.grad
I get None. (given that layer is a linear layer of the model). The parameters have the attributerequires_grad
set to True when I just printlayer.weight.requires_grad
If I understand it correctly, I should be seen some gradients here, and not None values. What am I doing wrong?
Thanks in advance !
Beta Was this translation helpful? Give feedback.
All reactions