Inheritance and save_hyperparameters

[lewtun]

Hello Lightning folks!

Suppose I have a base model class that I'd like to inherit from as follows:

import pytorch_lightning as pl


class ParentModel(pl.LightningModule):
    def __init__(
        self,
        lr: float = 0.001,
    ):
        super(ParentModel, self).__init__()
        self.lr = lr

class ChildModel(ParentModel):
    def __init__(
        self,
        lr: float = 0.005,
        loss: str = "mse",
    ):
        super(ParentModel, self).__init__()
        self.lr = lr
        self.loss = loss

I would like to be able to access the hyperparameters of ChildModel and one way to do that is by including save_hyperparameters() in the __init__ as follows:

class ChildModel(ParentModel):
    def __init__(
        self,
        lr: float = 0.005,
        loss: str = "mse",
    ):
        super(ParentModel, self).__init__()
        self.lr = lr
        self.loss = loss
        self.save_hyperparameters()

However, I would like to avoid the need to call save_hyperparameters() in every class that inherits from ParentModel and I was wondering whether it is possible to do this in PyTorch Lightning somehow?

One idea I have in mind is something like a __post_init__ that calls save_hyperparameters() after the __init__ is called, but this doesn't seem to be supported.

Thanks!

[justusschock]

You could do something like this:

import pytorch_lightning as pl


class ParentModel(pl.LightningModule):
    def __init__(
        self,
        lr: float = 0.001,
        **kwargs
    ):
        super(ParentModel, self).__init__()
        self.save_hyperparameters()
        self.lr = lr

class ChildModel(ParentModel):
    def __init__(
        self,
        lr: float = 0.005,
        loss: str = "mse",
    ):
        super(ParentModel, self).__init__(lr=lr, loss=loss)
        self.loss = loss

That would save all hparams passed to the parent model (including the ones passed through the kwargs). If you want to go one step further, you could also include the following there:

for k, v in kwargs.items():
    setattr(self, k, v)

which sets all attributes that are passed through kwargs automatically as model attributes.
That means you could also spare the self.loss=loss line in the child model :)

[lewtun]

Thanks a lot @justusschock - that's very helpful!