How can one use an external optimizer with LightningCLI? #11016
-
I would like to use Adafactor as my optimizer with LightningCLI. I've tried the method described in the documentation for custom optimizers but it didn't work. Can anybody tell me how they would train a model with this optimizer using LightningCLI? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
Hey @goncalomcorreia, Did you add the optimizer to the Optimizer registry? Here is where we pre-register all the optimizers: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/utilities/cli.py#L83 |
Beta Was this translation helpful? Give feedback.
-
Hi! I got it to work in the meantime. I added this to the main file where I call CLI:
The main issue was in the config file---apparently one needs to write:
instead of:
Doing the former, I got it to work. By the way, is there a way to have the optimizer register in a separate file than the one that calls CLI? |
Beta Was this translation helpful? Give feedback.
Hi! I got it to work in the meantime. I added this to the main file where I call CLI:
The main issue was in the config file---apparently one needs to write:
instead of:
Doing the former, I got it to work.
By the way, is there a way to have the optimizer register in a separate file than the one that calls CLI?