Recursion error when trying to use a LR scheduler with the CLI

[alzaia]

Hello, I am using the CLI with a standard config.yaml file for my training params. My config.yaml overall structure looks like this and works fine:

trainer:
...
data:
...
model:
...
optimizer:
...

However, when I add a scheduler (following the Lightning guidelines and examples), for example:

lr_scheduler:
  class_path: torch.optim.lr_scheduler.ExponentialLR
  init_args:
    gamma: 0.1

It gives me a recursion error:

  File ".../lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 570, in __init__
    super(ExponentialLR, self).__init__(optimizer, last_epoch, verbose)
  File ".../lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 570, in __init__
    super(ExponentialLR, self).__init__(optimizer, last_epoch, verbose)
  File ".../lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 570, in __init__
    super(ExponentialLR, self).__init__(optimizer, last_epoch, verbose)
  [Previous line repeated 997 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

Any idea what I'm doing wrong here?

[mauvilsa]

There is not enough information to tell. You could try to enable debugging of the CLI (see #how-do-i-troubleshoot-a-cli) and share the full stack trace. Though, I am not sure if the problem is related to the CLI. Why would the __init__ call itself? The only thing that comes to mind is a weird patch/hack of the ExponentialLR class.

[alzaia]

Hello @mauvilsa , I just tried it with another scheduler:

lr_scheduler:
  class_path: torch.optim.lr_scheduler.ReduceLROnPlateau
  init_args:
    factor: 0.1
    mode: min
    patience: 5
    min_lr: 0.00001

And it gives me another error this time:

usage: main.py [-h] [-c CONFIG] [--print_config [={comments,skip_null,skip_default}+]]
               {fit,validate,test,predict,tune} ...
main.py: error: Configuration check failed :: Parser key "lr_scheduler": Value "{'class_path': 'torch.optim.lr_scheduler.ReduceLROnPlateau', 'init_args': {'factor': 0.1, 'mode': 'min', 'patience': 5, 'min_lr': 1e-05}}" does not validate against any of the types in typing.Union[torch.optim.lr_scheduler._LRScheduler, pytorch_lightning.cli.ReduceLROnPlateau]:
  - "torch.optim.lr_scheduler.ReduceLROnPlateau" is not a subclass of <class 'torch.optim.lr_scheduler._LRScheduler'>
  - "torch.optim.lr_scheduler.ReduceLROnPlateau" is not a subclass of <class 'pytorch_lightning.cli.ReduceLROnPlateau'>

[mauvilsa]

For ReduceLROnPlateau you need to use pytorch_lightning.cli.ReduceLROnPlateau which is what the error is telling you. torch.optim.lr_scheduler.ReduceLROnPlateau is not like the other schedulers since it does not inherit from torch.optim.lr_scheduler._LRScheduler. This has no relation to the recursion error you asked about before.