Don't run validation/profiler/weights_summary when tuning?

[austinmw]

Hi, I'm using the learning rate finder via:

trainer = Trainer(auto_lr_find=True, profiler='simple', weights_summary='full')
trainer.tune(model)
trainer.fit(model)

However the tuning step runs validation, profiler, and weights summary. I'd prefer all of these steps to not run until the fit call. Is there an easy way to do this?

[austinmw]

Okay if I bump this?

[awaelchli]

You need to set profiler=None and weights_summary=None, run the tuning step separately. Once you found a good learning rate, switch the tuner off and use the other args.
A better experience is probably to set up argument parsing to quickly set flags and defaults from the command line:

A simple argparse.ArgumentParser example:
https://pytorch-lightning.readthedocs.io/en/latest/common/hyperparameters.html#argparser-best-practices

[austinmw]

Hi, what do you mean by switch the tuner off? Do I need to recreate the trainer object? Is there a way to run one after another as a single run instead of rerunning without then with these args?

Is this what you mean?

trainer = Trainer(auto_lr_find=True, profiler=None, weights_summary=None)
trainer.tune(model)
trainer = Trainer(auto_lr_find=False, profiler='simple', weights_summary='full')
trainer.fit(model)

I have argparse set up to run from SageMaker, but was hoping I could execute a single training session that first selected a learning rate, then printed weights summary, then trained, then printed profiler info.

[awaelchli]

Yes, this should work. If you have argparse setup you can do a small improvement:

# this will override the defaults:
trainer = Trainer.from_argparse_args(parser, auto_lr_find=True, profiler=None, weights_summary=None)
trainer.tune(model)

trainer = Trainer.from_argparse_args(parser, profiler='simple', weights_summary='full')
trainer.fit(model)

Our tuning integration at the moment is very simple. It's not designed to be highly customizable, but we try to set good defaults that cover most users.

But feature requests with improvements are always welcome.