mixed accuracy

[mactavish91]

If I want to conduct mixed accuracy training with little impact on accuracy, which parameter should I use in trainer? precision=32, amp_backend="native" or precision=16, amp_backend="native"？When the precision is set to 32, will mixed accuracy training be performed by default?

[akihironitta]

Trainer(precision=32) is the default and it means no automatic mixed precision (amp) training is performed. You can enable it with Trainer(precision=16).

Also, you don't need to set amp_backend unless you know what the flag does.

See these guides for more:

• https://pytorch-lightning.readthedocs.io/en/1.7.7/guides/speed.html?highlight=automatic%20mixed%20precision#speed-amp
• https://pytorch-lightning.readthedocs.io/en/1.7.7/common/trainer.html#precision