Trainer flags: amp_level vs. precision

[dianemarquette]

Hi!

I'm a bit confused regarding the trainer's flags. According to Lightning's documentation, the default settings are:

• amp_level='O2' (i.e., "Almost FP16" Mixed Precision, cf. Nvidia documentation)
• precision=32

Isn't it a bit contradictory ? Is the default training mode full precision or mixed precision?

Thanks in advance for your clarification :)

[tchaton]

Dear @dianemarquette,

precision=32 is the default one. However, if you turn on precision=16 and set amp_backend="apex", then amp_level=02 is considered as the default one.

Best,
T.C

[dianemarquette]

Thanks @tchaton for the quick and clear answer :)