Setting 16-bit Precision causes OOM

[aleSuglia]

Hi there,

I'm using PL to train a BERT-based model. However, I have the impression that enabling 16-bit precision in my trainer is not reducing the memory footprint or improving the speed. Actually, 32-bit precision trainer doesn't go OOM however, when I switch to 16-bit, I go OOM. Could you please advise as to how to benefit from this advanced feature?