fp16

[Qe42]

I have successfully used Multifit for a while now. Thanks for sharing!

I would like to use Multifit with fp16.

I used a pretrained language model (paperversion fp32) and followed the example in the notebook MLDoc-JA-multifit_fp16.ipynb. The training results did not make any sense (very low).

Is it necessary to pretrain a new language model on wikipedia data using fp16 or are there some cuda settings that need to be changed?

Thanks for the clarification.