Modifying Eq 12/13 to deal with negative loss

[MohitLamba94]

Hi,
Thanks for this great work. In Eq. 12/13 you have a log(variance) that might go negative for variance<1. Assuming, variance to be always +ve can we modify the Eq. 12/13 to log(1+variance) to prevent negative loss and allowing better optimization?
Will this modification hamper training or net output during inference?

[mattpoggi]

Hi,
I never tried, because the negative loss never gave issues in my experiments.
You can use sigmoids over the outputs, but I think your solution should work as well.