Dynamic thresholding percentile

[oconnoob]

imagen-pytorch/imagen_pytorch/imagen_pytorch.py

Line 1696 in 7d31add

dynamic_thresholding_percentile = 0.9, # unsure what this was based on perusal of paper

Explanation here

[jacobwjs]

It seems to have quite an impact on my results btw. Something to take into consideration before any serious training.

[oconnoob]

@jacobwjs thanks for that info - I can definitely see how it would have a big effect on training. Have you found the value of 0.9 to be optimal? How far away from this value did you have to go before results got poor?

Or do you mean dynamic thresholding in general is important for good results

[jacobwjs]

@oconnoob That's something I've spent little time rigorously flushing out to be honest. Best would be to do a sweep over various values. The few that I've tested align with the paper (of course).

• static thresholding definitely leads to higher contrast
• dynamic thresholding with Imagen seems (visually) to perform better with higher values (e.g. 0.95 or higher)

I recommend dynamic thresholding, but choosing values on the higher end.

I've only done a few tests, but would love to hear your results if you do as well.