FusedAdam vs ADOPT

[s-smits]

Hi Wing,
Quick question - between ADOPT and FusedAdam, which performs better on A100/H100s in your experience? I know the wall-clock winner comes down to:
FusedAdam's faster steps (usually 1.5-2x)
ADOPT needing fewer steps to converge (50, even 80% less?)

Have you tested this comparison? I'm leaning towards using ADOPT.