Bad step rejection

[mh510]

Hi Dylan, thanks for sharing. One question regarding your implementation; why are you rejecting bad steps or what's your motivation for that? I guess that the original Adam paper is not suggesting it.

[DylanMuir]

You're right, it's not suggested. I found it to empirically work better for my problems, which is why it's in the code base.

I had intended to use an option to switch this feature on and off, but didn't have time to get that working with the options system. If you manage to get that working then please submit a pull request!