Feedback from a serious user

[ablaom]

I received some feedback from a valued user which I thought might be useful to post here for the record:

I don't remember having many problems with IteratedModel, so I was able to do all I wanted with little effort. It provides a lot of options off-the-shelf in an intuitive way. I only needed to spend some time thinking about two things:

• How to do learning-rate (LR) scheduling through this feature. A complication is that the Adam type is immutable, so whenever I wanted to update the LR, I had to get its value, alter it, and create a new Adam instance with it.
• The difference between "control cycles" and Step(n) and how they relate to training batches and epochs when using NNs. Understanding all these "progress measures" is needed to define how often the LR will be updated. When using decay, if updates are too frequent, the LR vanishes too fast and training halts