Understanding cfg.SOLVER.STEPS & cfg.SOLVER.GAMMA

[jwscot]

My understanding was as follows:

• Gamma is a number by which the learning rate will decrease after the number of iterations you set cfg.SOLVER.STEPS to.

However, I have seen in some tutorials a number smaller than the basal learning rate is used for gamma. Does this mean that gamma is actually a factor by which we decrease the learning rate by after a set number of iterations?

For example, if my understanding is correct, then by setting:

cfg.SOLVER.BASE_LR = 0.001
cfg.SOLVER.GAMMA = 0.5
cfg.SOLVER.STEPS = (2500, 3500)

Then the learning rate will reduce by 50% at the 2500th and 3500th iteration?

Thanks!

[thulard]

This is correct.
BASE_LR will be multiply by GAMMA at the provided STEPS (by default GAMMA is set to 0.1 but could be modify by your per-trained model config)