References Optimization June 5th - Convexity and optmization http://www.princeton.edu/~amirali/Public/Teaching/ORF523/S16/ORF523_S16_Lec7_gh.pdf June 6th/7th - Gradient Descent and Stochastic Gradient Descent Understanding Machine Learning - Shai Ben-David & Shai Shalev-Schwartz June 13th/14th - Stochastic Average Gradient (SAG) and SVRG https://arxiv.org/pdf/1309.2388.pdf http://www.cs.ubc.ca/~schmidtm/Software/SAG.html https://papers.nips.cc/paper/4937-accelerating-stochastic-gradient-descent-using-predictive-variance-reduction.pdf http://proceedings.mlr.press/v32/defazio14.pdf https://arxiv.org/pdf/1407.0202.pdf