Sebbouh

Othmane Sebbouh is a Data Science Master’s student at ENSAE & Ecole polytechnique. The topic of his internship at the DSAIDIS chair is “Towards closing the gap between the theory and practice of stochastic variance reduced methods”. SVRG is an inner-outer loop based method, where in the outer loop a reference full gradient is evaluated, after which m steps of an inner loop are executed where the reference gradient is used to build a variance reduced estimate of the current gradient. The simplicity of the SVRG method and its analysis has lead to multiple extensions and variants for even non-convex optimization. Yet there is a significant gap between what parameter settings the analysis suggests and what is known to work well in practice, which is why Othman’s work will be to take several steps towards closing this gap.