Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization

Mark Schmidt (1, 2) , Nicolas Le Roux (1, 2) , Francis Bach (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 LIENS - Laboratoire d'informatique de l'école normale supérieure

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using proximal-gradient methods, where an error is present in the calculation of the gradient of the smooth term or in the proximity operator with respect to the non-smooth term. We show that both the basic proximal-gradient method and the accelerated proximal-gradient method achieve the same convergence rate as in the error-free case, provided that the errors decrease at appropriate rates.Using these rates, we perform as well as or better than a carefully chosen fixed error level on a set of structured sparsity problems.

Mots clés: optimization; proximal method; convexity; strong convexity; accelerated method; convergence rate

Domaines: Apprentissage [cs.LG]; Optimisation et contrôle [math.OC]

Mark Schmidt, Nicolas Le Roux, Francis Bach. Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization. NIPS'11 - 25 th Annual Conference on Neural Information Processing Systems, Dec 2011, Grenada, Spain. ⟨inria-00618152v3⟩