Approximation Bounds for Sparse Programs

Armin Askari (1) , Alexandre d'Aspremont (2, 3, 4, 5) , Laurent El Ghaoui (1)

1 UC Berkeley - University of California [Berkeley]
2 LIENS - Laboratoire d'informatique de l'école normale supérieure
3 CNRS - Centre National de la Recherche Scientifique
4 SIERRA - Statistical Machine Learning and Parsimony
5 PSL - Université Paris sciences et lettres

We show that sparsity constrained optimization problems over low dimensional spaces tend to have a small duality gap. We use the Shapley-Folkman theorem to derive both data-driven bounds on the duality gap, and an efficient primalization procedure to recover feasible points satisfying these bounds. These error bounds are proportional to the rate of growth of the objective with the target cardinality, which means in particular that the relaxation is nearly tight as soon as the target cardinality is large enough so that only uninformative features are added.

Mots clés: Convex Relaxation; Sparsity; Duality Gap; Shapley-Folkman theorem

Domaines: Machine Learning [stat.ML]; Recherche opérationnelle [math.OC]

Armin Askari, Alexandre d'Aspremont, Laurent El Ghaoui. Approximation Bounds for Sparse Programs. SIAM Journal on Mathematics of Data Science, 2022, 4 (2), pp.514-530. ⟨10.1137/21M1398677⟩. ⟨hal-03165622⟩