A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression

Luis Briceño-Arias (1) , Giovanni Chierchia (2) , Emilie Chouzenoux (2, 3) , Jean-Christophe Pesquet (3, 4)

1 CMM - Centre de modélisation mathématique / Centro de Modelamiento Matemático [Santiago]
2 LIGM - Laboratoire d'Informatique Gaspard-Monge
3 CVN - Centre de vision numérique
4 OPIS - OPtimisation Imagerie et Santé

In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.

Mots clés: Proximity operator; Douglas-Rachford splitting; Block-coordinate descent; Logistic regression

Domaines: Optimisation et contrôle [math.OC]; Intelligence artificielle [cs.AI]

Luis Briceño-Arias, Giovanni Chierchia, Emilie Chouzenoux, Jean-Christophe Pesquet. A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression. Computational Optimization and Applications, 2019, 72 (3), pp.707-726. ⟨10.1007/s10589-019-00060-6⟩. ⟨hal-01672507⟩