Compressive Gaussian Mixture Estimation by Orthogonal Matching Pursuit with Replacement

Nicolas Keriven (1) , Rémi Gribonval (1)

1 PANAMA - Parcimonie et Nouveaux Algorithmes pour le Signal et la Modélisation Audio

This work deals with the problem of fitting a Gaussian Mixture Model (GMM) to a large collection of data. Usual approaches such as the classical Expectation Maximization (EM) algorithm are known to perform well but require extensive access to the data. The proposed method compresses the entire database into a single low-dimensional sketch that can be computed in one pass then directly used for GMM estimation. This sketch can be seen as resulting from the application of a linear operator to the underlying probability distribution, thus establishing a connection between our method and generalized compressive sensing. In particular, the new algorithms introduced to estimate GMMs are similar to usual greedy algorithms in compressive sensing.

Mots clés: Compressed Sensing; database sketch; compressed learning; Gaussian mixture

Domaines: Machine Learning [stat.ML]; Traitement du signal et de l'image [eess.SP]; Probabilités [math.PR]

Nicolas Keriven, Rémi Gribonval. Compressive Gaussian Mixture Estimation by Orthogonal Matching Pursuit with Replacement. SPARS 2015, Jul 2015, Cambridge, United Kingdom. ⟨hal-01165984⟩