Bayesian Non-Negative Matrix Factorization

Pierre Alquier (1) , Benjamin Guedj (2, 3)

1 CREST - Centre de Recherche en Économie et Statistique
2 MODAL - MOdel for Data Analysis and Learning
3 Inria Lille - Nord Europe

The aim of this paper is to provide some theoretical understanding of Bayesian non-negative matrix factorization methods, along with practical implementations. We provide a sharp oracle inequality for a quasi-Bayesian estimator, also known as the exponentially weighted aggregate (Dalalyan and Tsybakov, 2008). This result holds for a very general class of prior distributions and shows how the prior affects the rate of convergence. We then discuss possible algorithms. A natural choice in Bayesian statistics is the Gibbs sampler, used for example in Salakhutdinov and Mnih (2008). This algorithm is asymptotically exact , yet it suffers from the fact that the convergence might be very slow on large datasets. When faced with massive datasets, a more efficient path is to use approximate methods based on optimisation algorithms: we here describe a blockwise gradient descent which is a Bayesian version of the algorithm in Xu et al. (2012). Here again, the general form of the algorithm helps to understand the role of the prior, and some priors will clearly lead to more efficient (i.e., faster) implementations. We end the paper with a short simulation study and an application to finance. These numerical studies support our claim that the reconstruction of the matrix is usually not very sensitive to the choice of the hyperparameters whereas rank identification is.

Mots clés: Gibbs sampler; Non-negative matrix factorization; PAC-Bayesian theory

Domaines: Machine Learning [stat.ML]; Théorie [stat.TH]; Statistiques [math.ST]

Pierre Alquier, Benjamin Guedj. Bayesian Non-Negative Matrix Factorization. 2016. ⟨hal-01251878v1⟩