Slice inverse regression with score functions

Dmitry Babichev (1, 2) , Francis Bach (1, 2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 LIENS - Laboratoire d'informatique de l'école normale supérieure

We consider non-linear regression problems where we assume that the response depends non-linearly on a linear projection of the covariates. We propose score function extensions to sliced inverse regression problems, both for the first-order and second-order score functions. We show that they provably improve estimation in the population case over the non-sliced versions and we study finite sample estimators and their consistency given the exact score functions. We also propose to learn the score function as well, in two steps, i.e., first learning the score function and then learning the effective dimension reduction space, or directly, by solving a convex optimization problem regularized by the nuclear norm. We illustrate our results on a series of experiments.

Mots clés: projection pursuit; dimension reduction; nonlinear regression; slice inverse regression.

Domaines: Machine Learning [stat.ML]; Théorie [stat.TH]

Dmitry Babichev, Francis Bach. Slice inverse regression with score functions. 2016. ⟨hal-01388498v1⟩