Error Bounds for Piecewise Smooth and Switching Regression

Fabien Lauer (1)

1 ABC - Machine Learning and Computational Biology

The paper deals with regression problems, in which the nonsmooth target is assumed to switch between different operating modes. Specifically, piecewise smooth (PWS) regression considers target functions switching deterministically via a partition of the input space, while switching regression considers arbitrary switching laws. The paper derives generalization error bounds in these two settings by following the approach based on Rademacher complexities. For PWS regression, our derivation involves a chaining argument and a decomposition of the covering numbers of PWS classes in terms of the ones of their component function classes and the capacity of the classifier partitioning the input space. This yields error bounds with a radical dependency on the number of modes. For switching regression, the decomposition can be performed directly at the level of the Rademacher complexities, which yields bounds with a linear dependency on the number of modes. By using once more chaining and a decomposition at the level of covering numbers, we show how to recover a radical dependency. Examples of applications are given in particular for PWS and swichting regression with linear and kernel-based component functions.

Mots clés: Chaining; Rademacher complexity; Covering number; Learning theory; Regression

Domaines: Machine Learning [stat.ML]; Apprentissage [cs.LG]

Fabien Lauer. Error Bounds for Piecewise Smooth and Switching Regression. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31 (4), pp.1183-1195. ⟨10.1109/TNNLS.2019.2919444⟩. ⟨hal-01566136v3⟩