A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEs

Benoît Bonnet (1, 2) , Cristina Cipriani (3) , Massimo Fornasier (3) , Hui Huang (4)

1 CaGE - Control And GEometry
2 LJLL (UMR_7598) - Laboratoire Jacques-Louis Lions
3 TUM - Technische Universität Munchen - Technical University Munich - Université Technique de Munich
4 University of Calgary

In this paper we consider a measure-theoretical formulation of the training of NeurODEs in the form of a mean-field optimal control with L 2-regularization of the control. We derive first order optimality conditions for the NeurODE training problem in the form of a mean-field maximum principle, and show that it admits a unique control solution, which is Lipschitz continuous in time. As a consequence of this uniqueness property, the mean-field maximum principle also provides a strong quantitative generalization error for finite sample approximations. Our derivation of the mean-field maximum principle is much simpler than the ones currently available in the literature for mean-field optimal control problems, and is based on a generalized Lagrange multiplier theorem on convex sets of spaces of measures. The latter is also new, and can be considered as a result of independent interest.

Mots clés: NeurODEs; Mean-Field Optimal Control; Mean-Field Maximum Principle; Lagrange Multiplier Theorem

Domaines: Optimisation et contrôle [math.OC]; Automatique

Benoît Bonnet, Cristina Cipriani, Massimo Fornasier, Hui Huang. A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEs. 2021. ⟨hal-03289521v1⟩