Inertial Newton Algorithms Avoiding Strict Saddle Points

Camille Castera (1)

1 IRIT-SC - Signal et Communications

We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict saddle points. We also evidence the role played by the hyper-parameters of these methods in their qualitative behavior near critical points. The theoretical results are supported by numerical illustrations.

Mots clés: non-convex optimization; optimization; machine learning; algorithms; stable manifold

Domaines: Optimisation et contrôle [math.OC]; Apprentissage [cs.LG]

Camille Castera. Inertial Newton Algorithms Avoiding Strict Saddle Points. 2021. ⟨hal-03433202⟩