Almost sure convergence of stochastic gradient processes with matrix step sizes

Jean-Marie Monnez (1)

1 IECN - Institut Élie Cartan de Nancy

We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size a_{n} is replaced by a random matrix A_{n}: X_{n+1}=X_{n}-A_{n}∇g(X_{n})-A_{n}V_{n}. We give two theorems of almost sure convergence in the case where the equation ∇g=0 has a set of solutions.

Mots clés: Stochastic approximation; stochastic gradient

Domaines: Statistiques [math.ST]

Jean-Marie Monnez. Almost sure convergence of stochastic gradient processes with matrix step sizes. Statistics and Probability Letters, 2006, 76, pp.531-536. ⟨hal-00094713⟩