Learning in Games with Lossy Feedback

Zhengyuan Zhou (1) , Panayotis Mertikopoulos (2) , Susan Athey (1) , Nicholas Bambos (3) , Peter Glynn (4) , Yinyu Ye (4)

1 Department of Electrical Engineering [Stanford]
2 POLARIS - Performance analysis and optimization of LARge Infrastructures and Systems
3 Computer Science Department [Stanford]
4 Department of Management Science and Engineering [Stanford]

We consider a game-theoretical multi-agent learning problem where the feedback information can be lost during the learning process and rewards are given by a broad class of games known as variationally stable games. We propose a simple variant of the classical online gradient descent algorithm, called reweighted online gradient descent (ROGD) and show that in variationally stable games, if each agent adopts ROGD, then almost sure convergence to the set of Nash equilibria is guaranteed, even when the feedback loss is asynchronous and arbitrarily corrrelated among agents. We then extend the framework to deal with unknown feedback loss probabilities by using an estimator (constructed from past data) in its replacement. Finally, we further extend the framework to accomodate both asynchronous loss and stochastic rewards and establish that multi-agent ROGD learning still converges to the set of Nash equilibria in such settings. Together, these results contribute to the broad lanscape of multi-agent online learning by significantly relaxing the feedback information that is required to achieve desirable outcomes.

Domaines: Optimisation et contrôle [math.OC]

Zhengyuan Zhou, Panayotis Mertikopoulos, Susan Athey, Nicholas Bambos, Peter Glynn, et al.. Learning in Games with Lossy Feedback. NIPS 2018 - Thirty-second Conference on Neural Information Processing Systems, Dec 2018, Montreal, Canada. pp.1-11. ⟨hal-01904461⟩