Density Independent Self-organized Support for Q-Value Function Interpolation in Reinforcement Learning

Antonin Calba (1, 2) , Alain Dutech (3, 4, 2) , Jérémy Fix (3, 5, 2)

1 UL - Université de Lorraine
2 BISCUIT - Bio-Inspired, Situated and Cellular Unconventional Information Technologies
3 LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
4 Inria Nancy - Grand Est
5 CentraleSupélec [campus de Metz]

In this paper, we propose a contribution in the field of Reinforcement Learning (RL) with continuous state space. Our work is along the line of previous works involving a vector quantization algorithm for learning the state space representation on top of which a function approximation takes place. In particular, our contribution compares the performances of the Kohonen SOM and the Rougier DSOM with the Göppert function approximation scheme on both the mountain car problem. We give a particular focus to DSOM as it is less sensitive to the density of inputs and opens interesting perspectives in RL.

Domaines: Intelligence artificielle [cs.AI]

Antonin Calba, Alain Dutech, Jérémy Fix. Density Independent Self-organized Support for Q-Value Function Interpolation in Reinforcement Learning. 29th European Symposium on Artificial Neural Networks, Oct 2021, Bruges/Online, Belgium. ⟨10.14428/esann/2021.es2021-62⟩. ⟨hal-03550442⟩