Intrinsic Motivation for Autonomous Mental Development

Pierre-Yves Oudeyer (1, 2, 3) , Frédéric Kaplan (3) , Véréna Hafner (3)

1 Flowers - Flowing Epigenetic Robots and Systems
2 ENSTA Paris - École Nationale Supérieure de Techniques Avancées
3 SONY CSL-Paris - Sony Computer Science Laboratory Paris

Exploratory activities seem to be intrinsically re- warding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Exper- imental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology.

Mots clés: values; Active learning; autonomy; behavior; complexity; curiosity; development; developmental trajectory; epigenetic robotics; intrinsic motivation; learning; reinforcement learning; values.

Domaines: Robotique [cs.RO]

Pierre-Yves Oudeyer, Frédéric Kaplan, Véréna Hafner. Intrinsic Motivation for Autonomous Mental Development. IEEE Transactions on Evolutionary Computation, 2007, 11 (2), pp.265-286. ⟨10.1109/TEVC.2006.890271⟩. ⟨hal-00793610⟩