Training Dialogue Systems With Human Advice

Merwan Barlier (1) , Romain Laroche (2) , Olivier Pietquin (3)

1 SEQUEL - Sequential Learning
2 Microsoft Research
3 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189

One major drawback of Reinforcement Learning (RL) Spoken Dialogue Systems is that they inherit from the general exploration requirements of RL which makes them hard to deploy from an industry perspective. On the other hand, industrial systems rely on human expertise and hand written rules so as to avoid irrelevant behavior to happen and maintain acceptable experience from the user point of view. In this paper, we attempt to bridge the gap between those two worlds by providing an easy way to incorporate all kinds of human expertise in the training phase of a Reinforcement Learning Dialogue System. Our approach, based on the TAMER framework, enables safe and efficient policy learning by combining the traditional Reinforcement Learning reward signal with an additional reward, encoding expert advice. Experimental results show that our method leads to substantial improvements over more traditional Reinforcement Learning methods.

Mots clés: observation); communication; Learning agent capabilities (agent models; Human-robot/agent interaction

Domaines: Intelligence artificielle [cs.AI]; Apprentissage [cs.LG]

Merwan Barlier, Romain Laroche, Olivier Pietquin. Training Dialogue Systems With Human Advice. AAMAS 2018 - the 17th International Conference on Autonomous Agents and Multiagent Systems, Jul 2018, Stockholm, Sweden. pp.9. ⟨hal-01945831⟩