Multimodal Conversational Interaction with a Humanoid Robot

Adam Csapo (1) , Emer Gilmartin (2) , Jonathan Grizou (3) , Jingguang Han (2) , Raveesh Meena (4) , Dimitra Anastasiou (5) , Kristiina Jokinen (6) , Graham Wilcock (6)

1 BME - Budapest University of Technology and Economics [Budapest]
2 Trinity College Dublin
3 Flowers - Flowing Epigenetic Robots and Systems
4 KTH - KTH Royal Institute of Technology [Stockholm]
5 Universität Bremen
6 Helsingin yliopisto = Helsingfors universitet = University of Helsinki

The paper presents a multimodal conversational interaction system for the Nao humanoid robot. The system was developed at the 8th International Summer Workshop on Multi- modal Interfaces, Metz, 2012. We implemented WikiTalk, an existing spoken dialogue system for open-domain conversations, on Nao. This greatly extended the robot's interaction capabilities by enabling Nao to talk about an unlimited range of topics. In addition to speech interaction, we developed a wide range of multimodal interactive behaviours by the robot, including face- tracking, nodding, communicative gesturing, proximity detection and tactile interrupts. We made video recordings of user interac- tions and used questionnaires to evaluate the system. We further extended the robot's capabilities by linking Nao with Kinect.

Domaines: Robotique [cs.RO]

Adam Csapo, Emer Gilmartin, Jonathan Grizou, Jingguang Han, Raveesh Meena, et al.. Multimodal Conversational Interaction with a Humanoid Robot. CogInfoCom 2012 - 3rd IEEE International Conference on Cognitive Infocommunications, Dec 2012, Kosice, Slovakia. pp.667-672, ⟨10.1109/CogInfoCom.2012.6421935⟩. ⟨hal-00759810⟩