Crowd against the machine: A simulation-based benchmark tool to evaluate and compare robot capabilities to navigate a human crowd

Fabien Grzeskowiak (1) , David Gonon (2) , Daniel Dugas (3) , Diego Paez-Granados (2) , Jen Jen Chung (3) , Juan Nieto (3) , Roland Siegwart (3) , Aude Billard (2) , Marie Babel (2) , Julien Pettré (2)

1 RAINBOW - Sensor-based and interactive robotics
2 EPFL - Ecole Polytechnique Fédérale de Lausanne
3 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]

The evaluation of robot capabilities to navigate human crowds is essential to conceive new robots intended to operate in public spaces. This paper initiates the development of a benchmark tool to evaluate such capabilities; our long term vision is to provide the community with a simulation tool that generates virtual crowded environment to test robots, to establish standard scenarios and metrics to evaluate navigation techniques in terms of safety and efficiency, and thus, to install new methods to benchmarking robots' crowd navigation capabilities. This paper presents the architecture of the simulation tools, introduces first scenarios and evaluation metrics, as well as early results to demonstrate that our solution is relevant to be used as a benchmark tool.

Domaines: Robotique [cs.RO]

Fabien Grzeskowiak, David Gonon, Daniel Dugas, Diego Paez-Granados, Jen Jen Chung, et al.. Crowd against the machine: A simulation-based benchmark tool to evaluate and compare robot capabilities to navigate a human crowd. ICRA 2021 - IEEE International Conference on Robotics and Automation, May 2021, Xian, China. pp.3879-3885. ⟨hal-03206221⟩