Audio-Motor Integration for Robot Audition

Antoine Deleforge (1) , Alexander Schmidt (2) , Walter Kellermann (2)

1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
2 FAU - Friedrich-Alexander Universität Erlangen-Nürnberg = University of Erlangen-Nuremberg

In the context of robotics, audio signal processing in the wild amounts to dealing with sounds recorded by a system that moves and whose actuators produce noise. This creates additional challenges in sound source localization, signal enhancement and recognition. But the speci-ficity of such platforms also brings interesting opportunities: can information about the robot actuators' states be meaningfully integrated in the audio processing pipeline to improve performance and efficiency? While robot audition grew to become an established field, methods that explicitly use motor-state information as a complementary modality to audio are scarcer. This chapter proposes a unified view of this endeavour, referred to as audio-motor integration. A literature review and two learning-based methods for audio-motor integration in robot audition are presented, with application to single-microphone sound source localization and ego-noise reduction on real data.

Mots clés: audio-motor integration; robot audition; sound localization; ego-noise reduction; dictionary learning; single-channel; multi-channel

Domaines: Traitement du signal et de l'image [eess.SP]; Intelligence artificielle [cs.AI]; Robotique [cs.RO]

Antoine Deleforge, Alexander Schmidt, Walter Kellermann. Audio-Motor Integration for Robot Audition. Multimodal Behavior Analysis in the Wild, Academic Press, pp.1-27, 2018. ⟨hal-01929388⟩