Detecting and counting overlapping speakers in distant speech scenarios

Samuele Cornell (1) , Maurizio Omologo (2) , Stefano Squartini (1) , Emmanuel Vincent (3)

1 UNIVPM - Polytechnic University of Marche [Ancona, Italy] / Università Politecnica delle Marche [Ancona, Italia]
2 FBK - Fondazione Bruno Kessler [Trento, Italy]
3 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication

We consider the problem of detecting the activity and counting overlapping speakers in distant-microphone recordings. We treat supervised Voice Activity Detection (VAD), Overlapped Speech Detection (OSD), joint VAD+OSD, and speaker counting as instances of a general Overlapped Speech Detection and Counting (OSDC) task, and we design a Temporal Convolu-tional Network (TCN) based method to address it. We show that TCNs significantly outperform state-of-the-art methods on two real-world distant speech datasets. In particular our best architecture obtains, for OSD, 29.1% and 25.5% absolute improvement in Average Precision over previous techniques on, respectively, the AMI and CHiME-6 datasets. Furthermore, we find that generalization for joint VAD+OSD improves by using a speaker counting objective rather than a VAD+OSD objective. We also study the effectiveness of forced alignment based labeling and data augmentation, and show that both can improve OSD performance.

Mots clés: overlapped speech detection; speaker counting; distant speech; forced alignment

Domaines: Traitement du signal et de l'image [eess.SP]

Samuele Cornell, Maurizio Omologo, Stefano Squartini, Emmanuel Vincent. Detecting and counting overlapping speakers in distant speech scenarios. INTERSPEECH 2020, Oct 2020, Shanghai, China. ⟨hal-02908241v2⟩