Robust ASR using neural network based speech enhancement and feature simulation

Sunit Sivasankaran (1) , Aditya A Nugraha (1) , Emmanuel Vincent (1) , Juan Andrés Morales Cordovilla (1) , Siddharth Dalmia (1) , Irina Illina (1) , Antoine Liutkus (2)

1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
2 PAROLE - Analysis, perception and recognition of speech

We consider the problem of robust automatic speech recognition (ASR) in the context of the CHiME-3 Challenge. The proposed system combines three contributions. First, we propose a deep neural network (DNN) based multichannel speech enhancement technique, where the speech and noise spectra are estimated using a DNN based regressor and the spatial parameters are derived in an expectation-maximization (EM) like fashion. Second, a conditional restricted Boltz-mann machine (CRBM) model is trained using the obtained enhanced speech and used to generate simulated training and development datasets. The goal is to increase the similarity between simulated and real data, so as to increase the benefit of multicondition training. Finally, we make some changes to the ASR backend. Our system ranked 4th among 25 entries

Mots clés: CHiME-3; CRBM; feature simulation; speech enhancement; ASR

Domaines: Apprentissage [cs.LG]; Son [cs.SD]

Sunit Sivasankaran, Aditya A Nugraha, Emmanuel Vincent, Juan Andrés Morales Cordovilla, Siddharth Dalmia, et al.. Robust ASR using neural network based speech enhancement and feature simulation. ASRU, Dec 2015, Arizona, United States. ⟨hal-01204553⟩