Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR

Felix Weninger (1) , Hakan Erdogan (2, 3) , Shinji Watanabe (2) , Emmanuel Vincent (4) , Jonathan Le Roux (2) , John R. Hershey (2) , Björn Schuller (5)

1 TUM - Technische Universität Munchen - Technical University Munich - Université Technique de Munich
2 MERL - Mitsubishi Electric Research Laboratories
3 Sabanci University [Istanbul]
4 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
5 Imperial College London

We evaluate some recent developments in recurrent neural network (RNN) based speech enhancement in the light of noise-robust automatic speech recognition (ASR). The proposed framework is based on Long Short-Term Memory (LSTM) RNNs which are discriminatively trained according to an optimal speech reconstruction objective. We demonstrate that LSTM speech enhancement, even when used ' na¨vely ' as front-end processing, delivers competitive results on the CHiME-2 speech recognition task. Furthermore, simple, feature-level fusion based extensions to the framework are proposed to improve the integration with the ASR back-end. These yield a best result of 13.76 % average word error rate, which is, to our knowledge, the best score to date.

Domaines: Traitement du signal et de l'image [eess.SP]

Felix Weninger, Hakan Erdogan, Shinji Watanabe, Emmanuel Vincent, Jonathan Le Roux, et al.. Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. 12th International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA), Aug 2015, Liberec, Czech Republic. ⟨hal-01163493⟩